2024-04-02T16:48:23.181 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T16:48:23.181 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/orig.config.yaml 2024-04-02T16:48:23.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T16:48:23.339 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741 branch: wip-batrick-testing-20240402.004512 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7635741' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0PBAFqV/Q22FEAfo6IvUERCi2g8DxBjtVmbyEoVG7xzNjjbh67HeZUODTlnjDs3icS9VS/LMa1QCtyUwxPKKQ= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+/LCXaxbHmi1QtlK4diCXeI/IX5gDL6Yy5CK5vt8JAPhdyq4EI2J0W71d/5hyVwo02vwVVRhNL2+aWFP09Uxs= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGuWeq6VyF3aYT05Vlzd42xoDvvsas+dXzRTTJMnkWUPDepYSRShSaPWtogStPbRk8aODJ+PVTtS5dPjqUaaLHA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T16:48:23.339 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T16:48:23.340 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T16:48:23.340 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T16:48:23.342 INFO:teuthology.task.internal:Checking packages... 2024-04-02T16:48:23.366 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T16:48:23.366 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T16:48:23.366 INFO:teuthology.packaging:ref: None 2024-04-02T16:48:23.366 INFO:teuthology.packaging:tag: None 2024-04-02T16:48:23.366 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T16:48:23.366 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:48:23.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T16:48:23.599 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T16:48:23.601 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T16:48:23.607 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T16:48:23.607 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T16:48:23.613 INFO:teuthology.task.internal:Saving configuration 2024-04-02T16:48:23.637 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T16:48:23.643 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T16:48:23.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.765305', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0PBAFqV/Q22FEAfo6IvUERCi2g8DxBjtVmbyEoVG7xzNjjbh67HeZUODTlnjDs3icS9VS/LMa1QCtyUwxPKKQ='} 2024-04-02T16:48:23.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.768812', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+/LCXaxbHmi1QtlK4diCXeI/IX5gDL6Yy5CK5vt8JAPhdyq4EI2J0W71d/5hyVwo02vwVVRhNL2+aWFP09Uxs='} 2024-04-02T16:48:23.720 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.767155', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGuWeq6VyF3aYT05Vlzd42xoDvvsas+dXzRTTJMnkWUPDepYSRShSaPWtogStPbRk8aODJ+PVTtS5dPjqUaaLHA='} 2024-04-02T16:48:23.720 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T16:48:23.726 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-02T16:48:23.726 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-02T16:48:23.727 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-02T16:48:23.727 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T16:48:23.824 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f78b8e4c160>, signals=[15]) 2024-04-02T16:48:23.824 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T16:48:23.830 INFO:teuthology.task.internal:Opening connections... 2024-04-02T16:48:23.830 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T16:48:23.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:48:23.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-04-02T16:48:23.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:48:24.027 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-02T16:48:24.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:48:24.114 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T16:48:24.121 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-04-02T16:48:24.140 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-04-02T16:48:24.140 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-04-02T16:48:24.198 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T16:48:24.199 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T16:48:24.199 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T16:48:24.199 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-04-02T16:48:24.222 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-04-02T16:48:24.239 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-04-02T16:48:24.239 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-04-02T16:48:24.298 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-04-02T16:48:24.298 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-04-02T16:48:24.298 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-04-02T16:48:24.298 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-04-02T16:48:24.298 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T16:48:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T16:48:24.299 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-04-02T16:48:24.322 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-02T16:48:24.339 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-02T16:48:24.339 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T16:48:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T16:48:24.397 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T16:48:24.397 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-02T16:48:24.421 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T16:48:24.434 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T16:48:24.440 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T16:48:24.440 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-04-02T16:48:24.443 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-04-02T16:48:24.445 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-02T16:48:24.460 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T16:48:24.466 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T16:48:24.466 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-04-02T16:48:24.500 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-04-02T16:48:24.502 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-02T16:48:24.713 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T16:48:24.790 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T16:48:24.804 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-02T16:48:24.804 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-02T16:48:24.804 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-02T16:48:24.804 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T16:48:24.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:48:24.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-02T16:48:25.022 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T16:48:25.022 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-02T16:48:25.022 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-02T16:48:25.040 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-361.el9.x86_64 2024-04-02T16:48:25.040 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T16:48:25.041 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-02T16:48:25.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum upgrade -y linux-firmware 2024-04-02T16:48:27.144 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-02T16:48:28.571 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-04-02T16:48:34.068 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:05 2024-04-02T16:48:36.044 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-02T16:48:41.020 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.1 MB 00:01 2024-04-02T16:48:42.537 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-02T16:48:43.849 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-02T16:48:49.105 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-02T16:48:51.568 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-02T16:48:51.580 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 1 Package 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 375 M 2024-04-02T16:48:51.581 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-04-02T16:49:01.489 INFO:teuthology.orchestra.run.smithi059.stdout:linux-firmware-20240219-143.el9.noarch.rpm 38 MB/s | 375 MB 00:09 2024-04-02T16:49:01.490 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:49:01.491 INFO:teuthology.orchestra.run.smithi059.stdout:Total 38 MB/s | 375 MB 00:09 2024-04-02T16:49:01.491 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-04-02T16:49:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-04-02T16:49:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-04-02T16:49:04.090 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-04-02T16:49:04.091 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-04-02T16:49:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-04-02T16:49:10.711 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-02T16:49:10.800 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T16:49:12.154 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T16:49:12.154 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-02T16:49:12.572 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-02T16:49:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:49:12.573 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-04-02T16:49:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-02T16:49:12.573 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:49:12.573 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-02T16:49:12.851 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-02T16:49:12.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-02T16:49:12.998 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-02T16:49:12.999 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-02T16:49:13.001 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi059.front.sepia.ceph.com to complete... 2024-04-02T16:49:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02 16:49:24 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-02T16:49:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:FINISHED --2024-04-02 16:49:24-- 2024-04-02T16:49:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:Total wall clock time: 12s 2024-04-02T16:49:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:Downloaded: 1 files, 321M in 11s (28.0 MB/s) 2024-04-02T16:49:24.862 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-02T16:49:24.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-02T16:49:26.890 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying... ######################################## 2024-04-02T16:49:26.948 INFO:teuthology.orchestra.run.smithi059.stdout:Preparing... ######################################## 2024-04-02T16:49:27.032 INFO:teuthology.orchestra.run.smithi059.stdout:Updating / installing... 2024-04-02T16:49:34.868 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-02T16:49:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T16:50:14.194 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-02T16:50:14.194 DEBUG:teuthology.orchestra.run.smithi059:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-02T16:50:14.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-02T16:50:14.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-02T16:50:14.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-02T16:50:14.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-02T16:50:14.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-02T16:50:14.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-02T16:50:14.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-02T16:50:14.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-02T16:50:14.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-02T16:50:14.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-02T16:50:14.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-02T16:50:14.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-02T16:50:14.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-02T16:50:14.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-02T16:50:14.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-02T16:50:14.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-02T16:50:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-02T16:50:14.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-02T16:50:14.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-02T16:50:14.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-02T16:50:14.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-02T16:50:14.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-02T16:50:14.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-02T16:50:14.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-02T16:50:14.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-02T16:50:14.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-02T16:50:14.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-02T16:50:14.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-02T16:50:14.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-02T16:50:14.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-02T16:50:14.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-02T16:50:14.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-02T16:50:14.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-02T16:50:14.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-02T16:50:14.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-02T16:50:14.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-02T16:50:14.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-02T16:50:14.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-02T16:50:14.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-02T16:50:14.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-02T16:50:14.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-02T16:50:14.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-02T16:50:14.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-02T16:50:14.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-02T16:50:14.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-02T16:50:14.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-02T16:50:14.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-02T16:50:14.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-02T16:50:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-02T16:50:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-02T16:50:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-02T16:50:14.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-02T16:50:14.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-02T16:50:14.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-02T16:50:14.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-02T16:50:14.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-02T16:50:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-02T16:50:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-02T16:50:14.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-02T16:50:14.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-02T16:50:14.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-02T16:50:14.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-02T16:50:14.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-02T16:50:14.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-02T16:50:14.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-02T16:50:14.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-02T16:50:14.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-02T16:50:14.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-02T16:50:14.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-02T16:50:14.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-02T16:50:14.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-02T16:50:14.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-02T16:50:14.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-02T16:50:14.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-02T16:50:14.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-02T16:50:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-02T16:50:14.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-02T16:50:14.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-02T16:50:14.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-02T16:50:14.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-02T16:50:14.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-02T16:50:14.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-02T16:50:14.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-02T16:50:14.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-02T16:50:14.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-02T16:50:14.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-02T16:50:14.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-02T16:50:14.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-02T16:50:14.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-02T16:50:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-02T16:50:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-02T16:50:14.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-02T16:50:14.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-02T16:50:14.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-02T16:50:14.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-02T16:50:14.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-02T16:50:14.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-02T16:50:14.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-02T16:50:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-02T16:50:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-02T16:50:14.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-02T16:50:14.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-02T16:50:14.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-02T16:50:14.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-02T16:50:14.373 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-02T16:50:14.374 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-02T16:50:14.375 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-02T16:50:14.376 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-02T16:50:14.377 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-02T16:50:14.378 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-02T16:50:14.379 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-02T16:50:14.380 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-02T16:50:14.381 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-02T16:50:14.382 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-02T16:50:14.383 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-02T16:50:14.384 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-02T16:50:14.385 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-02T16:50:14.386 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-02T16:50:14.387 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-02T16:50:14.388 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-02T16:50:14.389 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-02T16:50:14.390 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-02T16:50:14.391 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-02T16:50:14.392 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-02T16:50:14.393 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-02T16:50:14.394 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-02T16:50:14.395 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-02T16:50:14.396 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-02T16:50:14.397 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-02T16:50:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-02T16:50:14.399 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-02T16:50:14.400 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-02T16:50:14.401 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-02T16:50:14.402 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-02T16:50:14.403 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-02T16:50:14.404 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-02T16:50:14.405 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-02T16:50:14.406 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-02T16:50:14.407 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-02T16:50:14.408 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-02T16:50:14.413 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-02T16:50:14.413 DEBUG:teuthology.orchestra.run.smithi059:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout:postinstall scriptlet (using /bin/sh): 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout:for file in vmlinuz System.map config; do 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-02T16:50:14.437 INFO:teuthology.orchestra.run.smithi059.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout: fi 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-02T16:50:14.438 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-02T16:50:14.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 68.el9 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-04-02T16:50:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8264254 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-04-02T16:50:14.523 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:50:14.524 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T16:50:14.525 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T16:50:14.525 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 6.8.0-g1df0c7e8a235 2024-04-02T16:50:14.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T16:50:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-04-02T16:50:16.668 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T16:50:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-04-02T16:50:16.708 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-04-02T16:50:16.723 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.MBJTGRvVgT 2024-04-02T16:50:16.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MBJTGRvVgT 2024-04-02T16:50:16.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.MBJTGRvVgT 2024-04-02T16:50:16.938 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.MBJTGRvVgT is 6KB 2024-04-02T16:50:16.986 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.MBJTGRvVgT 2024-04-02T16:50:17.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T16:50:17.070 INFO:teuthology.orchestra.run.smithi059.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T16:50:17.070 INFO:teuthology.orchestra.run.smithi059.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T16:50:17.070 INFO:teuthology.orchestra.run.smithi059.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T16:50:17.070 INFO:teuthology.orchestra.run.smithi059.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-02T16:50:17.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-02T16:50:17.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-04-02T16:50:47.232 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-02T16:50:47.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-02T16:50:47.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:51:47.236 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T16:51:56.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-02T16:51:56.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:51:57.655 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-02T16:51:58.435 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-02T16:51:58.435 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-04-02T16:51:58.435 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-02T16:51:58.473 INFO:teuthology.orchestra.run.smithi059.stdout:6.8.0-g1df0c7e8a235 2024-04-02T16:51:58.473 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-02T16:51:58.473 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T16:51:58.473 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T16:51:59.477 DEBUG:teuthology.parallel:result is None 2024-04-02T16:51:59.477 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T16:51:59.485 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T16:51:59.485 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:51:59.492 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:51:59.495 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:51:59.531 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T16:51:59.537 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T16:51:59.543 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T16:51:59.544 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:51:59.576 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:51:59.579 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:51:59.675 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T16:51:59.682 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T16:51:59.682 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T16:51:59.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:51:59.699 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T16:51:59.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:51:59.715 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:51:59.722 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:51:59.742 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:51:59.779 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.798 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.825 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.847 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.848 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.956 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:51:59.960 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T16:51:59.967 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T16:51:59.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:51:59.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:51:59.975 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:52:00.039 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T16:52:00.048 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T16:52:00.048 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:52:00.052 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:52:00.054 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:52:00.083 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.299 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:00.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:00.447 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T16:52:00.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:52:00.543 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.644 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:00.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:00.773 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T16:52:00.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:52:00.842 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.887 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:52:00.952 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:00.992 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:52:01.039 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T16:52:01.040 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:52:01.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-04-02T16:52:01.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-04-02T16:52:01.120 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-02T16:52:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T16:52:01.193 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T16:52:01.236 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T16:52:01.554 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T16:52:01.561 INFO:teuthology.task.internal:Starting timer... 2024-04-02T16:52:01.561 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T16:52:01.572 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T16:52:01.581 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T16:52:01.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-04-02T16:52:01.688 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-04-02T16:52:01.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-04-02T16:52:01.748 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-04-02T16:52:01.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-02T16:52:01.801 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-02T16:52:01.803 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T16:52:01.803 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-04-02T16:52:01.859 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-04-02T16:52:01.859 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-04-02T16:52:01.881 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-04-02T16:52:01.881 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-02T16:52:01.903 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-02T16:52:01.903 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-02T16:52:01.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T16:52:01.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:52:01.983 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T16:52:02.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:52:02.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T16:52:02.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:52:02.039 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T16:52:02.040 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-04-02T16:52:02.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-04-02T16:52:02.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-02T16:52:02.197 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T16:52:02.209 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-02T16:52:02.217 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-02T16:52:02.362 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T16:52:02.379 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T16:52:02.380 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi154.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T16:58:06.393 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-02T16:58:06.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-02T16:58:06.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:58:06.542 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-02T16:58:06.699 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-02T16:58:06.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-02T16:58:06.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:58:06.790 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-02T16:58:06.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-02T16:58:06.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-02T16:58:06.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:58:06.945 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-02T16:58:07.014 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-02T16:58:07.014 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T16:58:07.024 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T16:58:07.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:58:07.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:58:07.032 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:58:07.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:58:07.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:58:07.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:58:07.063 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T16:58:07.079 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T16:58:07.086 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T16:58:07.104 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T16:58:07.108 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-04-02T16:58:07.122 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-04-02T16:58:07.134 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-02T16:58:07.137 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T16:58:07.138 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T16:58:07.147 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-02T16:58:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T16:58:07.164 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T16:58:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T16:58:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T16:58:07.198 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.203 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.224 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.227 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.294 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-04-02T16:58:07.340 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-04-02T16:58:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T16:58:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T16:58:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-02T16:58:07.600 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T16:58:07.600 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-02T16:58:07.600 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.601 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.601 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.601 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:58:07.602 INFO:teuthology.run_tasks:Running task install... 2024-04-02T16:58:07.611 DEBUG:teuthology.task.install:project ceph 2024-04-02T16:58:07.611 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T16:58:07.611 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T16:58:07.611 INFO:teuthology.task.install:Using flavor: default 2024-04-02T16:58:07.617 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T16:58:07.617 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T16:58:07.618 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T16:58:07.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:58:07.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T16:58:07.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:58:07.621 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T16:58:07.622 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:58:07.760 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T16:58:07.760 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T16:58:07.830 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T16:58:07.831 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T16:58:07.839 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T16:58:07.839 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T16:58:07.940 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T16:58:07.941 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T16:58:07.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T16:58:07.948 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T16:58:07.949 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T16:58:07.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T16:58:07.955 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T16:58:07.955 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T16:58:07.955 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T16:58:07.972 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T16:58:07.972 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T16:58:07.977 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T16:58:07.977 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T16:58:08.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T16:58:08.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T16:58:08.063 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T16:58:08.063 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T16:58:08.124 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T16:58:08.125 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T16:58:08.192 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-04-02T16:58:08.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-04-02T16:58:08.195 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-02T16:58:08.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-02T16:58:08.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T16:58:08.350 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T16:58:08.429 INFO:teuthology.orchestra.run.smithi154.stdout:74 files removed 2024-04-02T16:58:08.431 INFO:teuthology.orchestra.run.smithi164.stdout:74 files removed 2024-04-02T16:58:08.450 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-04-02T16:58:08.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-04-02T16:58:08.456 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T16:58:08.457 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T16:58:08.844 INFO:teuthology.orchestra.run.smithi059.stdout:74 files removed 2024-04-02T16:58:08.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T16:58:09.225 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-02T16:58:09.268 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 160 kB/s | 83 kB 00:00 2024-04-02T16:58:09.635 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-02T16:58:09.801 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-02T16:58:09.996 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 141 kB/s | 83 kB 00:00 2024-04-02T16:58:10.035 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-02T16:58:10.201 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-02T16:58:10.413 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-04-02T16:58:10.605 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-02T16:58:10.612 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-02T16:58:11.189 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-02T16:58:11.800 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-02T16:58:11.880 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-04-02T16:58:11.979 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-04-02T16:58:13.559 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-04-02T16:58:14.071 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-02T16:58:14.680 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-02T16:58:17.098 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-02T16:58:19.936 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.0 MB 00:07 2024-04-02T16:58:21.490 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-04-02T16:58:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-02T16:58:24.318 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-02T16:58:30.468 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-02T16:58:31.836 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-02T16:58:33.361 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:58:33.363 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:58:33.385 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T16:58:33.386 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T16:58:33.387 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T16:58:33.389 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T16:58:33.391 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:58:33.395 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T16:58:33.396 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T16:58:33.490 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T16:58:33.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T16:58:33.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T16:58:33.521 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T16:58:33.522 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T16:58:33.523 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T16:58:33.524 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T16:58:33.525 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T16:58:33.526 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T16:58:33.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T16:58:33.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T16:58:33.532 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T16:58:33.533 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T16:58:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T16:58:33.535 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T16:58:33.536 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:Install 150 Packages 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 24 Packages 2024-04-02T16:58:33.537 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:58:33.538 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 226 M 2024-04-02T16:58:33.538 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-04-02T16:58:34.544 INFO:teuthology.orchestra.run.smithi059.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-02T16:58:34.736 INFO:teuthology.orchestra.run.smithi059.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 4.5 MB/s | 890 kB 00:00 2024-04-02T16:58:34.803 INFO:teuthology.orchestra.run.smithi059.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.2 MB/s | 145 kB 00:00 2024-04-02T16:58:35.128 INFO:teuthology.orchestra.run.smithi059.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 7.3 MB/s | 2.4 MB 00:00 2024-04-02T16:58:35.237 INFO:teuthology.orchestra.run.smithi059.stdout:(5/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-04-02T16:58:35.320 INFO:teuthology.orchestra.run.smithi059.stdout:(6/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-02T16:58:35.979 INFO:teuthology.orchestra.run.smithi059.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-04-02T16:58:36.003 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.1 MB 00:00 2024-04-02T16:58:36.521 INFO:teuthology.orchestra.run.smithi059.stdout:(8/174): ceph-common-19.0.0-2609.g191d963a.el9. 9.2 MB/s | 21 MB 00:02 2024-04-02T16:58:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 344 kB/s | 25 kB 00:00 2024-04-02T16:58:37.388 INFO:teuthology.orchestra.run.smithi059.stdout:(10/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 8.2 MB/s | 17 MB 00:02 2024-04-02T16:58:37.462 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-02T16:58:37.563 INFO:teuthology.orchestra.run.smithi059.stdout:(11/174): ceph-radosgw-19.0.0-2609.g191d963a.el 6.2 MB/s | 9.8 MB 00:01 2024-04-02T16:58:37.597 INFO:teuthology.orchestra.run.smithi059.stdout:(12/174): cephfs-mirror-19.0.0-2609.g191d963a.e 1.1 MB/s | 225 kB 00:00 2024-04-02T16:58:37.639 INFO:teuthology.orchestra.run.smithi059.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 365 kB/s | 27 kB 00:00 2024-04-02T16:58:37.672 INFO:teuthology.orchestra.run.smithi059.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 425 kB/s | 32 kB 00:00 2024-04-02T16:58:37.740 INFO:teuthology.orchestra.run.smithi059.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 2.4 MB/s | 164 kB 00:00 2024-04-02T16:58:37.806 INFO:teuthology.orchestra.run.smithi059.stdout:(16/174): libcephfs2-19.0.0-2609.g191d963a.el9. 4.4 MB/s | 747 kB 00:00 2024-04-02T16:58:37.840 INFO:teuthology.orchestra.run.smithi059.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 1.2 MB/s | 127 kB 00:00 2024-04-02T16:58:37.937 INFO:teuthology.orchestra.run.smithi059.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 3.8 MB/s | 500 kB 00:00 2024-04-02T16:58:38.032 INFO:teuthology.orchestra.run.smithi059.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 480 kB/s | 45 kB 00:00 2024-04-02T16:58:38.149 INFO:teuthology.orchestra.run.smithi059.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.1 MB/s | 134 kB 00:00 2024-04-02T16:58:38.258 INFO:teuthology.orchestra.run.smithi059.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.5 MB/s | 161 kB 00:00 2024-04-02T16:58:38.416 INFO:teuthology.orchestra.run.smithi059.stdout:(22/174): librgw2-19.0.0-2609.g191d963a.el9.x86 8.9 MB/s | 5.1 MB 00:00 2024-04-02T16:58:38.458 INFO:teuthology.orchestra.run.smithi059.stdout:(23/174): python3-rados-19.0.0-2609.g191d963a.e 1.6 MB/s | 320 kB 00:00 2024-04-02T16:58:38.492 INFO:teuthology.orchestra.run.smithi059.stdout:(24/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.9 MB/s | 299 kB 00:00 2024-04-02T16:58:38.634 INFO:teuthology.orchestra.run.smithi059.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 608 kB/s | 85 kB 00:00 2024-04-02T16:58:38.667 INFO:teuthology.orchestra.run.smithi059.stdout:(26/174): python3-rgw-19.0.0-2609.g191d963a.el9 478 kB/s | 99 kB 00:00 2024-04-02T16:58:38.751 INFO:teuthology.orchestra.run.smithi059.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-02T16:58:38.859 INFO:teuthology.orchestra.run.smithi059.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 315 kB/s | 34 kB 00:00 2024-04-02T16:58:39.002 INFO:teuthology.orchestra.run.smithi059.stdout:(29/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 8.4 MB/s | 3.1 MB 00:00 2024-04-02T16:58:39.043 INFO:teuthology.orchestra.run.smithi059.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 774 kB/s | 141 kB 00:00 2024-04-02T16:58:39.635 INFO:teuthology.orchestra.run.smithi059.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 5.6 MB/s | 3.5 MB 00:00 2024-04-02T16:58:39.727 INFO:teuthology.orchestra.run.smithi059.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.7 MB/s | 249 kB 00:00 2024-04-02T16:58:39.786 INFO:teuthology.orchestra.run.smithi059.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 862 kB/s | 49 kB 00:00 2024-04-02T16:58:39.869 INFO:teuthology.orchestra.run.smithi059.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 200 kB/s | 16 kB 00:00 2024-04-02T16:58:40.412 INFO:teuthology.orchestra.run.smithi059.stdout:(35/174): ceph-test-19.0.0-2609.g191d963a.el9.x 12 MB/s | 48 MB 00:03 2024-04-02T16:58:40.445 INFO:teuthology.orchestra.run.smithi059.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 459 kB/s | 264 kB 00:00 2024-04-02T16:58:40.555 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-02T16:58:40.595 INFO:teuthology.orchestra.run.smithi059.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-02T16:58:40.620 INFO:teuthology.orchestra.run.smithi059.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 104 kB/s | 18 kB 00:00 2024-04-02T16:58:40.662 INFO:teuthology.orchestra.run.smithi059.stdout:(39/174): cephadm-19.0.0-2609.g191d963a.el9.noa 3.0 MB/s | 762 kB 00:00 2024-04-02T16:58:40.929 INFO:teuthology.orchestra.run.smithi059.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 709 kB/s | 106 kB 00:00 2024-04-02T16:58:40.971 INFO:teuthology.orchestra.run.smithi059.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-04-02T16:58:41.013 INFO:teuthology.orchestra.run.smithi059.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 920 kB 00:00 2024-04-02T16:58:41.046 INFO:teuthology.orchestra.run.smithi059.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 307 kB 00:00 2024-04-02T16:58:41.089 INFO:teuthology.orchestra.run.smithi059.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 441 kB/s | 32 kB 00:00 2024-04-02T16:58:41.172 INFO:teuthology.orchestra.run.smithi059.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-02T16:58:41.222 INFO:teuthology.orchestra.run.smithi059.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-04-02T16:58:41.248 INFO:teuthology.orchestra.run.smithi059.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 69 kB/s | 10 kB 00:00 2024-04-02T16:58:41.273 INFO:teuthology.orchestra.run.smithi059.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 44 kB 00:00 2024-04-02T16:58:41.307 INFO:teuthology.orchestra.run.smithi059.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-04-02T16:58:41.340 INFO:teuthology.orchestra.run.smithi059.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-04-02T16:58:41.390 INFO:teuthology.orchestra.run.smithi059.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2024-04-02T16:58:41.416 INFO:teuthology.orchestra.run.smithi059.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 678 kB/s | 46 kB 00:00 2024-04-02T16:58:41.441 INFO:teuthology.orchestra.run.smithi059.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-04-02T16:58:41.466 INFO:teuthology.orchestra.run.smithi059.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.6 MB/s | 187 kB 00:00 2024-04-02T16:58:41.516 INFO:teuthology.orchestra.run.smithi059.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 473 kB/s | 34 kB 00:00 2024-04-02T16:58:41.558 INFO:teuthology.orchestra.run.smithi059.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.9 MB/s | 217 kB 00:00 2024-04-02T16:58:41.700 INFO:teuthology.orchestra.run.smithi059.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-02T16:58:41.759 INFO:teuthology.orchestra.run.smithi059.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 817 kB 00:00 2024-04-02T16:58:41.818 INFO:teuthology.orchestra.run.smithi059.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.5 MB 00:00 2024-04-02T16:58:41.843 INFO:teuthology.orchestra.run.smithi059.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 938 kB/s | 132 kB 00:00 2024-04-02T16:58:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-04-02T16:58:42.068 INFO:teuthology.orchestra.run.smithi059.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 372 kB/s | 78 kB 00:00 2024-04-02T16:58:42.118 INFO:teuthology.orchestra.run.smithi059.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-02T16:58:42.352 INFO:teuthology.orchestra.run.smithi059.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-04-02T16:58:42.369 INFO:teuthology.orchestra.run.smithi059.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 84 kB/s | 24 kB 00:00 2024-04-02T16:58:42.385 INFO:teuthology.orchestra.run.smithi059.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 18 kB 00:00 2024-04-02T16:58:42.411 INFO:teuthology.orchestra.run.smithi059.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-04-02T16:58:42.444 INFO:teuthology.orchestra.run.smithi059.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 563 kB/s | 32 kB 00:00 2024-04-02T16:58:42.681 INFO:teuthology.orchestra.run.smithi059.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-04-02T16:58:42.761 INFO:teuthology.orchestra.run.smithi059.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 641 kB/s | 217 kB 00:00 2024-04-02T16:58:42.787 INFO:teuthology.orchestra.run.smithi059.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 534 kB/s | 174 kB 00:00 2024-04-02T16:58:42.812 INFO:teuthology.orchestra.run.smithi059.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 69 kB 00:00 2024-04-02T16:58:42.837 INFO:teuthology.orchestra.run.smithi059.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 41 kB 00:00 2024-04-02T16:58:42.862 INFO:teuthology.orchestra.run.smithi059.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 238 kB 00:00 2024-04-02T16:58:42.888 INFO:teuthology.orchestra.run.smithi059.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T16:58:42.930 INFO:teuthology.orchestra.run.smithi059.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 736 kB/s | 47 kB 00:00 2024-04-02T16:58:42.963 INFO:teuthology.orchestra.run.smithi059.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2024-04-02T16:58:42.988 INFO:teuthology.orchestra.run.smithi059.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 433 kB/s | 43 kB 00:00 2024-04-02T16:58:43.376 INFO:teuthology.orchestra.run.smithi059.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-02T16:58:43.406 INFO:teuthology.orchestra.run.smithi059.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 94 kB/s | 39 kB 00:00 2024-04-02T16:58:43.431 INFO:teuthology.orchestra.run.smithi059.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 94 kB/s | 41 kB 00:00 2024-04-02T16:58:43.465 INFO:teuthology.orchestra.run.smithi059.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.5 MB/s | 220 kB 00:00 2024-04-02T16:58:43.490 INFO:teuthology.orchestra.run.smithi059.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 628 kB/s | 35 kB 00:00 2024-04-02T16:58:43.515 INFO:teuthology.orchestra.run.smithi059.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 576 kB/s | 46 kB 00:00 2024-04-02T16:58:43.549 INFO:teuthology.orchestra.run.smithi059.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-02T16:58:43.574 INFO:teuthology.orchestra.run.smithi059.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-02T16:58:43.599 INFO:teuthology.orchestra.run.smithi059.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 786 kB/s | 63 kB 00:00 2024-04-02T16:58:43.624 INFO:teuthology.orchestra.run.smithi059.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 593 kB/s | 44 kB 00:00 2024-04-02T16:58:43.658 INFO:teuthology.orchestra.run.smithi059.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.4 MB/s | 79 kB 00:00 2024-04-02T16:58:43.683 INFO:teuthology.orchestra.run.smithi059.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-04-02T16:58:43.733 INFO:teuthology.orchestra.run.smithi059.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-04-02T16:58:43.750 INFO:teuthology.orchestra.run.smithi059.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-02T16:58:43.775 INFO:teuthology.orchestra.run.smithi059.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T16:58:43.801 INFO:teuthology.orchestra.run.smithi059.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-04-02T16:58:43.868 INFO:teuthology.orchestra.run.smithi059.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-04-02T16:58:43.901 INFO:teuthology.orchestra.run.smithi059.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-02T16:58:43.977 INFO:teuthology.orchestra.run.smithi059.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-02T16:58:44.001 INFO:teuthology.orchestra.run.smithi059.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 941 kB/s | 125 kB 00:00 2024-04-02T16:58:44.027 INFO:teuthology.orchestra.run.smithi059.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 173 kB/s | 21 kB 00:00 2024-04-02T16:58:44.085 INFO:teuthology.orchestra.run.smithi059.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 236 kB/s | 25 kB 00:00 2024-04-02T16:58:44.269 INFO:teuthology.orchestra.run.smithi059.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-04-02T16:58:44.294 INFO:teuthology.orchestra.run.smithi059.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 923 kB/s | 245 kB 00:00 2024-04-02T16:58:44.328 INFO:teuthology.orchestra.run.smithi059.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-02T16:58:44.395 INFO:teuthology.orchestra.run.smithi059.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-04-02T16:58:44.554 INFO:teuthology.orchestra.run.smithi059.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.8 MB/s | 863 kB 00:00 2024-04-02T16:58:44.987 INFO:teuthology.orchestra.run.smithi059.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.3 MB/s | 4.1 MB 00:00 2024-04-02T16:58:45.021 INFO:teuthology.orchestra.run.smithi059.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 878 kB/s | 548 kB 00:00 2024-04-02T16:58:45.054 INFO:teuthology.orchestra.run.smithi059.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 618 kB/s | 309 kB 00:00 2024-04-02T16:58:45.146 INFO:teuthology.orchestra.run.smithi059.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 285 kB/s | 26 kB 00:00 2024-04-02T16:58:45.213 INFO:teuthology.orchestra.run.smithi059.stdout:(110/174): grpc-data-1.46.7-10.el9.noarch.rpm 87 kB/s | 19 kB 00:00 2024-04-02T16:58:45.230 INFO:teuthology.orchestra.run.smithi059.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 586 kB/s | 49 kB 00:00 2024-04-02T16:58:45.331 INFO:teuthology.orchestra.run.smithi059.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-02T16:58:45.439 INFO:teuthology.orchestra.run.smithi059.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-02T16:58:45.464 INFO:teuthology.orchestra.run.smithi059.stdout:(114/174): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2024-04-02T16:58:45.514 INFO:teuthology.orchestra.run.smithi059.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-02T16:58:45.574 INFO:teuthology.orchestra.run.smithi059.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-02T16:58:45.580 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-02T16:58:45.598 INFO:teuthology.orchestra.run.smithi059.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-04-02T16:58:45.624 INFO:teuthology.orchestra.run.smithi059.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 298 kB/s | 32 kB 00:00 2024-04-02T16:58:45.666 INFO:teuthology.orchestra.run.smithi059.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-02T16:58:45.724 INFO:teuthology.orchestra.run.smithi059.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-02T16:58:45.766 INFO:teuthology.orchestra.run.smithi059.stdout:(121/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-02T16:58:45.800 INFO:teuthology.orchestra.run.smithi059.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-02T16:58:45.825 INFO:teuthology.orchestra.run.smithi059.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-04-02T16:58:45.867 INFO:teuthology.orchestra.run.smithi059.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-02T16:58:45.917 INFO:teuthology.orchestra.run.smithi059.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-02T16:58:45.959 INFO:teuthology.orchestra.run.smithi059.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-02T16:58:46.018 INFO:teuthology.orchestra.run.smithi059.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-02T16:58:46.060 INFO:teuthology.orchestra.run.smithi059.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-02T16:58:46.110 INFO:teuthology.orchestra.run.smithi059.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-04-02T16:58:46.152 INFO:teuthology.orchestra.run.smithi059.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-04-02T16:58:46.235 INFO:teuthology.orchestra.run.smithi059.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-02T16:58:46.261 INFO:teuthology.orchestra.run.smithi059.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-04-02T16:58:46.328 INFO:teuthology.orchestra.run.smithi059.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-04-02T16:58:46.361 INFO:teuthology.orchestra.run.smithi059.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-04-02T16:58:46.420 INFO:teuthology.orchestra.run.smithi059.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-02T16:58:46.454 INFO:teuthology.orchestra.run.smithi059.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-02T16:58:46.512 INFO:teuthology.orchestra.run.smithi059.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 986 kB/s | 90 kB 00:00 2024-04-02T16:58:46.554 INFO:teuthology.orchestra.run.smithi059.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-02T16:58:46.604 INFO:teuthology.orchestra.run.smithi059.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 337 kB/s | 31 kB 00:00 2024-04-02T16:58:46.671 INFO:teuthology.orchestra.run.smithi059.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-02T16:58:46.696 INFO:teuthology.orchestra.run.smithi059.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 648 kB/s | 59 kB 00:00 2024-04-02T16:58:46.772 INFO:teuthology.orchestra.run.smithi059.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-04-02T16:58:46.805 INFO:teuthology.orchestra.run.smithi059.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-04-02T16:58:46.873 INFO:teuthology.orchestra.run.smithi059.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-04-02T16:58:46.889 INFO:teuthology.orchestra.run.smithi059.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-02T16:58:46.948 INFO:teuthology.orchestra.run.smithi059.stdout:(146/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.8 MB/s | 2.0 MB 00:01 2024-04-02T16:58:46.981 INFO:teuthology.orchestra.run.smithi059.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-02T16:58:47.007 INFO:teuthology.orchestra.run.smithi059.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-02T16:58:47.049 INFO:teuthology.orchestra.run.smithi059.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-02T16:58:47.199 INFO:teuthology.orchestra.run.smithi059.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-02T16:58:47.466 INFO:teuthology.orchestra.run.smithi059.stdout:(151/174): librados2-19.0.0-2609.g191d963a.el9. 7.4 MB/s | 3.4 MB 00:00 2024-04-02T16:58:47.525 INFO:teuthology.orchestra.run.smithi059.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2024-04-02T16:58:47.633 INFO:teuthology.orchestra.run.smithi059.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-04-02T16:58:47.650 INFO:teuthology.orchestra.run.smithi059.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 358 kB/s | 41 kB 00:00 2024-04-02T16:58:47.684 INFO:teuthology.orchestra.run.smithi059.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.9 MB/s | 247 kB 00:00 2024-04-02T16:58:47.741 INFO:teuthology.orchestra.run.smithi059.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.6 MB/s | 321 kB 00:00 2024-04-02T16:58:47.834 INFO:teuthology.orchestra.run.smithi059.stdout:(157/174): librbd1-19.0.0-2609.g191d963a.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-02T16:58:47.859 INFO:teuthology.orchestra.run.smithi059.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 416 kB/s | 71 kB 00:00 2024-04-02T16:58:47.885 INFO:teuthology.orchestra.run.smithi059.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 256 kB/s | 36 kB 00:00 2024-04-02T16:58:47.943 INFO:teuthology.orchestra.run.smithi059.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 405 kB/s | 23 kB 00:00 2024-04-02T16:58:47.977 INFO:teuthology.orchestra.run.smithi059.stdout:(161/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-04-02T16:58:48.010 INFO:teuthology.orchestra.run.smithi059.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-04-02T16:58:48.061 INFO:teuthology.orchestra.run.smithi059.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 933 kB/s | 108 kB 00:00 2024-04-02T16:58:48.111 INFO:teuthology.orchestra.run.smithi059.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-04-02T16:58:48.136 INFO:teuthology.orchestra.run.smithi059.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-02T16:58:48.162 INFO:teuthology.orchestra.run.smithi059.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 642 kB/s | 64 kB 00:00 2024-04-02T16:58:48.163 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:58:48.164 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:58:48.179 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T16:58:48.180 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T16:58:48.181 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T16:58:48.182 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T16:58:48.184 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:58:48.187 INFO:teuthology.orchestra.run.smithi059.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 386 kB/s | 29 kB 00:00 2024-04-02T16:58:48.187 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T16:58:48.188 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T16:58:48.268 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-04-02T16:58:48.294 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T16:58:48.295 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T16:58:48.296 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T16:58:48.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T16:58:48.298 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T16:58:48.299 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T16:58:48.300 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T16:58:48.301 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T16:58:48.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T16:58:48.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T16:58:48.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T16:58:48.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T16:58:48.306 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:Install 150 Packages 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 24 Packages 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:58:48.307 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 226 M 2024-04-02T16:58:48.308 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-04-02T16:58:48.337 INFO:teuthology.orchestra.run.smithi059.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.1 MB/s | 1.2 MB 00:00 2024-04-02T16:58:48.521 INFO:teuthology.orchestra.run.smithi059.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 2.2 MB 00:00 2024-04-02T16:58:48.554 INFO:teuthology.orchestra.run.smithi059.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.1 MB/s | 465 kB 00:00 2024-04-02T16:58:48.624 INFO:teuthology.orchestra.run.smithi059.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.2 MB/s | 2.3 MB 00:00 2024-04-02T16:58:48.649 INFO:teuthology.orchestra.run.smithi059.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 245 kB/s | 31 kB 00:00 2024-04-02T16:58:48.707 INFO:teuthology.orchestra.run.smithi059.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 900 kB/s | 74 kB 00:00 2024-04-02T16:58:48.827 INFO:teuthology.orchestra.run.smithi059.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-04-02T16:58:48.835 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:58:48.836 INFO:teuthology.orchestra.run.smithi059.stdout:Total 15 MB/s | 226 MB 00:15 2024-04-02T16:58:48.939 INFO:teuthology.orchestra.run.smithi154.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-02T16:58:49.356 INFO:teuthology.orchestra.run.smithi154.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-04-02T16:58:49.415 INFO:teuthology.orchestra.run.smithi154.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.4 MB/s | 145 kB 00:00 2024-04-02T16:58:49.657 INFO:teuthology.orchestra.run.smithi154.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-04-02T16:58:49.723 INFO:teuthology.orchestra.run.smithi154.stdout:(5/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-02T16:58:49.824 INFO:teuthology.orchestra.run.smithi154.stdout:(6/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-02T16:58:50.044 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-04-02T16:58:50.349 INFO:teuthology.orchestra.run.smithi154.stdout:(7/174): ceph-common-19.0.0-2609.g191d963a.el9. 13 MB/s | 21 MB 00:01 2024-04-02T16:58:50.441 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-04-02T16:58:50.441 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-04-02T16:58:51.158 INFO:teuthology.orchestra.run.smithi154.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 12 MB/s | 9.8 MB 00:00 2024-04-02T16:58:51.333 INFO:teuthology.orchestra.run.smithi154.stdout:(9/174): ceph-osd-19.0.0-2609.g191d963a.el9.x86 11 MB/s | 17 MB 00:01 2024-04-02T16:58:51.358 INFO:teuthology.orchestra.run.smithi154.stdout:(10/174): ceph-selinux-19.0.0-2609.g191d963a.el 126 kB/s | 25 kB 00:00 2024-04-02T16:58:51.425 INFO:teuthology.orchestra.run.smithi154.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.3 MB/s | 225 kB 00:00 2024-04-02T16:58:51.525 INFO:teuthology.orchestra.run.smithi154.stdout:(12/174): ceph-mon-19.0.0-2609.g191d963a.el9.x8 2.6 MB/s | 4.7 MB 00:01 2024-04-02T16:58:51.551 INFO:teuthology.orchestra.run.smithi154.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 215 kB/s | 27 kB 00:00 2024-04-02T16:58:51.593 INFO:teuthology.orchestra.run.smithi154.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 473 kB/s | 32 kB 00:00 2024-04-02T16:58:51.643 INFO:teuthology.orchestra.run.smithi154.stdout:(15/174): libcephfs2-19.0.0-2609.g191d963a.el9. 7.9 MB/s | 747 kB 00:00 2024-04-02T16:58:51.685 INFO:teuthology.orchestra.run.smithi154.stdout:(16/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.7 MB/s | 164 kB 00:00 2024-04-02T16:58:51.710 INFO:teuthology.orchestra.run.smithi154.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 1.8 MB/s | 127 kB 00:00 2024-04-02T16:58:51.877 INFO:teuthology.orchestra.run.smithi154.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 2.5 MB/s | 500 kB 00:00 2024-04-02T16:58:51.994 INFO:teuthology.orchestra.run.smithi154.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 387 kB/s | 45 kB 00:00 2024-04-02T16:58:52.145 INFO:teuthology.orchestra.run.smithi154.stdout:(20/174): librgw2-19.0.0-2609.g191d963a.el9.x86 12 MB/s | 5.1 MB 00:00 2024-04-02T16:58:52.161 INFO:teuthology.orchestra.run.smithi154.stdout:(21/174): python3-ceph-common-19.0.0-2609.g191d 805 kB/s | 134 kB 00:00 2024-04-02T16:58:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:(22/174): python3-cephfs-19.0.0-2609.g191d963a. 2.7 MB/s | 161 kB 00:00 2024-04-02T16:58:52.254 INFO:teuthology.orchestra.run.smithi154.stdout:(23/174): python3-rados-19.0.0-2609.g191d963a.e 3.4 MB/s | 320 kB 00:00 2024-04-02T16:58:52.279 INFO:teuthology.orchestra.run.smithi154.stdout:(24/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.9 MB/s | 299 kB 00:00 2024-04-02T16:58:52.338 INFO:teuthology.orchestra.run.smithi154.stdout:(25/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.2 MB/s | 99 kB 00:00 2024-04-02T16:58:52.363 INFO:teuthology.orchestra.run.smithi154.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-02T16:58:52.438 INFO:teuthology.orchestra.run.smithi154.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-02T16:58:52.497 INFO:teuthology.orchestra.run.smithi154.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 584 kB/s | 34 kB 00:00 2024-04-02T16:58:52.572 INFO:teuthology.orchestra.run.smithi154.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.8 MB/s | 141 kB 00:00 2024-04-02T16:58:52.849 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-04-02T16:58:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-04-02T16:58:52.914 INFO:teuthology.orchestra.run.smithi154.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 10 MB/s | 3.5 MB 00:00 2024-04-02T16:58:53.031 INFO:teuthology.orchestra.run.smithi154.stdout:(31/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 4.5 MB/s | 3.1 MB 00:00 2024-04-02T16:58:53.131 INFO:teuthology.orchestra.run.smithi154.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.4 MB/s | 249 kB 00:00 2024-04-02T16:58:53.223 INFO:teuthology.orchestra.run.smithi154.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 537 kB/s | 49 kB 00:00 2024-04-02T16:58:53.332 INFO:teuthology.orchestra.run.smithi154.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 152 kB/s | 16 kB 00:00 2024-04-02T16:58:53.515 INFO:teuthology.orchestra.run.smithi154.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-02T16:58:53.540 INFO:teuthology.orchestra.run.smithi154.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 1.2 MB/s | 264 kB 00:00 2024-04-02T16:58:53.599 INFO:teuthology.orchestra.run.smithi154.stdout:(37/174): cephadm-19.0.0-2609.g191d963a.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-02T16:58:53.658 INFO:teuthology.orchestra.run.smithi154.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 156 kB/s | 18 kB 00:00 2024-04-02T16:58:53.832 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 589 kB/s | 19 MB 00:32 2024-04-02T16:58:53.991 INFO:teuthology.orchestra.run.smithi154.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-04-02T16:58:54.043 INFO:teuthology.orchestra.run.smithi154.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2024-04-02T16:58:54.093 INFO:teuthology.orchestra.run.smithi154.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2024-04-02T16:58:54.119 INFO:teuthology.orchestra.run.smithi154.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-04-02T16:58:54.553 INFO:teuthology.orchestra.run.smithi154.stdout:(43/174): ceph-test-19.0.0-2609.g191d963a.el9.x 15 MB/s | 48 MB 00:03 2024-04-02T16:58:54.570 INFO:teuthology.orchestra.run.smithi154.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 66 kB/s | 32 kB 00:00 2024-04-02T16:58:54.654 INFO:teuthology.orchestra.run.smithi154.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-02T16:58:54.671 INFO:teuthology.orchestra.run.smithi154.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 149 kB/s | 17 kB 00:00 2024-04-02T16:58:54.688 INFO:teuthology.orchestra.run.smithi154.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 86 kB/s | 10 kB 00:00 2024-04-02T16:58:54.713 INFO:teuthology.orchestra.run.smithi154.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-04-02T16:58:54.730 INFO:teuthology.orchestra.run.smithi154.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-02T16:58:54.780 INFO:teuthology.orchestra.run.smithi154.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-04-02T16:58:54.806 INFO:teuthology.orchestra.run.smithi154.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-04-02T16:58:54.822 INFO:teuthology.orchestra.run.smithi154.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 507 kB/s | 46 kB 00:00 2024-04-02T16:58:54.848 INFO:teuthology.orchestra.run.smithi154.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-04-02T16:58:54.873 INFO:teuthology.orchestra.run.smithi154.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-04-02T16:58:54.898 INFO:teuthology.orchestra.run.smithi154.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-02T16:58:54.915 INFO:teuthology.orchestra.run.smithi154.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 510 kB/s | 34 kB 00:00 2024-04-02T16:58:55.182 INFO:teuthology.orchestra.run.smithi154.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-02T16:58:55.224 INFO:teuthology.orchestra.run.smithi154.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 4.7 MB/s | 1.5 MB 00:00 2024-04-02T16:58:55.258 INFO:teuthology.orchestra.run.smithi154.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 2.3 MB/s | 817 kB 00:00 2024-04-02T16:58:55.283 INFO:teuthology.orchestra.run.smithi154.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 132 kB 00:00 2024-04-02T16:58:55.550 INFO:teuthology.orchestra.run.smithi154.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-04-02T16:58:55.575 INFO:teuthology.orchestra.run.smithi154.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 267 kB/s | 78 kB 00:00 2024-04-02T16:58:55.609 INFO:teuthology.orchestra.run.smithi154.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.1 MB/s | 1.0 MB 00:00 2024-04-02T16:58:55.851 INFO:teuthology.orchestra.run.smithi154.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-04-02T16:58:55.876 INFO:teuthology.orchestra.run.smithi154.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 82 kB/s | 24 kB 00:00 2024-04-02T16:58:55.901 INFO:teuthology.orchestra.run.smithi154.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 62 kB/s | 18 kB 00:00 2024-04-02T16:58:55.926 INFO:teuthology.orchestra.run.smithi154.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T16:58:55.960 INFO:teuthology.orchestra.run.smithi154.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 544 kB/s | 32 kB 00:00 2024-04-02T16:58:56.202 INFO:teuthology.orchestra.run.smithi154.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-04-02T16:58:56.227 INFO:teuthology.orchestra.run.smithi154.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 723 kB/s | 217 kB 00:00 2024-04-02T16:58:56.253 INFO:teuthology.orchestra.run.smithi154.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 601 kB/s | 174 kB 00:00 2024-04-02T16:58:56.278 INFO:teuthology.orchestra.run.smithi154.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-04-02T16:58:56.295 INFO:teuthology.orchestra.run.smithi154.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 989 kB/s | 41 kB 00:00 2024-04-02T16:58:56.320 INFO:teuthology.orchestra.run.smithi154.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-02T16:58:56.345 INFO:teuthology.orchestra.run.smithi154.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-02T16:58:56.371 INFO:teuthology.orchestra.run.smithi154.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-02T16:58:56.387 INFO:teuthology.orchestra.run.smithi154.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 704 kB/s | 47 kB 00:00 2024-04-02T16:58:56.405 INFO:teuthology.orchestra.run.smithi154.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 729 kB/s | 43 kB 00:00 2024-04-02T16:58:56.580 INFO:teuthology.orchestra.run.smithi154.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 201 kB/s | 39 kB 00:00 2024-04-02T16:58:56.597 INFO:teuthology.orchestra.run.smithi154.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 214 kB/s | 41 kB 00:00 2024-04-02T16:58:56.781 INFO:teuthology.orchestra.run.smithi154.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-02T16:58:56.806 INFO:teuthology.orchestra.run.smithi154.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 980 kB/s | 220 kB 00:00 2024-04-02T16:58:56.822 INFO:teuthology.orchestra.run.smithi154.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 156 kB/s | 35 kB 00:00 2024-04-02T16:58:56.839 INFO:teuthology.orchestra.run.smithi154.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 801 kB/s | 46 kB 00:00 2024-04-02T16:58:56.856 INFO:teuthology.orchestra.run.smithi154.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-04-02T16:58:56.882 INFO:teuthology.orchestra.run.smithi154.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-02T16:58:56.899 INFO:teuthology.orchestra.run.smithi154.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-04-02T16:58:56.915 INFO:teuthology.orchestra.run.smithi154.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 760 kB/s | 44 kB 00:00 2024-04-02T16:58:56.941 INFO:teuthology.orchestra.run.smithi154.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-04-02T16:58:56.966 INFO:teuthology.orchestra.run.smithi154.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-04-02T16:58:57.000 INFO:teuthology.orchestra.run.smithi154.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T16:58:57.025 INFO:teuthology.orchestra.run.smithi154.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-02T16:58:57.050 INFO:teuthology.orchestra.run.smithi154.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T16:58:57.067 INFO:teuthology.orchestra.run.smithi154.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-04-02T16:58:57.229 INFO:teuthology.orchestra.run.smithi154.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-04-02T16:58:57.259 INFO:teuthology.orchestra.run.smithi154.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-02T16:58:57.285 INFO:teuthology.orchestra.run.smithi154.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 125 kB 00:00 2024-04-02T16:58:57.346 INFO:teuthology.orchestra.run.smithi154.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-02T16:58:57.363 INFO:teuthology.orchestra.run.smithi154.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 207 kB/s | 21 kB 00:00 2024-04-02T16:58:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 267 kB/s | 25 kB 00:00 2024-04-02T16:58:57.464 INFO:teuthology.orchestra.run.smithi154.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.4 MB/s | 245 kB 00:00 2024-04-02T16:58:57.506 INFO:teuthology.orchestra.run.smithi154.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-02T16:58:57.531 INFO:teuthology.orchestra.run.smithi154.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.6 MB/s | 248 kB 00:00 2024-04-02T16:58:57.598 INFO:teuthology.orchestra.run.smithi154.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 16 MB/s | 4.1 MB 00:00 2024-04-02T16:58:57.774 INFO:teuthology.orchestra.run.smithi154.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-04-02T16:58:58.274 INFO:teuthology.orchestra.run.smithi154.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-02T16:58:58.733 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.1 MB 00:01 2024-04-02T16:59:00.007 INFO:teuthology.orchestra.run.smithi154.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 138 kB/s | 309 kB 00:02 2024-04-02T16:59:00.041 INFO:teuthology.orchestra.run.smithi154.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 11 kB/s | 19 kB 00:01 2024-04-02T16:59:00.067 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 16 kB 00:00 2024-04-02T16:59:00.475 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-04-02T16:59:00.558 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T16:59:00.605 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T16:59:00.794 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:00.839 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:00.918 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-04-02T16:59:00.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T16:59:01.038 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T16:59:01.268 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T16:59:01.303 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T16:59:01.403 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T16:59:01.835 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T16:59:01.895 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:02.064 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:02.099 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:02.231 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:02.273 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:59:02.352 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T16:59:02.686 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T16:59:02.754 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T16:59:02.779 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:02.934 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:02.980 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T16:59:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:03.324 INFO:teuthology.orchestra.run.smithi154.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 7.9 kB/s | 26 kB 00:03 2024-04-02T16:59:03.541 INFO:teuthology.orchestra.run.smithi154.stdout:(110/174): abseil-cpp-20211102.0-3.el9.x86_64.r 92 kB/s | 548 kB 00:05 2024-04-02T16:59:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:03.732 INFO:teuthology.orchestra.run.smithi154.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 352 kB/s | 67 kB 00:00 2024-04-02T16:59:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T16:59:03.958 INFO:teuthology.orchestra.run.smithi154.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 77 kB/s | 49 kB 00:00 2024-04-02T16:59:03.991 INFO:teuthology.orchestra.run.smithi154.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 584 kB/s | 151 kB 00:00 2024-04-02T16:59:04.066 INFO:teuthology.orchestra.run.smithi154.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:04 2024-04-02T16:59:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T16:59:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:04.642 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:04.979 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:05.083 INFO:teuthology.orchestra.run.smithi154.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 43 kB/s | 43 kB 00:01 2024-04-02T16:59:05.142 INFO:teuthology.orchestra.run.smithi154.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 553 kB/s | 32 kB 00:00 2024-04-02T16:59:05.163 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T16:59:05.184 INFO:teuthology.orchestra.run.smithi154.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 684 kB/s | 838 kB 00:01 2024-04-02T16:59:05.210 INFO:teuthology.orchestra.run.smithi154.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-02T16:59:05.234 INFO:teuthology.orchestra.run.smithi154.stdout:(119/174): python3-asyncssh-2.13.2-5.el9.noarch 441 kB/s | 548 kB 00:01 2024-04-02T16:59:05.283 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T16:59:05.335 INFO:teuthology.orchestra.run.smithi154.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-02T16:59:05.357 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T16:59:05.368 INFO:teuthology.orchestra.run.smithi154.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-02T16:59:05.403 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T16:59:05.471 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T16:59:05.519 INFO:teuthology.orchestra.run.smithi154.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-04-02T16:59:05.602 INFO:teuthology.orchestra.run.smithi154.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 868 kB/s | 203 kB 00:00 2024-04-02T16:59:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T16:59:05.685 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T16:59:05.728 INFO:teuthology.orchestra.run.smithi154.stdout:(124/174): python3-colorama-0.4.4-12.el9.noarch 90 kB/s | 35 kB 00:00 2024-04-02T16:59:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T16:59:05.818 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-02T16:59:05.840 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T16:59:05.869 INFO:teuthology.orchestra.run.smithi154.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 541 kB/s | 144 kB 00:00 2024-04-02T16:59:05.887 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T16:59:05.928 INFO:teuthology.orchestra.run.smithi154.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-04-02T16:59:05.987 INFO:teuthology.orchestra.run.smithi154.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 398 kB/s | 23 kB 00:00 2024-04-02T16:59:06.045 INFO:teuthology.orchestra.run.smithi154.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-04-02T16:59:06.073 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T16:59:06.104 INFO:teuthology.orchestra.run.smithi154.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 338 kB/s | 20 kB 00:00 2024-04-02T16:59:06.120 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T16:59:06.158 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T16:59:06.281 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T16:59:06.339 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T16:59:06.400 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T16:59:06.438 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T16:59:06.563 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T16:59:06.633 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T16:59:06.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T16:59:06.932 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T16:59:07.000 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T16:59:07.181 INFO:teuthology.orchestra.run.smithi154.stdout:(130/174): python3-jaraco-8.2.1-3.el9.noarch.rp 7.3 kB/s | 11 kB 00:01 2024-04-02T16:59:07.215 INFO:teuthology.orchestra.run.smithi154.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 941 kB/s | 1.0 MB 00:01 2024-04-02T16:59:07.274 INFO:teuthology.orchestra.run.smithi154.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-02T16:59:07.390 INFO:teuthology.orchestra.run.smithi154.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 495 kB/s | 58 kB 00:00 2024-04-02T16:59:07.457 INFO:teuthology.orchestra.run.smithi154.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-02T16:59:07.508 INFO:teuthology.orchestra.run.smithi154.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-02T16:59:07.566 INFO:teuthology.orchestra.run.smithi154.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-02T16:59:07.715 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T16:59:07.821 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T16:59:07.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T16:59:07.964 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:07.991 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:08.034 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T16:59:08.091 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T16:59:08.117 INFO:teuthology.orchestra.run.smithi154.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 51 kB/s | 28 kB 00:00 2024-04-02T16:59:08.145 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T16:59:08.186 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T16:59:08.390 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:59:08.391 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:59:08.405 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T16:59:08.406 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T16:59:08.407 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T16:59:08.408 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T16:59:08.411 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:59:08.414 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T16:59:08.415 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T16:59:08.467 INFO:teuthology.orchestra.run.smithi154.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 88 kB/s | 31 kB 00:00 2024-04-02T16:59:08.492 INFO:teuthology.orchestra.run.smithi154.stdout:(139/174): python3-logutils-0.3.5-21.el9.noarch 35 kB/s | 46 kB 00:01 2024-04-02T16:59:08.499 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-02T16:59:08.527 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-02T16:59:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-02T16:59:08.527 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-02T16:59:08.527 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T16:59:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T16:59:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T16:59:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T16:59:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T16:59:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T16:59:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T16:59:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T16:59:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T16:59:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T16:59:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T16:59:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T16:59:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T16:59:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T16:59:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:Install 150 Packages 2024-04-02T16:59:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 24 Packages 2024-04-02T16:59:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T16:59:08.543 INFO:teuthology.orchestra.run.smithi154.stdout:(140/174): python3-grpcio-1.46.7-10.el9.x86_64. 692 kB/s | 2.0 MB 00:03 2024-04-02T16:59:08.544 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 226 M 2024-04-02T16:59:08.544 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-02T16:59:08.559 INFO:teuthology.orchestra.run.smithi154.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-02T16:59:08.655 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T16:59:08.735 INFO:teuthology.orchestra.run.smithi154.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 291 kB/s | 51 kB 00:00 2024-04-02T16:59:08.793 INFO:teuthology.orchestra.run.smithi154.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-02T16:59:08.843 INFO:teuthology.orchestra.run.smithi154.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-02T16:59:08.859 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T16:59:08.934 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T16:59:08.990 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T16:59:09.143 INFO:teuthology.orchestra.run.smithi154.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-04-02T16:59:09.200 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T16:59:09.202 INFO:teuthology.orchestra.run.smithi154.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-02T16:59:09.255 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T16:59:09.333 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T16:59:09.428 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T16:59:09.472 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T16:59:09.528 INFO:teuthology.orchestra.run.smithi164.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-02T16:59:09.536 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T16:59:09.609 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T16:59:09.653 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T16:59:09.685 INFO:teuthology.orchestra.run.smithi154.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 396 kB/s | 192 kB 00:00 2024-04-02T16:59:09.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T16:59:09.784 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T16:59:09.877 INFO:teuthology.orchestra.run.smithi154.stdout:(148/174): python3-tempora-5.0.0-2.el9.noarch.r 27 kB/s | 36 kB 00:01 2024-04-02T16:59:09.903 INFO:teuthology.orchestra.run.smithi154.stdout:(149/174): python3-rsa-4.9-2.el9.noarch.rpm 42 kB/s | 59 kB 00:01 2024-04-02T16:59:09.988 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T16:59:10.041 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T16:59:10.061 INFO:teuthology.orchestra.run.smithi154.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-04-02T16:59:10.087 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T16:59:10.136 INFO:teuthology.orchestra.run.smithi154.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 31 MB/s | 2.3 MB 00:00 2024-04-02T16:59:10.148 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T16:59:10.212 INFO:teuthology.orchestra.run.smithi154.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T16:59:10.237 INFO:teuthology.orchestra.run.smithi154.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-04-02T16:59:10.262 INFO:teuthology.orchestra.run.smithi164.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-02T16:59:10.279 INFO:teuthology.orchestra.run.smithi154.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T16:59:10.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T16:59:10.321 INFO:teuthology.orchestra.run.smithi154.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-02T16:59:10.336 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T16:59:10.383 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T16:59:10.404 INFO:teuthology.orchestra.run.smithi154.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 856 kB/s | 71 kB 00:00 2024-04-02T16:59:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T16:59:10.446 INFO:teuthology.orchestra.run.smithi154.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 877 kB/s | 36 kB 00:00 2024-04-02T16:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T16:59:10.462 INFO:teuthology.orchestra.run.smithi164.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 724 kB/s | 145 kB 00:00 2024-04-02T16:59:10.530 INFO:teuthology.orchestra.run.smithi154.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-04-02T16:59:10.614 INFO:teuthology.orchestra.run.smithi154.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-04-02T16:59:10.614 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T16:59:10.655 INFO:teuthology.orchestra.run.smithi154.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 555 kB/s | 23 kB 00:00 2024-04-02T16:59:10.670 INFO:teuthology.orchestra.run.smithi164.stdout:(4/174): ceph-common-19.0.0-2609.g191d963a.el9. 15 MB/s | 21 MB 00:01 2024-04-02T16:59:10.715 INFO:teuthology.orchestra.run.smithi154.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-02T16:59:10.773 INFO:teuthology.orchestra.run.smithi154.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-04-02T16:59:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 14 MB/s | 1.5 MB 00:00 2024-04-02T16:59:10.799 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T16:59:10.832 INFO:teuthology.orchestra.run.smithi154.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-04-02T16:59:10.874 INFO:teuthology.orchestra.run.smithi154.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-04-02T16:59:10.915 INFO:teuthology.orchestra.run.smithi154.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 694 kB/s | 29 kB 00:00 2024-04-02T16:59:10.940 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T16:59:10.995 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T16:59:11.057 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T16:59:11.087 INFO:teuthology.orchestra.run.smithi164.stdout:(6/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 15 MB/s | 4.7 MB 00:00 2024-04-02T16:59:11.091 INFO:teuthology.orchestra.run.smithi154.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 6.9 MB/s | 1.2 MB 00:00 2024-04-02T16:59:11.142 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T16:59:11.299 INFO:teuthology.orchestra.run.smithi154.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 10 MB/s | 2.2 MB 00:00 2024-04-02T16:59:11.395 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T16:59:11.457 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T16:59:11.483 INFO:teuthology.orchestra.run.smithi154.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-04-02T16:59:11.541 INFO:teuthology.orchestra.run.smithi154.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-04-02T16:59:11.583 INFO:teuthology.orchestra.run.smithi154.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 748 kB/s | 31 kB 00:00 2024-04-02T16:59:11.632 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T16:59:11.688 INFO:teuthology.orchestra.run.smithi164.stdout:(7/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-02T16:59:11.704 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T16:59:11.738 INFO:teuthology.orchestra.run.smithi164.stdout:(8/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-02T16:59:11.745 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T16:59:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:59:11.801 INFO:teuthology.orchestra.run.smithi154.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.9 MB/s | 1.1 MB 00:00 2024-04-02T16:59:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T16:59:11.830 INFO:teuthology.orchestra.run.smithi164.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 274 kB/s | 25 kB 00:00 2024-04-02T16:59:11.851 INFO:teuthology.orchestra.run.smithi154.stdout:(172/174): librbd1-19.0.0-2609.g191d963a.el9.x8 1.6 MB/s | 3.1 MB 00:01 2024-04-02T16:59:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T16:59:11.884 INFO:teuthology.orchestra.run.smithi154.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 893 kB/s | 74 kB 00:00 2024-04-02T16:59:11.941 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T16:59:11.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T16:59:12.045 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T16:59:12.072 INFO:teuthology.orchestra.run.smithi164.stdout:(10/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 17 MB/s | 17 MB 00:00 2024-04-02T16:59:12.139 INFO:teuthology.orchestra.run.smithi164.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.3 MB/s | 225 kB 00:00 2024-04-02T16:59:12.151 INFO:teuthology.orchestra.run.smithi154.stdout:(174/174): librados2-19.0.0-2609.g191d963a.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-02T16:59:12.156 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:59:12.156 INFO:teuthology.orchestra.run.smithi154.stdout:Total 9.5 MB/s | 226 MB 00:23 2024-04-02T16:59:12.180 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T16:59:12.189 INFO:teuthology.orchestra.run.smithi164.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 542 kB/s | 27 kB 00:00 2024-04-02T16:59:12.225 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T16:59:12.239 INFO:teuthology.orchestra.run.smithi164.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 634 kB/s | 32 kB 00:00 2024-04-02T16:59:12.267 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T16:59:12.306 INFO:teuthology.orchestra.run.smithi164.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 11 MB/s | 747 kB 00:00 2024-04-02T16:59:12.334 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T16:59:12.348 INFO:teuthology.orchestra.run.smithi164.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 3.8 MB/s | 164 kB 00:00 2024-04-02T16:59:12.382 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T16:59:12.398 INFO:teuthology.orchestra.run.smithi164.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 2.5 MB/s | 127 kB 00:00 2024-04-02T16:59:12.457 INFO:teuthology.orchestra.run.smithi164.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 8.4 MB/s | 500 kB 00:00 2024-04-02T16:59:12.520 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-04-02T16:59:12.529 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T16:59:12.657 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T16:59:12.716 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T16:59:12.765 INFO:teuthology.orchestra.run.smithi164.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 17 MB/s | 5.1 MB 00:00 2024-04-02T16:59:12.785 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T16:59:12.793 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-04-02T16:59:12.793 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-04-02T16:59:12.799 INFO:teuthology.orchestra.run.smithi164.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 1.3 MB/s | 45 kB 00:00 2024-04-02T16:59:12.849 INFO:teuthology.orchestra.run.smithi164.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.6 MB/s | 134 kB 00:00 2024-04-02T16:59:12.900 INFO:teuthology.orchestra.run.smithi164.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 3.2 MB/s | 161 kB 00:00 2024-04-02T16:59:12.967 INFO:teuthology.orchestra.run.smithi164.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 4.7 MB/s | 320 kB 00:00 2024-04-02T16:59:13.017 INFO:teuthology.orchestra.run.smithi164.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 5.9 MB/s | 299 kB 00:00 2024-04-02T16:59:13.067 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T16:59:13.067 INFO:teuthology.orchestra.run.smithi164.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.9 MB/s | 99 kB 00:00 2024-04-02T16:59:13.118 INFO:teuthology.orchestra.run.smithi164.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-02T16:59:13.218 INFO:teuthology.orchestra.run.smithi164.stdout:(26/174): ceph-radosgw-19.0.0-2609.g191d963a.el 6.4 MB/s | 9.8 MB 00:01 2024-04-02T16:59:13.301 INFO:teuthology.orchestra.run.smithi164.stdout:(27/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 17 MB/s | 3.1 MB 00:00 2024-04-02T16:59:13.326 INFO:teuthology.orchestra.run.smithi164.stdout:(28/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-02T16:59:13.343 INFO:teuthology.orchestra.run.smithi164.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2609.g 814 kB/s | 34 kB 00:00 2024-04-02T16:59:13.385 INFO:teuthology.orchestra.run.smithi164.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 2.4 MB/s | 141 kB 00:00 2024-04-02T16:59:13.454 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T16:59:13.535 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T16:59:13.569 INFO:teuthology.orchestra.run.smithi164.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 16 MB/s | 3.5 MB 00:00 2024-04-02T16:59:13.620 INFO:teuthology.orchestra.run.smithi164.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 4.9 MB/s | 249 kB 00:00 2024-04-02T16:59:13.670 INFO:teuthology.orchestra.run.smithi164.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 995 kB/s | 49 kB 00:00 2024-04-02T16:59:13.728 INFO:teuthology.orchestra.run.smithi164.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 282 kB/s | 16 kB 00:00 2024-04-02T16:59:13.795 INFO:teuthology.orchestra.run.smithi164.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.9 MB/s | 264 kB 00:00 2024-04-02T16:59:13.879 INFO:teuthology.orchestra.run.smithi164.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-02T16:59:13.938 INFO:teuthology.orchestra.run.smithi164.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 313 kB/s | 18 kB 00:00 2024-04-02T16:59:14.113 INFO:teuthology.orchestra.run.smithi164.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-04-02T16:59:14.171 INFO:teuthology.orchestra.run.smithi164.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-04-02T16:59:14.205 INFO:teuthology.orchestra.run.smithi164.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-02T16:59:14.247 INFO:teuthology.orchestra.run.smithi164.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-02T16:59:14.281 INFO:teuthology.orchestra.run.smithi164.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 950 kB/s | 32 kB 00:00 2024-04-02T16:59:14.372 INFO:teuthology.orchestra.run.smithi164.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-02T16:59:14.464 INFO:teuthology.orchestra.run.smithi164.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-02T16:59:14.490 INFO:teuthology.orchestra.run.smithi164.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 146 kB/s | 17 kB 00:00 2024-04-02T16:59:14.502 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-04-02T16:59:14.502 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-04-02T16:59:14.515 INFO:teuthology.orchestra.run.smithi164.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-04-02T16:59:14.532 INFO:teuthology.orchestra.run.smithi164.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-02T16:59:14.547 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T16:59:14.551 INFO:teuthology.orchestra.run.smithi164.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.2 MB/s | 44 kB 00:00 2024-04-02T16:59:14.585 INFO:teuthology.orchestra.run.smithi164.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 12 MB/s | 661 kB 00:00 2024-04-02T16:59:14.597 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T16:59:14.610 INFO:teuthology.orchestra.run.smithi164.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-02T16:59:14.627 INFO:teuthology.orchestra.run.smithi164.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-02T16:59:14.630 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T16:59:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.5 MB/s | 282 kB 00:00 2024-04-02T16:59:14.678 INFO:teuthology.orchestra.run.smithi164.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2024-04-02T16:59:14.687 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T16:59:14.704 INFO:teuthology.orchestra.run.smithi164.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-02T16:59:14.720 INFO:teuthology.orchestra.run.smithi164.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 824 kB/s | 34 kB 00:00 2024-04-02T16:59:14.802 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T16:59:14.848 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T16:59:14.854 INFO:teuthology.orchestra.run.smithi164.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-02T16:59:14.887 INFO:teuthology.orchestra.run.smithi164.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-04-02T16:59:14.921 INFO:teuthology.orchestra.run.smithi164.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-02T16:59:14.946 INFO:teuthology.orchestra.run.smithi164.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-02T16:59:15.079 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T16:59:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T16:59:15.155 INFO:teuthology.orchestra.run.smithi164.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-04-02T16:59:15.197 INFO:teuthology.orchestra.run.smithi164.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.1 MB/s | 1.0 MB 00:00 2024-04-02T16:59:15.197 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T16:59:15.231 INFO:teuthology.orchestra.run.smithi164.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 78 kB 00:00 2024-04-02T16:59:15.423 INFO:teuthology.orchestra.run.smithi164.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-04-02T16:59:15.448 INFO:teuthology.orchestra.run.smithi164.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 113 kB/s | 24 kB 00:00 2024-04-02T16:59:15.473 INFO:teuthology.orchestra.run.smithi164.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-04-02T16:59:15.498 INFO:teuthology.orchestra.run.smithi164.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-02T16:59:15.524 INFO:teuthology.orchestra.run.smithi164.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-04-02T16:59:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-04-02T16:59:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-04-02T16:59:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-02T16:59:15.758 INFO:teuthology.orchestra.run.smithi164.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-02T16:59:15.783 INFO:teuthology.orchestra.run.smithi164.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-04-02T16:59:15.809 INFO:teuthology.orchestra.run.smithi164.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-02T16:59:15.834 INFO:teuthology.orchestra.run.smithi164.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-02T16:59:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-02T16:59:15.910 INFO:teuthology.orchestra.run.smithi164.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-04-02T16:59:15.935 INFO:teuthology.orchestra.run.smithi164.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-04-02T16:59:16.030 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T16:59:16.110 INFO:teuthology.orchestra.run.smithi164.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 222 kB/s | 39 kB 00:00 2024-04-02T16:59:16.177 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T16:59:16.352 INFO:teuthology.orchestra.run.smithi164.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-02T16:59:16.377 INFO:teuthology.orchestra.run.smithi164.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 155 kB/s | 41 kB 00:00 2024-04-02T16:59:16.403 INFO:teuthology.orchestra.run.smithi164.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-02T16:59:16.428 INFO:teuthology.orchestra.run.smithi164.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 697 kB/s | 35 kB 00:00 2024-04-02T16:59:16.453 INFO:teuthology.orchestra.run.smithi164.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 927 kB/s | 46 kB 00:00 2024-04-02T16:59:16.478 INFO:teuthology.orchestra.run.smithi164.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-02T16:59:16.504 INFO:teuthology.orchestra.run.smithi164.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T16:59:16.529 INFO:teuthology.orchestra.run.smithi164.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T16:59:16.554 INFO:teuthology.orchestra.run.smithi164.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-04-02T16:59:16.571 INFO:teuthology.orchestra.run.smithi164.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-04-02T16:59:16.597 INFO:teuthology.orchestra.run.smithi164.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-04-02T16:59:16.638 INFO:teuthology.orchestra.run.smithi164.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-04-02T16:59:16.664 INFO:teuthology.orchestra.run.smithi164.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-04-02T16:59:16.689 INFO:teuthology.orchestra.run.smithi164.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-02T16:59:16.715 INFO:teuthology.orchestra.run.smithi164.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T16:59:16.798 INFO:teuthology.orchestra.run.smithi164.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-04-02T16:59:16.924 INFO:teuthology.orchestra.run.smithi164.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-02T16:59:16.957 INFO:teuthology.orchestra.run.smithi164.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-02T16:59:17.032 INFO:teuthology.orchestra.run.smithi164.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-02T16:59:17.116 INFO:teuthology.orchestra.run.smithi164.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 135 kB/s | 21 kB 00:00 2024-04-02T16:59:17.141 INFO:teuthology.orchestra.run.smithi164.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-04-02T16:59:17.225 INFO:teuthology.orchestra.run.smithi164.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.9 MB/s | 245 kB 00:00 2024-04-02T16:59:17.276 INFO:teuthology.orchestra.run.smithi164.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 4.8 MB/s | 248 kB 00:00 2024-04-02T16:59:17.343 INFO:teuthology.orchestra.run.smithi164.stdout:(102/174): openssl-devel-3.0.7-27.el9.x86_64.rp 18 MB/s | 4.1 MB 00:00 2024-04-02T16:59:17.368 INFO:teuthology.orchestra.run.smithi164.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-04-02T16:59:17.763 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T16:59:17.890 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T16:59:17.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T16:59:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T16:59:17.999 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T16:59:18.396 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T16:59:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:18.535 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:18.569 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T16:59:18.627 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T16:59:19.176 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T16:59:19.207 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:19.343 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:19.368 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T16:59:19.468 INFO:teuthology.orchestra.run.smithi164.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 10 kB/s | 22 kB 00:02 2024-04-02T16:59:20.135 INFO:teuthology.orchestra.run.smithi164.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 823 kB/s | 548 kB 00:00 2024-04-02T16:59:20.277 INFO:teuthology.orchestra.run.smithi164.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-04-02T16:59:20.393 INFO:teuthology.orchestra.run.smithi164.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-02T16:59:20.811 INFO:teuthology.orchestra.run.smithi164.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-02T16:59:20.944 INFO:teuthology.orchestra.run.smithi164.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 243 kB/s | 26 kB 00:00 2024-04-02T16:59:21.061 INFO:teuthology.orchestra.run.smithi164.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 420 kB/s | 49 kB 00:00 2024-04-02T16:59:21.169 INFO:teuthology.orchestra.run.smithi164.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-04-02T16:59:21.278 INFO:teuthology.orchestra.run.smithi164.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-02T16:59:21.319 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-04-02T16:59:21.404 INFO:teuthology.orchestra.run.smithi164.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-02T16:59:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T16:59:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T16:59:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:21.520 INFO:teuthology.orchestra.run.smithi164.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-02T16:59:21.552 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T16:59:21.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T16:59:21.629 INFO:teuthology.orchestra.run.smithi164.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-02T16:59:21.640 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T16:59:21.674 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T16:59:21.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T16:59:21.737 INFO:teuthology.orchestra.run.smithi164.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-02T16:59:21.854 INFO:teuthology.orchestra.run.smithi164.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-02T16:59:21.963 INFO:teuthology.orchestra.run.smithi164.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-02T16:59:22.047 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T16:59:22.072 INFO:teuthology.orchestra.run.smithi164.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-02T16:59:22.093 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:22.099 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:22.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:22.180 INFO:teuthology.orchestra.run.smithi164.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-02T16:59:22.221 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:22.259 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:22.289 INFO:teuthology.orchestra.run.smithi164.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-02T16:59:22.374 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:22.398 INFO:teuthology.orchestra.run.smithi164.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-02T16:59:22.405 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:59:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T16:59:22.565 INFO:teuthology.orchestra.run.smithi164.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-02T16:59:22.611 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T16:59:22.669 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T16:59:22.689 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:22.690 INFO:teuthology.orchestra.run.smithi164.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-02T16:59:22.789 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:22.793 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:22.798 INFO:teuthology.orchestra.run.smithi164.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-02T16:59:22.815 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:22.834 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:22.862 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:22.902 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T16:59:22.907 INFO:teuthology.orchestra.run.smithi164.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-02T16:59:22.925 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:23.016 INFO:teuthology.orchestra.run.smithi164.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-02T16:59:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:23.018 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T16:59:23.018 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T16:59:23.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:23.059 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:59:23.124 INFO:teuthology.orchestra.run.smithi164.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-04-02T16:59:23.133 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:23.158 INFO:teuthology.orchestra.run.smithi164.stdout:(129/174): protobuf-compiler-3.14.0-13.el9.x86_ 149 kB/s | 863 kB 00:05 2024-04-02T16:59:23.164 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:23.195 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T16:59:23.233 INFO:teuthology.orchestra.run.smithi164.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-02T16:59:23.292 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T16:59:23.342 INFO:teuthology.orchestra.run.smithi164.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-04-02T16:59:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:23.450 INFO:teuthology.orchestra.run.smithi164.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2024-04-02T16:59:23.499 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:23.543 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:23.559 INFO:teuthology.orchestra.run.smithi164.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 533 kB/s | 58 kB 00:00 2024-04-02T16:59:23.668 INFO:teuthology.orchestra.run.smithi164.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-02T16:59:23.747 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:23.776 INFO:teuthology.orchestra.run.smithi164.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-04-02T16:59:23.856 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T16:59:23.885 INFO:teuthology.orchestra.run.smithi164.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 831 kB/s | 90 kB 00:00 2024-04-02T16:59:23.919 INFO:teuthology.orchestra.run.smithi164.stdout:(137/174): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-04-02T16:59:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T16:59:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T16:59:23.994 INFO:teuthology.orchestra.run.smithi164.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-02T16:59:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T16:59:24.027 INFO:teuthology.orchestra.run.smithi164.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-02T16:59:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T16:59:24.103 INFO:teuthology.orchestra.run.smithi164.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-02T16:59:24.108 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T16:59:24.136 INFO:teuthology.orchestra.run.smithi164.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-04-02T16:59:24.171 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T16:59:24.209 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T16:59:24.211 INFO:teuthology.orchestra.run.smithi164.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-02T16:59:24.245 INFO:teuthology.orchestra.run.smithi164.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-02T16:59:24.272 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T16:59:24.308 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T16:59:24.320 INFO:teuthology.orchestra.run.smithi164.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-02T16:59:24.346 INFO:teuthology.orchestra.run.smithi164.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-04-02T16:59:24.352 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T16:59:24.384 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T16:59:24.412 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T16:59:24.437 INFO:teuthology.orchestra.run.smithi164.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-02T16:59:24.463 INFO:teuthology.orchestra.run.smithi164.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-02T16:59:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T16:59:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T16:59:24.546 INFO:teuthology.orchestra.run.smithi164.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-02T16:59:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T16:59:24.581 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T16:59:24.650 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T16:59:24.680 INFO:teuthology.orchestra.run.smithi164.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-02T16:59:24.695 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T16:59:24.770 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T16:59:24.902 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T16:59:24.955 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T16:59:24.997 INFO:teuthology.orchestra.run.smithi164.stdout:(150/174): librbd1-19.0.0-2609.g191d963a.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-04-02T16:59:25.080 INFO:teuthology.orchestra.run.smithi164.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-04-02T16:59:25.147 INFO:teuthology.orchestra.run.smithi164.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-02T16:59:25.206 INFO:teuthology.orchestra.run.smithi164.stdout:(153/174): librados2-19.0.0-2609.g191d963a.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-02T16:59:25.232 INFO:teuthology.orchestra.run.smithi164.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 488 kB/s | 41 kB 00:00 2024-04-02T16:59:25.257 INFO:teuthology.orchestra.run.smithi164.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-04-02T16:59:25.282 INFO:teuthology.orchestra.run.smithi164.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-02T16:59:25.341 INFO:teuthology.orchestra.run.smithi164.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 859 kB/s | 71 kB 00:00 2024-04-02T16:59:25.366 INFO:teuthology.orchestra.run.smithi164.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 436 kB/s | 36 kB 00:00 2024-04-02T16:59:25.441 INFO:teuthology.orchestra.run.smithi164.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 192 kB 00:00 2024-04-02T16:59:25.473 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T16:59:25.483 INFO:teuthology.orchestra.run.smithi164.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 552 kB/s | 23 kB 00:00 2024-04-02T16:59:25.509 INFO:teuthology.orchestra.run.smithi164.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-04-02T16:59:25.551 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T16:59:25.601 INFO:teuthology.orchestra.run.smithi164.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 928 kB/s | 108 kB 00:00 2024-04-02T16:59:25.626 INFO:teuthology.orchestra.run.smithi164.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-04-02T16:59:25.638 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T16:59:25.671 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:25.680 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:25.693 INFO:teuthology.orchestra.run.smithi164.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 961 kB/s | 64 kB 00:00 2024-04-02T16:59:25.711 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T16:59:25.735 INFO:teuthology.orchestra.run.smithi164.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-04-02T16:59:25.751 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T16:59:25.760 INFO:teuthology.orchestra.run.smithi164.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 429 kB/s | 29 kB 00:00 2024-04-02T16:59:25.790 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T16:59:25.819 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T16:59:25.852 INFO:teuthology.orchestra.run.smithi164.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-02T16:59:25.960 INFO:teuthology.orchestra.run.smithi164.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-02T16:59:26.059 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T16:59:26.145 INFO:teuthology.orchestra.run.smithi164.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-04-02T16:59:26.234 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T16:59:26.288 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T16:59:26.332 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T16:59:26.469 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T16:59:26.510 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T16:59:26.565 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T16:59:26.620 INFO:teuthology.orchestra.run.smithi164.stdout:(170/174): ceph-test-19.0.0-2609.g191d963a.el9. 3.2 MB/s | 48 MB 00:14 2024-04-02T16:59:26.638 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T16:59:26.645 INFO:teuthology.orchestra.run.smithi164.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 62 kB/s | 31 kB 00:00 2024-04-02T16:59:26.669 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T16:59:26.705 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T16:59:26.745 INFO:teuthology.orchestra.run.smithi164.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 745 kB/s | 74 kB 00:00 2024-04-02T16:59:26.753 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T16:59:26.787 INFO:teuthology.orchestra.run.smithi164.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.4 MB/s | 2.3 MB 00:00 2024-04-02T16:59:26.788 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T16:59:26.831 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T16:59:26.874 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T16:59:26.937 INFO:teuthology.orchestra.run.smithi164.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.4 MB/s | 1.1 MB 00:00 2024-04-02T16:59:26.942 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:59:26.942 INFO:teuthology.orchestra.run.smithi164.stdout:Total 12 MB/s | 226 MB 00:18 2024-04-02T16:59:27.007 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T16:59:27.053 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T16:59:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T16:59:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T16:59:27.229 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T16:59:27.267 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T16:59:27.307 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T16:59:27.309 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-02T16:59:27.333 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T16:59:27.357 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T16:59:27.429 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T16:59:27.550 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T16:59:27.579 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-02T16:59:27.579 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-02T16:59:27.649 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T16:59:27.690 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T16:59:27.738 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T16:59:27.794 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T16:59:27.978 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T16:59:28.020 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T16:59:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T16:59:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T16:59:28.225 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T16:59:28.256 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:59:28.284 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T16:59:28.318 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T16:59:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T16:59:28.388 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T16:59:28.418 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T16:59:28.486 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T16:59:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T16:59:28.562 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T16:59:28.620 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T16:59:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T16:59:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T16:59:28.825 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T16:59:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T16:59:28.909 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T16:59:28.955 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T16:59:29.052 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T16:59:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T16:59:29.280 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-02T16:59:29.280 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-02T16:59:29.832 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T16:59:29.882 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T16:59:29.916 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T16:59:29.964 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T16:59:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T16:59:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T16:59:30.261 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T16:59:30.289 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T16:59:30.367 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T16:59:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T16:59:31.120 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T16:59:32.337 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T16:59:32.414 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T16:59:32.445 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T16:59:32.476 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T16:59:32.504 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T16:59:32.825 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T16:59:32.859 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:32.950 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:32.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T16:59:33.024 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T16:59:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T16:59:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:33.582 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:33.605 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T16:59:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-02T16:59:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T16:59:36.352 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T16:59:36.381 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:36.391 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:36.400 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:59:36.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:36.449 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T16:59:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T16:59:36.503 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:36.503 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T16:59:36.503 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T16:59:36.503 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T16:59:36.539 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:59:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T16:59:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T16:59:36.632 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:59:36.632 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-04-02T16:59:36.632 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-04-02T16:59:36.632 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-04-02T16:59:36.632 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-04-02T16:59:36.633 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-04-02T16:59:36.633 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-04-02T16:59:36.633 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-04-02T16:59:36.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:36.863 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:59:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:59:36.913 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:36.913 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T16:59:36.913 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T16:59:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:37.101 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:59:37.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:37.213 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:59:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:59:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T16:59:37.443 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T16:59:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T16:59:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:37.614 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:59:37.637 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:59:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T16:59:37.772 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:37.786 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:59:37.980 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:38.011 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:59:38.043 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T16:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T16:59:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:59:38.356 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:38.398 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:38.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:59:38.608 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:59:38.666 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:59:38.666 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:38.667 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T16:59:38.667 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:59:38.667 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:59:38.667 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:38.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T16:59:38.933 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:59:38.959 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:59:39.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:39.054 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T16:59:39.080 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T16:59:39.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T16:59:39.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T16:59:39.244 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T16:59:39.291 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T16:59:39.355 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T16:59:39.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T16:59:39.433 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T16:59:39.466 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T16:59:39.494 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T16:59:39.525 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:59:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T16:59:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:59:39.600 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:39.642 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T16:59:39.671 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T16:59:39.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T16:59:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T16:59:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T16:59:39.885 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T16:59:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T16:59:40.071 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T16:59:40.236 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:59:40.269 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:59:40.322 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T16:59:40.427 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T16:59:40.490 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:59:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T16:59:40.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T16:59:40.757 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T16:59:40.803 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:40.813 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:59:40.843 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T16:59:40.892 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T16:59:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T16:59:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T16:59:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T16:59:41.357 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T16:59:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T16:59:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T16:59:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T16:59:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T16:59:41.726 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T16:59:41.799 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T16:59:41.839 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:59:41.883 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:59:41.913 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T16:59:41.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T16:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T16:59:42.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T16:59:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T16:59:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T16:59:42.252 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T16:59:42.298 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T16:59:42.329 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T16:59:42.371 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T16:59:42.465 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T16:59:42.504 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T16:59:42.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T16:59:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T16:59:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T16:59:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T16:59:42.795 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T16:59:42.902 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T16:59:42.944 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T16:59:42.993 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T16:59:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:59:43.048 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T16:59:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:59:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:59:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T16:59:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T16:59:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T16:59:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:59:43.445 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T16:59:43.489 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T16:59:43.521 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:59:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:59:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T16:59:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:59:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T16:59:43.626 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:59:43.653 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T16:59:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T16:59:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T16:59:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T16:59:43.790 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T16:59:43.825 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T16:59:43.867 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T16:59:43.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T16:59:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T16:59:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T16:59:44.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T16:59:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T16:59:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T16:59:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T16:59:44.342 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T16:59:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T16:59:44.760 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T16:59:44.828 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:59:44.902 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:59:44.903 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:44.903 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T16:59:44.903 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:59:44.903 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:59:44.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:44.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:59:45.051 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:59:45.051 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:45.051 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T16:59:45.052 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:45.125 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T16:59:45.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:59:45.171 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T16:59:45.204 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T16:59:45.237 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:59:45.238 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:45.238 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T16:59:45.238 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:59:45.238 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:59:45.238 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:45.253 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T16:59:45.341 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T16:59:45.372 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T16:59:45.550 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T16:59:45.578 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T16:59:45.624 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:59:45.656 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:59:45.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:46.292 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T16:59:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T16:59:47.616 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T16:59:47.713 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T16:59:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T16:59:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T16:59:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T16:59:48.117 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T16:59:48.140 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:48.165 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:59:48.165 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-04-02T16:59:48.165 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-04-02T16:59:48.165 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-04-02T16:59:48.165 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-04-02T16:59:48.166 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-04-02T16:59:48.166 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-04-02T16:59:48.166 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-04-02T16:59:48.166 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:59:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T16:59:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T16:59:48.352 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:59:48.368 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:59:48.368 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:48.368 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T16:59:48.368 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:48.761 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T16:59:48.794 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:48.873 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:59:48.895 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T16:59:49.813 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:59:49.839 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:50.064 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:59:50.086 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:59:50.087 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:50.087 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T16:59:50.087 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:59:50.087 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:59:50.087 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:50.523 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:59:50.546 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:50.650 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T16:59:50.788 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:59:50.842 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T16:59:50.878 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T16:59:50.917 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T16:59:51.093 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:59:51.123 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:59:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:51.155 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T16:59:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:51.220 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T16:59:51.224 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T16:59:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:59:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T16:59:51.396 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T16:59:51.464 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T16:59:51.521 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T16:59:51.565 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T16:59:51.600 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T16:59:51.636 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T16:59:51.690 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T16:59:51.709 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:59:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:51.783 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T16:59:51.826 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:59:51.826 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T16:59:51.826 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T16:59:51.826 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T16:59:51.855 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:59:51.903 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T16:59:51.936 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T16:59:52.284 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:59:52.323 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:59:52.906 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:59:52.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:59:53.027 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:59:53.285 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:59:53.317 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:59:53.417 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:59:53.579 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T16:59:53.610 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:59:53.634 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:53.678 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:59:53.689 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:59:53.734 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T16:59:54.446 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T16:59:54.555 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T16:59:54.606 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T16:59:54.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T16:59:54.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:59:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:59:54.678 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:54.678 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T16:59:54.678 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:59:54.678 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:59:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:59:54.745 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T16:59:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:59:54.752 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:54.752 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T16:59:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:54.784 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T16:59:54.798 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:59:54.817 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T16:59:54.820 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:59:54.820 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:54.820 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T16:59:54.820 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:59:54.820 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:59:54.821 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:54.852 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T16:59:54.893 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T16:59:54.893 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:59:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:59:55.080 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T16:59:55.081 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:59:55.132 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:59:55.154 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T16:59:55.218 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:59:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T16:59:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T16:59:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T16:59:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T16:59:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T16:59:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T16:59:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T16:59:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T16:59:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T16:59:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T16:59:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T16:59:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:59:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:59:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:59:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:59:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:59:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T16:59:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T16:59:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T16:59:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T16:59:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:59:56.742 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:59:56.743 INFO:teuthology.orchestra.run.smithi059.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-02T16:59:56.743 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T16:59:59.645 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:59:59.686 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T16:59:59.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T16:59:59.743 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T16:59:59.924 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T17:00:00.042 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T17:00:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T17:00:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T17:00:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T17:00:00.218 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T17:00:00.263 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T17:00:00.282 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T17:00:00.321 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T17:00:00.375 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T17:00:00.436 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T17:00:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T17:00:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T17:00:01.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T17:00:01.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T17:00:01.653 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T17:00:01.669 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T17:00:01.670 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T17:00:01.671 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T17:00:01.672 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T17:00:01.673 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T17:00:01.674 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T17:00:01.675 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T17:00:01.676 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T17:00:01.677 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T17:00:01.678 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T17:00:01.679 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T17:00:01.680 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T17:00:01.998 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T17:00:02.070 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T17:00:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T17:00:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T17:00:02.240 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T17:00:02.265 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T17:00:02.290 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T17:00:02.317 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T17:00:02.317 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T17:00:02.405 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T17:00:02.435 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T17:00:02.435 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T17:00:02.502 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T17:00:02.529 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T17:00:02.562 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T17:00:02.587 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T17:00:02.612 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T17:00:02.637 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T17:00:02.676 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T17:00:02.708 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T17:00:02.734 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T17:00:02.759 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T17:00:02.784 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T17:00:02.810 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T17:00:02.816 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:02.816 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:02.996 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:02.996 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T17:00:03.079 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T17:00:03.110 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T17:00:03.162 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T17:00:03.188 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T17:00:03.221 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T17:00:03.285 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-02T17:00:03.464 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-02T17:00:03.465 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-02T17:00:03.465 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-02T17:00:03.465 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T17:00:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T17:00:03.660 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:03.660 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T17:00:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T17:00:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-16.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T17:00:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T17:00:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T17:00:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T17:00:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T17:00:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T17:00:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T17:00:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T17:00:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:00:04.368 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-02T17:00:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T17:00:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T17:00:05.148 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:05.149 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T17:00:05.149 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T17:00:05.149 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T17:00:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:05.188 DEBUG:teuthology.parallel:result is None 2024-04-02T17:00:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T17:00:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:05.832 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T17:00:05.854 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T17:00:05.854 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:05.854 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T17:00:05.855 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T17:00:05.855 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T17:00:05.855 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T17:00:06.394 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T17:00:06.424 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T17:00:06.456 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T17:00:06.517 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T17:00:06.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T17:00:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T17:00:07.615 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T17:00:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T17:00:08.607 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T17:00:08.649 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T17:00:08.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T17:00:08.651 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T17:00:08.652 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T17:00:08.652 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T17:00:08.652 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T17:00:08.652 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T17:00:08.652 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T17:00:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T17:00:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T17:00:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T17:00:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T17:00:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T17:00:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T17:00:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T17:00:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T17:00:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T17:00:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T17:00:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T17:00:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T17:00:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T17:00:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T17:00:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T17:00:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T17:00:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T17:00:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T17:00:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T17:00:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T17:00:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T17:00:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T17:00:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T17:00:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T17:00:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T17:00:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T17:00:08.710 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T17:00:08.862 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T17:00:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T17:00:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T17:00:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T17:00:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T17:00:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T17:00:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T17:00:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T17:00:10.004 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T17:00:10.004 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:10.004 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T17:00:10.004 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:10.049 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T17:00:10.071 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T17:00:10.397 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T17:00:10.398 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T17:00:10.398 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T17:00:10.398 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T17:00:10.398 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T17:00:10.398 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:11.636 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T17:00:11.636 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T17:00:11.637 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.638 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T17:00:11.639 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.640 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.641 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T17:00:11.642 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-16.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T17:00:11.643 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T17:00:11.644 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T17:00:11.645 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T17:00:11.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T17:00:11.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T17:00:11.648 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T17:00:11.649 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T17:00:11.650 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:11.651 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-02T17:00:11.652 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-04-02T17:00:12.112 DEBUG:teuthology.parallel:result is None 2024-04-02T17:00:14.855 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T17:00:14.896 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T17:00:14.926 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T17:00:14.953 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T17:00:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T17:00:15.261 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T17:00:15.302 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T17:00:15.341 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T17:00:15.393 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T17:00:15.436 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T17:00:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T17:00:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T17:00:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T17:00:15.568 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T17:00:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T17:00:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T17:00:16.233 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:16.270 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:16.289 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T17:00:17.143 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T17:00:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T17:00:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T17:00:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T17:00:17.402 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T17:00:17.427 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T17:00:17.452 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T17:00:17.479 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T17:00:17.479 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T17:00:17.558 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T17:00:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T17:00:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T17:00:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T17:00:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T17:00:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T17:00:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T17:00:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T17:00:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T17:00:17.847 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T17:00:17.880 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T17:00:17.905 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T17:00:17.930 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T17:00:17.955 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T17:00:17.980 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T17:00:17.987 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:17.987 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T17:00:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T17:00:18.266 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T17:00:18.298 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T17:00:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T17:00:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T17:00:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T17:00:18.440 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T17:00:23.779 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T17:00:23.780 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T17:00:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T17:00:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T17:00:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T17:00:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T17:00:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T17:00:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T17:00:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T17:00:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T17:00:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T17:00:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T17:00:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T17:00:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T17:00:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T17:00:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T17:00:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T17:00:26.889 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T17:00:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:26.891 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T17:00:26.892 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.893 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T17:00:26.894 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: jq-1.6-16.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.895 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T17:00:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T17:00:26.897 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T17:00:26.898 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T17:00:26.899 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T17:00:26.900 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T17:00:26.901 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T17:00:26.902 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T17:00:26.903 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:00:26.904 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-02T17:00:27.352 DEBUG:teuthology.parallel:result is None 2024-04-02T17:00:27.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:27.570 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T17:00:27.647 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T17:00:27.648 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T17:00:27.648 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T17:00:27.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:27.867 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T17:00:27.893 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T17:00:27.893 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T17:00:27.893 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T17:00:27.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:28.038 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T17:00:28.060 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T17:00:28.061 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T17:00:28.061 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T17:00:28.063 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T17:00:28.063 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:00:28.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T17:00:28.173 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:00:28.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T17:00:28.209 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:00:28.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T17:00:28.242 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T17:00:28.242 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:00:28.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T17:00:28.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T17:00:28.486 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:00:28.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T17:00:28.518 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T17:00:28.588 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:00:28.588 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T17:00:28.620 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T17:00:28.692 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T17:00:28.692 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:00:28.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T17:00:28.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T17:00:28.941 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:00:28.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T17:00:28.974 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T17:00:29.043 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:00:29.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T17:00:29.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T17:00:29.146 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T17:00:29.147 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:00:29.147 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T17:00:29.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T17:00:29.353 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:00:29.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T17:00:29.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T17:00:29.456 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:00:29.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T17:00:29.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T17:00:29.560 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T17:00:29.663 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-02T17:00:29.663 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:29.663 INFO:tasks.cephadm:Cluster fsid is 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:00:29.664 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T17:00:29.664 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.59', 'mon.b': '172.21.15.154', 'mon.c': '172.21.15.164'} 2024-04-02T17:00:29.664 INFO:tasks.cephadm:First mon is mon.a on smithi059 2024-04-02T17:00:29.664 INFO:tasks.cephadm:First mgr is x 2024-04-02T17:00:29.664 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T17:00:29.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-04-02T17:00:29.771 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-04-02T17:00:29.803 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-04-02T17:00:29.835 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-02T17:00:29.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:30.051 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78154/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2609-g191d963a', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2609.g191d963a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-02 02:04:04.615852', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240402.004512', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-02T17:00:30.195 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240402.004512, sha1 191d963ac9de7322da8aff20b567b3fa458e2ef6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:00:30.196 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm 2024-04-02T17:00:30.197 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:30.741 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 17:00 /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:30.742 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:31.343 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 17:00 /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:31.344 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 17:00 /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:32.180 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:32.242 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:32.261 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T17:00:32.298 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 on all hosts... 2024-04-02T17:00:32.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T17:00:32.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T17:00:32.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T17:00:33.098 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T17:00:33.114 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T17:00:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-04-02T17:01:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-02T17:01:05.593 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-02T17:01:05.593 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T17:01:05.593 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T17:01:05.594 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-04-02T17:01:05.594 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T17:01:05.594 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-04-02T17:01:05.594 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-02T17:01:21.533 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:01:21.534 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:01:21.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-04-02T17:01:21.702 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-04-02T17:01:21.738 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-04-02T17:01:21.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-04-02T17:01:21.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-04-02T17:01:21.913 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-04-02T17:01:21.941 INFO:tasks.cephadm:Writing seed config... 2024-04-02T17:01:21.942 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T17:01:21.942 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T17:01:21.943 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T17:01:21.944 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-02T17:01:21.945 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-02T17:01:21.946 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-02T17:01:21.947 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:01:21.947 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T17:01:22.007 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8286be62-f112-11ee-b647-cb9ed24678a4 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T17:01:22.008 DEBUG:teuthology.orchestra.run.smithi059:mon.a> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service 2024-04-02T17:01:22.055 DEBUG:teuthology.orchestra.run.smithi059:mgr.x> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x.service 2024-04-02T17:01:22.100 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T17:01:22.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 -v bootstrap --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.59 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:01:22.382 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-04-02T17:01:22.383 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6', '-v', 'bootstrap', '--fsid', '8286be62-f112-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.59', '--skip-admin-label'] 2024-04-02T17:01:22.383 INFO:teuthology.orchestra.run.smithi059.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T17:01:22.383 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying podman|docker is present... 2024-04-02T17:01:22.443 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T17:01:22.444 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying lvm2 is present... 2024-04-02T17:01:22.444 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying time synchronization is in place... 2024-04-02T17:01:22.462 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T17:01:22.463 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T17:01:22.479 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T17:01:22.479 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:01:22.497 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-04-02T17:01:22.513 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-04-02T17:01:22.513 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-04-02T17:01:22.513 INFO:teuthology.orchestra.run.smithi059.stdout:Repeating the final host check... 2024-04-02T17:01:22.572 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T17:01:22.572 INFO:teuthology.orchestra.run.smithi059.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T17:01:22.572 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl is present 2024-04-02T17:01:22.572 INFO:teuthology.orchestra.run.smithi059.stdout:lvcreate is present 2024-04-02T17:01:22.587 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T17:01:22.587 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T17:01:22.604 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T17:01:22.604 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:01:22.628 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-04-02T17:01:22.650 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-04-02T17:01:22.650 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-04-02T17:01:22.650 INFO:teuthology.orchestra.run.smithi059.stdout:Host looks OK 2024-04-02T17:01:22.651 INFO:teuthology.orchestra.run.smithi059.stdout:Cluster fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:22.651 INFO:teuthology.orchestra.run.smithi059.stdout:Acquiring lock 140371598037680 on /run/cephadm/8286be62-f112-11ee-b647-cb9ed24678a4.lock 2024-04-02T17:01:22.652 INFO:teuthology.orchestra.run.smithi059.stdout:Lock 140371598037680 acquired on /run/cephadm/8286be62-f112-11ee-b647-cb9ed24678a4.lock 2024-04-02T17:01:22.653 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 3300 ... 2024-04-02T17:01:22.654 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 6789 ... 2024-04-02T17:01:22.655 INFO:teuthology.orchestra.run.smithi059.stdout:Base mon IP(s) is [172.21.15.59:3300, 172.21.15.59:6789], mon addrv is [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-04-02T17:01:22.669 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.59 metric 100 2024-04-02T17:01:22.669 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.59 metric 100 2024-04-02T17:01:22.682 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T17:01:22.682 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-02T17:01:22.682 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-02T17:01:22.693 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T17:01:22.693 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T17:01:22.693 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T17:01:22.693 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-02T17:01:22.693 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-02T17:01:22.694 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T17:01:22.696 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-04-02T17:01:22.696 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-04-02T17:01:22.696 INFO:teuthology.orchestra.run.smithi059.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T17:01:22.699 INFO:teuthology.orchestra.run.smithi059.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T17:01:22.702 INFO:teuthology.orchestra.run.smithi059.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:db8c11fadded9ebce76f2e720ef87f32e8913958c61c526c89a5e4e2c41a8200 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T17:01:23.390 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying config sha256:f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T17:01:23.391 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T17:01:24.527 INFO:teuthology.orchestra.run.smithi059.stdout:ceph: stdout ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T17:01:24.528 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph version: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T17:01:24.528 INFO:teuthology.orchestra.run.smithi059.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T17:01:25.611 INFO:teuthology.orchestra.run.smithi059.stdout:stat: stdout 167 167 2024-04-02T17:01:25.611 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial keys... 2024-04-02T17:01:26.693 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDmOQxmeDnqARAA2A82gddmURcoqq34ezcx6w== 2024-04-02T17:01:27.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDnOQxmayCSChAAJuo3z7xGj3AxMpRLLhOAUA== 2024-04-02T17:01:28.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDoOQxm4nzqDBAAtypyI3ksdlz1mo6YQ6Xk7w== 2024-04-02T17:01:28.857 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial monmap... 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool for a [v2:172.21.15.59:3300,v1:172.21.15.59:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-04-02T17:01:29.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: set fsid to 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:29.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T17:01:29.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:01:29.951 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mon... 2024-04-02T17:01:31.210 INFO:teuthology.orchestra.run.smithi059.stdout:create mon.a on 2024-04-02T17:01:31.568 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T17:01:31.814 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T17:01:32.129 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target. 2024-04-02T17:01:32.129 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target. 2024-04-02T17:01:32.508 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T17:01:32.508 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-04-02T17:01:32.790 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target.wants/ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4@.service. 2024-04-02T17:01:33.692 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T17:01:33.692 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-04-02T17:01:33.708 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T17:01:33.708 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:01:33.708 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-04-02T17:01:33.708 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T17:01:33.709 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon to start... 2024-04-02T17:01:33.709 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon... 2024-04-02T17:01:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:33 smithi059 systemd[1]: Started Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: monmap epoch 1 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: last_changed 2024-04-02T17:01:29.288578+0000 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:01:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: min_mon_release 19 (squid) 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: election_strategy: 1 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: fsmap 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: mgrmap e1: no daemons active 2024-04-02T17:01:34.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:34 smithi059 ceph-mon[31282]: from='client.? 172.21.15.59:0/2649515083' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout id: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout services: 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.886958s) 2024-04-02T17:01:35.245 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout data: 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.431+0000 7fbc79600700 1 Processor -- start 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.432+0000 7fbc79600700 1 -- start start 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.432+0000 7fbc79600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.432+0000 7fbc79600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74063870 con 0x7fbc741069f0 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.433+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:35.246 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.433+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48606/0 (socket says 172.21.15.59:48606) 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.433+0000 7fbc72a00700 1 -- 172.21.15.59:0/2195532592 learned_addr learned my addr 172.21.15.59:0/2195532592 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.433+0000 7fbc72a00700 1 -- 172.21.15.59:0/2195532592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc74107350 con 0x7fbc741069f0 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.434+0000 7fbc72a00700 1 --2- 172.21.15.59:0/2195532592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098a0 tx=0x7fbc64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd1ff474a5a9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.434+0000 7fbc71600700 1 -- 172.21.15.59:0/2195532592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc64004160 con 0x7fbc741069f0 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.434+0000 7fbc71600700 1 -- 172.21.15.59:0/2195532592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc64004300 con 0x7fbc741069f0 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.434+0000 7fbc71600700 1 -- 172.21.15.59:0/2195532592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc640036a0 con 0x7fbc741069f0 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 -- 172.21.15.59:0/2195532592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 msgr2=0x7fbc74106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 --2- 172.21.15.59:0/2195532592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098a0 tx=0x7fbc64009bb0 comp rx=0 tx=0).stop 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 -- 172.21.15.59:0/2195532592 shutdown_connections 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 --2- 172.21.15.59:0/2195532592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc74106e10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 -- 172.21.15.59:0/2195532592 >> 172.21.15.59:0/2195532592 conn(0x7fbc741025d0 msgr2=0x7fbc741049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 -- 172.21.15.59:0/2195532592 shutdown_connections 2024-04-02T17:01:35.247 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.435+0000 7fbc79600700 1 -- 172.21.15.59:0/2195532592 wait complete. 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.437+0000 7fbc79600700 1 Processor -- start 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.437+0000 7fbc79600700 1 -- start start 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.437+0000 7fbc79600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.437+0000 7fbc79600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74107880 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.438+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.438+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48612/0 (socket says 172.21.15.59:48612) 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.438+0000 7fbc72a00700 1 -- 172.21.15.59:0/2649515083 learned_addr learned my addr 172.21.15.59:0/2649515083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.439+0000 7fbc72a00700 1 -- 172.21.15.59:0/2649515083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7419dfd0 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.439+0000 7fbc72a00700 1 --2- 172.21.15.59:0/2649515083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.439+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc64003db0 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.439+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc64011630 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.439+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc64011950 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.440+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7419e260 con 0x7fbc741069f0 2024-04-02T17:01:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.440+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc741a0f50 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.440+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc6401a460 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.440+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc6401baf0 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.441+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc74063c20 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.444+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbc64004260 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.526+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fbc74196ff0 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.527+0000 7fbc6b400700 1 -- 172.21.15.59:0/2649515083 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fbc74196ff0 con 0x7fbc741069f0 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.528+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 msgr2=0x7fbc7419da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.528+0000 7fbc79600700 1 --2- 172.21.15.59:0/2649515083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64004750 comp rx=0 tx=0).stop 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.528+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 shutdown_connections 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.528+0000 7fbc79600700 1 --2- 172.21.15.59:0/2649515083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc741069f0 0x7fbc7419da90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.528+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 >> 172.21.15.59:0/2649515083 conn(0x7fbc741025d0 msgr2=0x7fbc741940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.529+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 shutdown_connections 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:34.529+0000 7fbc79600700 1 -- 172.21.15.59:0/2649515083 wait complete. 2024-04-02T17:01:35.249 INFO:teuthology.orchestra.run.smithi059.stdout:mon is available 2024-04-02T17:01:35.250 INFO:teuthology.orchestra.run.smithi059.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T17:01:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:35 smithi059 ceph-mon[31282]: from='client.? 172.21.15.59:0/655903849' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T17:01:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:35 smithi059 ceph-mon[31282]: from='client.? 172.21.15.59:0/655903849' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T17:01:36.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:36.589 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T17:01:36.595 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:36.595 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-04-02T17:01:36.595 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T17:01:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.794+0000 7f9c6d200700 1 Processor -- start 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.794+0000 7f9c6d200700 1 -- start start 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.795+0000 7f9c6d200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.795+0000 7f9c6d200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680638d0 con 0x7f9c68108ba0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.796+0000 7f9c65e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.796+0000 7f9c65e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48616/0 (socket says 172.21.15.59:48616) 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.796+0000 7f9c65e00700 1 -- 172.21.15.59:0/995162305 learned_addr learned my addr 172.21.15.59:0/995162305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.796+0000 7f9c65e00700 1 -- 172.21.15.59:0/995162305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6810b4d0 con 0x7f9c68108ba0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.797+0000 7f9c65e00700 1 --2- 172.21.15.59:0/995162305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009e80 tx=0x7f9c5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192cb39da6d62126 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.797+0000 7f9c64a00700 1 -- 172.21.15.59:0/995162305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58017070 con 0x7f9c68108ba0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.797+0000 7f9c64a00700 1 -- 172.21.15.59:0/995162305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c58003a20 con 0x7f9c68108ba0 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.797+0000 7f9c64a00700 1 -- 172.21.15.59:0/995162305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58003d20 con 0x7f9c68108ba0 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 -- 172.21.15.59:0/995162305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 msgr2=0x7f9c6810af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 --2- 172.21.15.59:0/995162305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009e80 tx=0x7f9c5800b330 comp rx=0 tx=0).stop 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 -- 172.21.15.59:0/995162305 shutdown_connections 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 --2- 172.21.15.59:0/995162305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c6810af90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 -- 172.21.15.59:0/995162305 >> 172.21.15.59:0/995162305 conn(0x7f9c681025d0 msgr2=0x7f9c681049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 -- 172.21.15.59:0/995162305 shutdown_connections 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.798+0000 7f9c6d200700 1 -- 172.21.15.59:0/995162305 wait complete. 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.799+0000 7f9c6d200700 1 Processor -- start 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.799+0000 7f9c6d200700 1 -- start start 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.800+0000 7f9c6d200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.800+0000 7f9c6d200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6810b940 con 0x7f9c68108ba0 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.800+0000 7f9c65e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:36.598 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.800+0000 7f9c65e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48620/0 (socket says 172.21.15.59:48620) 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.800+0000 7f9c65e00700 1 -- 172.21.15.59:0/655903849 learned_addr learned my addr 172.21.15.59:0/655903849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c65e00700 1 -- 172.21.15.59:0/655903849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c68108960 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c65e00700 1 --2- 172.21.15.59:0/655903849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9c68062d10 tx=0x7f9c58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58022020 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c58018040 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58017070 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c68106b50 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.801+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c68107090 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.802+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c58025020 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.802+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c5800f050 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.803+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c68063c20 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.805+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9c5802c030 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.882+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9c681951b0 con 0x7f9c68108ba0 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.893+0000 7f9c5ea00700 1 -- 172.21.15.59:0/655903849 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f9c5801e480 con 0x7f9c68108ba0 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 msgr2=0x7f9c68108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 --2- 172.21.15.59:0/655903849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9c68062d10 tx=0x7f9c58004ad0 comp rx=0 tx=0).stop 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 shutdown_connections 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 --2- 172.21.15.59:0/655903849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c68108ba0 0x7f9c68108420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 >> 172.21.15.59:0/655903849 conn(0x7f9c681025d0 msgr2=0x7f9c681029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 shutdown_connections 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:35.895+0000 7f9c6d200700 1 -- 172.21.15.59:0/655903849 wait complete. 2024-04-02T17:01:36.600 INFO:teuthology.orchestra.run.smithi059.stdout:Generating new minimal ceph.conf... 2024-04-02T17:01:46.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:46 smithi059 ceph-mon[31282]: from='client.? 172.21.15.59:0/2665467682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:01:46.900 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.144+0000 7fdf47a00700 1 Processor -- start 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.144+0000 7fdf47a00700 1 -- start start 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.145+0000 7fdf47a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.145+0000 7fdf47a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4010f8a0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.145+0000 7fdf45000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.145+0000 7fdf45000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58140/0 (socket says 172.21.15.59:58140) 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.145+0000 7fdf45000700 1 -- 172.21.15.59:0/2264529425 learned_addr learned my addr 172.21.15.59:0/2264529425 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.146+0000 7fdf45000700 1 -- 172.21.15.59:0/2264529425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4010fa20 con 0x7fdf4010eeb0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.146+0000 7fdf45000700 1 --2- 172.21.15.59:0/2264529425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf340098a0 tx=0x7fdf34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b621589d35fefe0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.146+0000 7fdf3f400700 1 -- 172.21.15.59:0/2264529425 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34004160 con 0x7fdf4010eeb0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.146+0000 7fdf3f400700 1 -- 172.21.15.59:0/2264529425 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf34004300 con 0x7fdf4010eeb0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.146+0000 7fdf3f400700 1 -- 172.21.15.59:0/2264529425 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34011b50 con 0x7fdf4010eeb0 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 -- 172.21.15.59:0/2264529425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 msgr2=0x7fdf4010f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:46.901 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 --2- 172.21.15.59:0/2264529425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf340098a0 tx=0x7fdf34009bb0 comp rx=0 tx=0).stop 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 -- 172.21.15.59:0/2264529425 shutdown_connections 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 --2- 172.21.15.59:0/2264529425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf4010f2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 -- 172.21.15.59:0/2264529425 >> 172.21.15.59:0/2264529425 conn(0x7fdf4007d940 msgr2=0x7fdf4007fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 -- 172.21.15.59:0/2264529425 shutdown_connections 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.147+0000 7fdf47a00700 1 -- 172.21.15.59:0/2264529425 wait complete. 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.148+0000 7fdf47a00700 1 Processor -- start 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf47a00700 1 -- start start 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf47a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf47a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf40112b80 con 0x7fdf4010eeb0 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf45000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf45000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58142/0 (socket says 172.21.15.59:58142) 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.149+0000 7fdf45000700 1 -- 172.21.15.59:0/2665467682 learned_addr learned my addr 172.21.15.59:0/2665467682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:46.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf45000700 1 -- 172.21.15.59:0/2665467682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf401a2cc0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf45000700 1 --2- 172.21.15.59:0/2665467682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf34011b60 tx=0x7fdf34011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34003b20 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf34003cc0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34024840 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf401a2fb0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.150+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401a5c40 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.151+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf34022020 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.151+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf34011e00 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.152+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40063c20 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.155+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdf3402dd40 con 0x7fdf4010eeb0 2024-04-02T17:01:46.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.233+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdf401a60e0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.234+0000 7fdf3d600700 1 -- 172.21.15.59:0/2665467682 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fdf3402d3f0 con 0x7fdf4010eeb0 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 msgr2=0x7fdf401a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 --2- 172.21.15.59:0/2665467682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf34011b60 tx=0x7fdf34011c40 comp rx=0 tx=0).stop 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 shutdown_connections 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 --2- 172.21.15.59:0/2665467682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf4010eeb0 0x7fdf401a2780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 >> 172.21.15.59:0/2665467682 conn(0x7fdf4007d940 msgr2=0x7fdf4007e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:46.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 shutdown_connections 2024-04-02T17:01:46.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:46.236+0000 7fdf47a00700 1 -- 172.21.15.59:0/2665467682 wait complete. 2024-04-02T17:01:46.905 INFO:teuthology.orchestra.run.smithi059.stdout:Restarting the monitor... 2024-04-02T17:01:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:46 smithi059 systemd[1]: Stopping Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:01:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31277]: 2024-04-02T17:01:47.389+0000 7fa285800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:01:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31277]: 2024-04-02T17:01:47.389+0000 7fa285800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T17:01:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:47 smithi059 podman[31535]: 2024-04-02 17:01:47.504231379 +0000 UTC m=+0.238618560 container died 8bcea777d7c1fe3035a7119952e1ffa790f9ae1b52fe220acc9f37901e0b6e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T17:01:48.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:47 smithi059 podman[31535]: 2024-04-02 17:01:47.883055431 +0000 UTC m=+0.617442599 container cleanup 8bcea777d7c1fe3035a7119952e1ffa790f9ae1b52fe220acc9f37901e0b6e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T17:01:48.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:47 smithi059 bash[31535]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T17:01:48.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 podman[31549]: 2024-04-02 17:01:48.025288753 +0000 UTC m=+0.506311861 container remove 8bcea777d7c1fe3035a7119952e1ffa790f9ae1b52fe220acc9f37901e0b6e10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-02T17:01:48.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-02T17:01:48.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 systemd[1]: Stopped Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:01:48.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service: Consumed 1.985s CPU time. 2024-04-02T17:01:48.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 systemd[1]: Starting Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:01:49.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 podman[31644]: 2024-04-02 17:01:48.867796327 +0000 UTC m=+0.103345490 container create b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T17:01:49.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:48 smithi059 podman[31644]: 2024-04-02 17:01:48.800202875 +0000 UTC m=+0.035752014 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:01:49.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 podman[31644]: 2024-04-02 17:01:49.088198688 +0000 UTC m=+0.323747858 container init b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-02T17:01:49.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 podman[31644]: 2024-04-02 17:01:49.100012352 +0000 UTC m=+0.335561521 container start b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-02T17:01:49.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T17:01:49.150 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T17:01:49.189 INFO:teuthology.orchestra.run.smithi059.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T17:01:49.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: pidfile_write: ignore empty --pid-file 2024-04-02T17:01:49.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: load: jerasure load: lrc 2024-04-02T17:01:49.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: RocksDB version: 7.9.2 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Git sha 0 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: DB SUMMARY 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: DB Session ID: 0H5QRYA1ME0QMPH4QA53 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: CURRENT file: CURRENT 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105512 ; 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.error_if_exists: 0 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.create_if_missing: 0 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.paranoid_checks: 1 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T17:01:49.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.env: 0x56230baf7c60 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.info_log: 0x56230db09b20 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.statistics: (nil) 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.use_fsync: 0 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_log_file_size: 0 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_fallocate: 1 2024-04-02T17:01:49.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.use_direct_reads: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.db_log_dir: 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.wal_dir: 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T17:01:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.write_buffer_manager: 0x56230dada1e0 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T17:01:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.unordered_write: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.row_cache: None 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.wal_filter: None 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.two_write_queues: 0 2024-04-02T17:01:49.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.wal_compression: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.atomic_flush: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.log_readahead_size: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T17:01:49.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_background_jobs: 2 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_background_compactions: -1 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_subcompactions: 1 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_open_files: -1 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T17:01:49.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_background_flushes: -1 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Compression algorithms supported: 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kZSTD supported: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kXpressCompression supported: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kZlibCompression supported: 1 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kSnappyCompression supported: 1 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kLZ4Compression supported: 1 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: kBZip2Compression supported: 0 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-02T17:01:49.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.merge_operator: 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_filter: None 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56230db09d00) 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks: 1 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_top_level_index_and_filter: 1 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_type: 0 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_index_type: 0 2024-04-02T17:01:49.464 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_shortening: 1 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: checksum: 4 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: no_block_cache: 0 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache: 0x56230db0d090 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_name: BinnedLRUCache 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_options: 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: capacity : 536870912 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_shard_bits : 4 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: strict_capacity_limit : 0 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: high_pri_pool_ratio: 0.000 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_compressed: (nil) 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: persistent_cache: (nil) 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size: 4096 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size_deviation: 10 2024-04-02T17:01:49.465 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_restart_interval: 16 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_block_restart_interval: 1 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: metadata_block_size: 4096 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: partition_filters: 0 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: use_delta_encoding: 1 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: filter_policy: bloomfilter 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: whole_key_filtering: 1 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: verify_compression: 0 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: read_amp_bytes_per_bit: 0 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: format_version: 5 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_index_compression: 1 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_align: 0 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: max_auto_readahead_size: 262144 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: prepopulate_block_cache: 0 2024-04-02T17:01:49.466 INFO:journalctl@ceph.mon.a.smithi059.stdout: initial_auto_readahead_size: 8192 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression: NoCompression 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.num_levels: 7 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T17:01:49.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T17:01:49.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T17:01:49.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T17:01:49.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.inplace_update_support: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.bloom_locality: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.max_successive_merges: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T17:01:49.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.ttl: 2592000 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enable_blob_files: false 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.min_blob_size: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T17:01:49.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7aaf44e-74fb-4e62-b835-1af8a80f169e 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712077309161127, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712077309164053, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100601, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98486, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14290, "raw_average_key_size": 49, "raw_value_size": 90737, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712077309, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7aaf44e-74fb-4e62-b835-1af8a80f169e", "db_session_id": "0H5QRYA1ME0QMPH4QA53", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712077309164293, "job": 1, "event": "recovery_finished"} 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56230dbfc000 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: DB pointer 0x56230dbe6000 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** DB Stats ** 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T17:01:49.473 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: L0 2/0 100.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Sum 2/0 100.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-04-02T17:01:49.474 INFO:journalctl@ceph.mon.a.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 52.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative compaction: 0.00 GB write, 9.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval compaction: 0.00 GB write, 9.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T17:01:49.475 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache BinnedLRUCache@0x56230db0d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: starting mon.a rank 0 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???) e1 preinit fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).mds e1 new map 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).mds e1 print_map 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: e1 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: legacy client fscid: -1 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout: No filesystems configured 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T17:01:49.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 bash[31644]: b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 systemd[1]: Started Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: monmap epoch 1 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: last_changed 2024-04-02T17:01:29.288578+0000 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: min_mon_release 19 (squid) 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: election_strategy: 1 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: fsmap 2024-04-02T17:01:49.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T17:01:49.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:49 smithi059 ceph-mon[31664]: mgrmap e1: no daemons active 2024-04-02T17:01:50.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.764+0000 7f3d40c00700 1 Processor -- start 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.764+0000 7f3d40c00700 1 -- start start 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.764+0000 7f3d40c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.764+0000 7f3d40c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10ae90 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.765+0000 7f3d39e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.765+0000 7f3d39e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58150/0 (socket says 172.21.15.59:58150) 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.765+0000 7f3d39e00700 1 -- 172.21.15.59:0/69955193 learned_addr learned my addr 172.21.15.59:0/69955193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.765+0000 7f3d39e00700 1 -- 172.21.15.59:0/69955193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c10b600 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.766+0000 7f3d39e00700 1 --2- 172.21.15.59:0/69955193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0098a0 tx=0x7f3d2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7f9de8b4b501fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.766+0000 7f3d38a00700 1 -- 172.21.15.59:0/69955193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2c004160 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.473 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.766+0000 7f3d38a00700 1 -- 172.21.15.59:0/69955193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d2c004300 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.766+0000 7f3d38a00700 1 -- 172.21.15.59:0/69955193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2c011b50 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 -- 172.21.15.59:0/69955193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 msgr2=0x7f3d3c10a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 --2- 172.21.15.59:0/69955193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0098a0 tx=0x7f3d2c009bb0 comp rx=0 tx=0).stop 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 -- 172.21.15.59:0/69955193 shutdown_connections 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 --2- 172.21.15.59:0/69955193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c10a8c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 -- 172.21.15.59:0/69955193 >> 172.21.15.59:0/69955193 conn(0x7f3d3c07d960 msgr2=0x7f3d3c108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 -- 172.21.15.59:0/69955193 shutdown_connections 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.767+0000 7f3d40c00700 1 -- 172.21.15.59:0/69955193 wait complete. 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.768+0000 7f3d40c00700 1 Processor -- start 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d40c00700 1 -- start start 2024-04-02T17:01:50.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d40c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d40c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10e9b0 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d39e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d39e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58154/0 (socket says 172.21.15.59:58154) 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.769+0000 7f3d39e00700 1 -- 172.21.15.59:0/3366193986 learned_addr learned my addr 172.21.15.59:0/3366193986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d39e00700 1 -- 172.21.15.59:0/3366193986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c19e990 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d39e00700 1 --2- 172.21.15.59:0/3366193986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2c003b20 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d2c003cc0 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2c024910 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.770+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c19ec20 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.771+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a1910 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.771+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d2c022020 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.771+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d2c011e00 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.772+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c063c20 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.775+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3d2c01acd0 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.853+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3d3c1a1ae0 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.857+0000 7f3d32a00700 1 -- 172.21.15.59:0/3366193986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3d2c032030 con 0x7f3d3c10a4a0 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.858+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 msgr2=0x7f3d3c19e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.858+0000 7f3d40c00700 1 --2- 172.21.15.59:0/3366193986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c011c10 comp rx=0 tx=0).stop 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.858+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 shutdown_connections 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.859+0000 7f3d40c00700 1 --2- 172.21.15.59:0/3366193986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c10a4a0 0x7f3d3c19e450 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.859+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 >> 172.21.15.59:0/3366193986 conn(0x7f3d3c07d960 msgr2=0x7f3d3c107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.859+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 shutdown_connections 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:49.859+0000 7f3d40c00700 1 -- 172.21.15.59:0/3366193986 wait complete. 2024-04-02T17:01:50.476 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T17:01:50.524 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:01:50.524 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mgr... 2024-04-02T17:01:50.526 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T17:01:50.526 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T17:01:51.095 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T17:01:51.095 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-04-02T17:01:51.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:50 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3366193986' entity='client.admin' 2024-04-02T17:01:51.435 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4.target.wants/ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-8286be62-f112-11ee-b647-cb9ed24678a4@.service. 2024-04-02T17:01:51.737 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:51 smithi059 systemd[1]: Starting Ceph mgr.x for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:01:52.026 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:51 smithi059 podman[31878]: 2024-04-02 17:01:51.745261395 +0000 UTC m=+0.039087100 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:01:52.026 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:51 smithi059 podman[31878]: 2024-04-02 17:01:51.869061112 +0000 UTC m=+0.162886828 container create fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T17:01:52.205 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T17:01:52.205 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-04-02T17:01:52.221 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T17:01:52.221 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:01:52.221 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-04-02T17:01:52.221 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T17:01:52.242 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T17:01:52.242 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr to start... 2024-04-02T17:01:52.258 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr... 2024-04-02T17:01:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:52 smithi059 podman[31878]: 2024-04-02 17:01:52.108672322 +0000 UTC m=+0.402498072 container init fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T17:01:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:52 smithi059 podman[31878]: 2024-04-02 17:01:52.119066381 +0000 UTC m=+0.412892160 container start fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:01:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:52 smithi059 bash[31878]: fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 2024-04-02T17:01:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:52 smithi059 systemd[1]: Started Ceph mgr.x for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:01:53.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:52 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3195521979' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T17:01:53.032 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:52.896+0000 7f9f1c778200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T17:01:53.032 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:53.031+0000 7f9f1c778200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T17:01:53.780 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:53.790 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:01:53.790 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:01:53.790 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-02T17:01:53.791 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T17:01:53.792 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T17:01:53.793 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T17:01:53.794 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T17:01:33.642718+0000", 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.792+0000 7f6bc8c00700 1 Processor -- start 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.792+0000 7f6bc8c00700 1 -- start start 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.792+0000 7f6bc8c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.792+0000 7f6bc8c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4108b10 con 0x7f6bc4108120 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.793+0000 7f6bc1e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:53.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.793+0000 7f6bc1e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39372/0 (socket says 172.21.15.59:39372) 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.793+0000 7f6bc1e00700 1 -- 172.21.15.59:0/4087991961 learned_addr learned my addr 172.21.15.59:0/4087991961 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.793+0000 7f6bc1e00700 1 -- 172.21.15.59:0/4087991961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc4109320 con 0x7f6bc4108120 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.794+0000 7f6bc1e00700 1 --2- 172.21.15.59:0/4087991961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40098a0 tx=0x7f6bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47c5f7910d2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.794+0000 7f6bc0a00700 1 -- 172.21.15.59:0/4087991961 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb4004160 con 0x7f6bc4108120 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.794+0000 7f6bc0a00700 1 -- 172.21.15.59:0/4087991961 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bb4004300 con 0x7f6bc4108120 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.794+0000 7f6bc0a00700 1 -- 172.21.15.59:0/4087991961 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb4011b50 con 0x7f6bc4108120 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 -- 172.21.15.59:0/4087991961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 msgr2=0x7f6bc4108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 --2- 172.21.15.59:0/4087991961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40098a0 tx=0x7f6bb4009bb0 comp rx=0 tx=0).stop 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 -- 172.21.15.59:0/4087991961 shutdown_connections 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 --2- 172.21.15.59:0/4087991961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc4108120 0x7f6bc4108540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 -- 172.21.15.59:0/4087991961 >> 172.21.15.59:0/4087991961 conn(0x7f6bc4103740 msgr2=0x7f6bc4105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 -- 172.21.15.59:0/4087991961 shutdown_connections 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.795+0000 7f6bc8c00700 1 -- 172.21.15.59:0/4087991961 wait complete. 2024-04-02T17:01:53.796 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.796+0000 7f6bc8c00700 1 Processor -- start 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.796+0000 7f6bc8c00700 1 -- start start 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc8c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc8c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc410c6d0 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc1e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc1e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39376/0 (socket says 172.21.15.59:39376) 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc1e00700 1 -- 172.21.15.59:0/3195521979 learned_addr learned my addr 172.21.15.59:0/3195521979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc1e00700 1 -- 172.21.15.59:0/3195521979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419ef20 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.797+0000 7f6bc1e00700 1 --2- 172.21.15.59:0/3195521979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011850 tx=0x7f6bb4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb40036a0 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bb401a460 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb401b440 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419f1b0 con 0x7f6bc419e5c0 2024-04-02T17:01:53.797 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a1c90 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6bb401ba00 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.798+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6bb4024430 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.799+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4063c20 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.804+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6bb4043dd0 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.882+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6bc41a2430 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.883+0000 7f6bbaa00700 1 -- 172.21.15.59:0/3195521979 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6bb4032030 con 0x7f6bc419e5c0 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.884+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 msgr2=0x7f6bc419e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.884+0000 7f6bc8c00700 1 --2- 172.21.15.59:0/3195521979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011850 tx=0x7f6bb4011c10 comp rx=0 tx=0).stop 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.884+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 shutdown_connections 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.884+0000 7f6bc8c00700 1 --2- 172.21.15.59:0/3195521979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc419e5c0 0x7f6bc419e9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.884+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 >> 172.21.15.59:0/3195521979 conn(0x7f6bc4103740 msgr2=0x7f6bc4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.885+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 shutdown_connections 2024-04-02T17:01:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:52.885+0000 7f6bc8c00700 1 -- 172.21.15.59:0/3195521979 wait complete. 2024-04-02T17:01:53.799 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (1/15)... 2024-04-02T17:01:54.049 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:53.886+0000 7f9f1c778200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T17:01:54.049 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.010+0000 7f9f1c778200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T17:01:54.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.190+0000 7f9f1c778200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T17:01:54.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.318+0000 7f9f1c778200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T17:01:54.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.432+0000 7f9f1c778200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T17:01:55.032 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.772+0000 7f9f1c778200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T17:01:55.033 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:54.917+0000 7f9f1c778200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T17:01:55.301 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:55.031+0000 7f9f1c778200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T17:01:55.301 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:55.157+0000 7f9f1c778200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T17:01:55.798 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:55.379+0000 7f9f1c778200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T17:01:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2288788793' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T17:01:57.028 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:56.738+0000 7f9f1c778200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T17:01:57.029 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:56.916+0000 7f9f1c778200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T17:01:57.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T17:01:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T17:01:57.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T17:01:57.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T17:01:57.054 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T17:01:33.642718+0000", 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.327+0000 7f6f1ea00700 1 Processor -- start 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f1ea00700 1 -- start start 2024-04-02T17:01:57.055 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f1ea00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f1ea00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18108ad0 con 0x7f6f181080e0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f17e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f17e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39392/0 (socket says 172.21.15.59:39392) 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.328+0000 7f6f17e00700 1 -- 172.21.15.59:0/396759626 learned_addr learned my addr 172.21.15.59:0/396759626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.329+0000 7f6f17e00700 1 -- 172.21.15.59:0/396759626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f181092e0 con 0x7f6f181080e0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.329+0000 7f6f17e00700 1 --2- 172.21.15.59:0/396759626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098a0 tx=0x7f6f08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd05d12aa04c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.329+0000 7f6f16a00700 1 -- 172.21.15.59:0/396759626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f08004160 con 0x7f6f181080e0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f16a00700 1 -- 172.21.15.59:0/396759626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f08004300 con 0x7f6f181080e0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f16a00700 1 -- 172.21.15.59:0/396759626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f08011b50 con 0x7f6f181080e0 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 -- 172.21.15.59:0/396759626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 msgr2=0x7f6f18108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 --2- 172.21.15.59:0/396759626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098a0 tx=0x7f6f08009bb0 comp rx=0 tx=0).stop 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 -- 172.21.15.59:0/396759626 shutdown_connections 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 --2- 172.21.15.59:0/396759626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f18108500 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 -- 172.21.15.59:0/396759626 >> 172.21.15.59:0/396759626 conn(0x7f6f18103640 msgr2=0x7f6f18105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:57.056 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 -- 172.21.15.59:0/396759626 shutdown_connections 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.330+0000 7f6f1ea00700 1 -- 172.21.15.59:0/396759626 wait complete. 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.331+0000 7f6f1ea00700 1 Processor -- start 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f1ea00700 1 -- start start 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f1ea00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f1ea00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810c690 con 0x7f6f181080e0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f17e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f17e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39398/0 (socket says 172.21.15.59:39398) 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f17e00700 1 -- 172.21.15.59:0/2288788793 learned_addr learned my addr 172.21.15.59:0/2288788793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.332+0000 7f6f17e00700 1 -- 172.21.15.59:0/2288788793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1819e8d0 con 0x7f6f181080e0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f17e00700 1 --2- 172.21.15.59:0/2288788793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011850 tx=0x7f6f08011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f08003af0 con 0x7f6f181080e0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f08020070 con 0x7f6f181080e0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f080195a0 con 0x7f6f181080e0 2024-04-02T17:01:57.057 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1819eb60 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.333+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181a1850 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.334+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f08019ad0 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.334+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f08011e00 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.334+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f18063c20 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.337+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6f08003c90 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.415+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6f181a1d90 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.415+0000 7f6f14c00700 1 -- 172.21.15.59:0/2288788793 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6f08025460 con 0x7f6f181080e0 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.417+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 msgr2=0x7f6f1819e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.417+0000 7f6f1ea00700 1 --2- 172.21.15.59:0/2288788793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011850 tx=0x7f6f08011c10 comp rx=0 tx=0).stop 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.417+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 shutdown_connections 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.417+0000 7f6f1ea00700 1 --2- 172.21.15.59:0/2288788793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f181080e0 0x7f6f1819e390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.417+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 >> 172.21.15.59:0/2288788793 conn(0x7f6f18103640 msgr2=0x7f6f18104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.418+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 shutdown_connections 2024-04-02T17:01:57.058 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:56.418+0000 7f6f1ea00700 1 -- 172.21.15.59:0/2288788793 wait complete. 2024-04-02T17:01:57.059 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (2/15)... 2024-04-02T17:01:57.731 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:57.346+0000 7f9f1c778200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T17:01:57.731 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:57.466+0000 7f9f1c778200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T17:01:57.731 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:57.730+0000 7f9f1c778200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:01:58.048 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:57.979+0000 7f9f1c778200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T17:01:58.685 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:58.362+0000 7f9f1c778200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:01:58.997 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:58.684+0000 7f9f1c778200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T17:01:59.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:01:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:01:58.996+0000 7f9f1c778200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T17:02:00.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:01:59 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3797699659' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T17:02:00.275 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:00.080+0000 7f9f1c778200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T17:02:00.281 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:02:00.281 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:00.282 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:02:00.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T17:02:00.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T17:02:00.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T17:02:00.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T17:02:00.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T17:02:00.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T17:02:00.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-04-02T17:02:00.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T17:02:00.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:00.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T17:02:00.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T17:02:00.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T17:02:00.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T17:02:00.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T17:02:00.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:00.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T17:02:00.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T17:02:00.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T17:02:00.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T17:02:00.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T17:01:33.642718+0000", 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:00.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T17:02:00.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.640+0000 7f2a07800700 1 Processor -- start 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.641+0000 7f2a07800700 1 -- start start 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.641+0000 7f2a07800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.641+0000 7f2a07800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a001079e0 con 0x7f2a00106ff0 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.642+0000 7f2a04e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.642+0000 7f2a04e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39404/0 (socket says 172.21.15.59:39404) 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.642+0000 7f2a04e00700 1 -- 172.21.15.59:0/3194547841 learned_addr learned my addr 172.21.15.59:0/3194547841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.642+0000 7f2a04e00700 1 -- 172.21.15.59:0/3194547841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a001081f0 con 0x7f2a00106ff0 2024-04-02T17:02:00.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.643+0000 7f2a04e00700 1 --2- 172.21.15.59:0/3194547841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098a0 tx=0x7f29f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765615efc21de901 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:00.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.643+0000 7f29ff400700 1 -- 172.21.15.59:0/3194547841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4004160 con 0x7f2a00106ff0 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.643+0000 7f29ff400700 1 -- 172.21.15.59:0/3194547841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29f4004300 con 0x7f2a00106ff0 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.643+0000 7f29ff400700 1 -- 172.21.15.59:0/3194547841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4011b50 con 0x7f2a00106ff0 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 -- 172.21.15.59:0/3194547841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 msgr2=0x7f2a00107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 --2- 172.21.15.59:0/3194547841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098a0 tx=0x7f29f4009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 -- 172.21.15.59:0/3194547841 shutdown_connections 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 --2- 172.21.15.59:0/3194547841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a00107410 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 -- 172.21.15.59:0/3194547841 >> 172.21.15.59:0/3194547841 conn(0x7f2a001025d0 msgr2=0x7f2a001049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 -- 172.21.15.59:0/3194547841 shutdown_connections 2024-04-02T17:02:00.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.644+0000 7f2a07800700 1 -- 172.21.15.59:0/3194547841 wait complete. 2024-04-02T17:02:00.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.645+0000 7f2a07800700 1 Processor -- start 2024-04-02T17:02:00.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.645+0000 7f2a07800700 1 -- start start 2024-04-02T17:02:00.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.645+0000 7f2a07800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:00.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.645+0000 7f2a07800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0010b5a0 con 0x7f2a00106ff0 2024-04-02T17:02:00.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.646+0000 7f2a04e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:00.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.646+0000 7f2a04e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39420/0 (socket says 172.21.15.59:39420) 2024-04-02T17:02:00.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.646+0000 7f2a04e00700 1 -- 172.21.15.59:0/3797699659 learned_addr learned my addr 172.21.15.59:0/3797699659 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.646+0000 7f2a04e00700 1 -- 172.21.15.59:0/3797699659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a001926e0 con 0x7f2a00106ff0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.646+0000 7f2a04e00700 1 --2- 172.21.15.59:0/3797699659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29f4011850 tx=0x7f29f4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4003af0 con 0x7f2a00106ff0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29f4020070 con 0x7f2a00106ff0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a001908d0 con 0x7f2a00106ff0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f40195a0 con 0x7f2a00106ff0 2024-04-02T17:02:00.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a00190e10 con 0x7f2a00106ff0 2024-04-02T17:02:00.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29f4019be0 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.647+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29f4011e00 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.648+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00063c20 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.651+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f29f4003c90 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.730+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2a00195ab0 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.731+0000 7f29fd600700 1 -- 172.21.15.59:0/3797699659 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f29f4025460 con 0x7f2a00106ff0 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.732+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 msgr2=0x7f2a001921a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.732+0000 7f2a07800700 1 --2- 172.21.15.59:0/3797699659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29f4011850 tx=0x7f29f4011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.733+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 shutdown_connections 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.733+0000 7f2a07800700 1 --2- 172.21.15.59:0/3797699659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a00106ff0 0x7f2a001921a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.733+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 >> 172.21.15.59:0/3797699659 conn(0x7f2a001025d0 msgr2=0x7f2a00103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:00.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.733+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 shutdown_connections 2024-04-02T17:02:00.303 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:01:59.733+0000 7f2a07800700 1 -- 172.21.15.59:0/3797699659 wait complete. 2024-04-02T17:02:00.303 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (3/15)... 2024-04-02T17:02:01.048 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:00.796+0000 7f9f1c778200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T17:02:01.049 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:00.928+0000 7f9f1c778200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T17:02:01.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:01.218+0000 7f9f1c778200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T17:02:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: Activating manager daemon x 2024-04-02T17:02:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: mgrmap e2: x(active, starting, since 0.0115252s) 2024-04-02T17:02:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: Manager daemon x is now available 2024-04-02T17:02:01.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T17:02:01.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T17:02:01.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:01 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' 2024-04-02T17:02:02.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:02 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' 2024-04-02T17:02:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:02 smithi059 ceph-mon[31664]: from='mgr.14100 172.21.15.59:0/1323629270' entity='mgr.x' 2024-04-02T17:02:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:02 smithi059 ceph-mon[31664]: mgrmap e3: x(active, since 1.02305s) 2024-04-02T17:02:03.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:03 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1722733975' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T17:02:03.849 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:02:03.849 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:03.849 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:02:03.849 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T17:02:03.849 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:03.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T17:02:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T17:02:03.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T17:01:33.642718+0000", 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T17:02:03.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.871+0000 7f852c400700 1 Processor -- start 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f852c400700 1 -- start start 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f852c400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f852c400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f852407f1d0 con 0x7f852407e7e0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f8529a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f8529a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39366/0 (socket says 172.21.15.59:39366) 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.872+0000 7f8529a00700 1 -- 172.21.15.59:0/123425440 learned_addr learned my addr 172.21.15.59:0/123425440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.873+0000 7f8529a00700 1 -- 172.21.15.59:0/123425440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852407f9e0 con 0x7f852407e7e0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.873+0000 7f8529a00700 1 --2- 172.21.15.59:0/123425440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85180098a0 tx=0x7f8518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187182d4a6c32b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.874+0000 7f8523e00700 1 -- 172.21.15.59:0/123425440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8518004160 con 0x7f852407e7e0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.874+0000 7f8523e00700 1 -- 172.21.15.59:0/123425440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8518004300 con 0x7f852407e7e0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.874+0000 7f8523e00700 1 -- 172.21.15.59:0/123425440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8518011b50 con 0x7f852407e7e0 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.874+0000 7f852c400700 1 -- 172.21.15.59:0/123425440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 msgr2=0x7f852407ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:03.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.874+0000 7f852c400700 1 --2- 172.21.15.59:0/123425440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85180098a0 tx=0x7f8518009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.875+0000 7f852c400700 1 -- 172.21.15.59:0/123425440 shutdown_connections 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.875+0000 7f852c400700 1 --2- 172.21.15.59:0/123425440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f852407ec00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.875+0000 7f852c400700 1 -- 172.21.15.59:0/123425440 >> 172.21.15.59:0/123425440 conn(0x7f852407d940 msgr2=0x7f8524108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.875+0000 7f852c400700 1 -- 172.21.15.59:0/123425440 shutdown_connections 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.875+0000 7f852c400700 1 -- 172.21.15.59:0/123425440 wait complete. 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.876+0000 7f852c400700 1 Processor -- start 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f852c400700 1 -- start start 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f852c400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f852c400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8524080da0 con 0x7f852407e7e0 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f8529a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f8529a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39382/0 (socket says 172.21.15.59:39382) 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.877+0000 7f8529a00700 1 -- 172.21.15.59:0/1722733975 learned_addr learned my addr 172.21.15.59:0/1722733975 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f8529a00700 1 -- 172.21.15.59:0/1722733975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241a2d30 con 0x7f852407e7e0 2024-04-02T17:02:03.855 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f8529a00700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8518011b60 tx=0x7f8518011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85180039e0 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8518003b80 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8518024760 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.878+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241a2fc0 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.879+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a5cb0 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.880+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8524063c20 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.880+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f8518022020 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.880+0000 7f8522000700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 0x7f850c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.880+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8518011e00 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.881+0000 7f8529000700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 0x7f850c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.881+0000 7f8529000700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 0x7f850c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:02.887+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f851801a900 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.190+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f852402dec0 con 0x7f852407e7e0 2024-04-02T17:02:03.856 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.191+0000 7f8522000700 1 -- 172.21.15.59:0/1722733975 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f851802c560 con 0x7f852407e7e0 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 msgr2=0x7f850c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 0x7f850c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 msgr2=0x7f85241a27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8518011b60 tx=0x7f8518011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 shutdown_connections 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f850c039490 0x7f850c03b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 --2- 172.21.15.59:0/1722733975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852407e7e0 0x7f85241a27f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.194+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 >> 172.21.15.59:0/1722733975 conn(0x7f852407d940 msgr2=0x7f85241994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.195+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 shutdown_connections 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:03.195+0000 7f852c400700 1 -- 172.21.15.59:0/1722733975 wait complete. 2024-04-02T17:02:03.857 INFO:teuthology.orchestra.run.smithi059.stdout:mgr is available 2024-04-02T17:02:04.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:04 smithi059 ceph-mon[31664]: mgrmap e4: x(active, since 2s) 2024-04-02T17:02:05.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:02:05.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T17:02:05.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T17:02:05.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.382+0000 7f934f600700 1 Processor -- start 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.382+0000 7f934f600700 1 -- start start 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.382+0000 7f934f600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.382+0000 7f934f600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f934810f8a0 con 0x7f934810eeb0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.383+0000 7f934cc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.383+0000 7f934cc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39384/0 (socket says 172.21.15.59:39384) 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.383+0000 7f934cc00700 1 -- 172.21.15.59:0/2795856367 learned_addr learned my addr 172.21.15.59:0/2795856367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.383+0000 7f934cc00700 1 -- 172.21.15.59:0/2795856367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934810fa20 con 0x7f934810eeb0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.384+0000 7f934cc00700 1 --2- 172.21.15.59:0/2795856367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22af834056e65760 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.384+0000 7f9347400700 1 -- 172.21.15.59:0/2795856367 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c004160 con 0x7f934810eeb0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.384+0000 7f9347400700 1 -- 172.21.15.59:0/2795856367 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f933c004300 con 0x7f934810eeb0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.384+0000 7f9347400700 1 -- 172.21.15.59:0/2795856367 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c011b50 con 0x7f934810eeb0 2024-04-02T17:02:05.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 -- 172.21.15.59:0/2795856367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 msgr2=0x7f934810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 --2- 172.21.15.59:0/2795856367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f933c0098a0 tx=0x7f933c009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 -- 172.21.15.59:0/2795856367 shutdown_connections 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 --2- 172.21.15.59:0/2795856367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f934810f2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 -- 172.21.15.59:0/2795856367 >> 172.21.15.59:0/2795856367 conn(0x7f934807d940 msgr2=0x7f934807fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 -- 172.21.15.59:0/2795856367 shutdown_connections 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.385+0000 7f934f600700 1 -- 172.21.15.59:0/2795856367 wait complete. 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.386+0000 7f934f600700 1 Processor -- start 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934f600700 1 -- start start 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934f600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934f600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9348112b80 con 0x7f934810eeb0 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934cc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934cc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39394/0 (socket says 172.21.15.59:39394) 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.387+0000 7f934cc00700 1 -- 172.21.15.59:0/3621541168 learned_addr learned my addr 172.21.15.59:0/3621541168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:05.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f934cc00700 1 -- 172.21.15.59:0/3621541168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93481a2cc0 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f934cc00700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f933c011b60 tx=0x7f933c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c003b20 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f933c003cc0 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f933c024840 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93481a2fb0 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.388+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93481a5c40 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.390+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f933c022020 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.390+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9348063c20 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.390+0000 7f9345600700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 0x7f933003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.390+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f933c011e00 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.390+0000 7f9347e00700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 0x7f933003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.391+0000 7f9347e00700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 0x7f933003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9334006fd0 tx=0x7f9334006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.397+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f933c00b5a0 con 0x7f934810eeb0 2024-04-02T17:02:05.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.627+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f93481a5e10 con 0x7f934810eeb0 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.629+0000 7f9345600700 1 -- 172.21.15.59:0/3621541168 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7f933c04d160 con 0x7f934810eeb0 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 msgr2=0x7f933003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 0x7f933003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9334006fd0 tx=0x7f9334006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 msgr2=0x7f93481a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f933c011b60 tx=0x7f933c011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 shutdown_connections 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f93300395b0 0x7f933003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 --2- 172.21.15.59:0/3621541168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f934810eeb0 0x7f93481a2780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 >> 172.21.15.59:0/3621541168 conn(0x7f934807d940 msgr2=0x7f934807e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 shutdown_connections 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:04.632+0000 7f934f600700 1 -- 172.21.15.59:0/3621541168 wait complete. 2024-04-02T17:02:05.276 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling cephadm module... 2024-04-02T17:02:05.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:05 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3621541168' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T17:02:06.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:06 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/422541534' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T17:02:06.680 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:06 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: ignoring --setuser ceph since I am not root 2024-04-02T17:02:06.680 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:06 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: ignoring --setgroup ceph since I am not root 2024-04-02T17:02:06.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.825+0000 7fa206200700 1 Processor -- start 2024-04-02T17:02:06.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.826+0000 7fa206200700 1 -- start start 2024-04-02T17:02:06.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.826+0000 7fa206200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.826+0000 7fa206200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa200108bd0 con 0x7fa200108270 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.826+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.827+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39396/0 (socket says 172.21.15.59:39396) 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.827+0000 7fa1ff400700 1 -- 172.21.15.59:0/2098939698 learned_addr learned my addr 172.21.15.59:0/2098939698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.827+0000 7fa1ff400700 1 -- 172.21.15.59:0/2098939698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200108d50 con 0x7fa200108270 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.827+0000 7fa1ff400700 1 --2- 172.21.15.59:0/2098939698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098a0 tx=0x7fa1f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1f887a99f7e639 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa1fe000700 1 -- 172.21.15.59:0/2098939698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0004160 con 0x7fa200108270 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa1fe000700 1 -- 172.21.15.59:0/2098939698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f0004300 con 0x7fa200108270 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa1fe000700 1 -- 172.21.15.59:0/2098939698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0011b50 con 0x7fa200108270 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa206200700 1 -- 172.21.15.59:0/2098939698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 msgr2=0x7fa200108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa206200700 1 --2- 172.21.15.59:0/2098939698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098a0 tx=0x7fa1f0009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:06.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa206200700 1 -- 172.21.15.59:0/2098939698 shutdown_connections 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa206200700 1 --2- 172.21.15.59:0/2098939698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa200108690 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.828+0000 7fa206200700 1 -- 172.21.15.59:0/2098939698 >> 172.21.15.59:0/2098939698 conn(0x7fa200103910 msgr2=0x7fa200105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.829+0000 7fa206200700 1 -- 172.21.15.59:0/2098939698 shutdown_connections 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.829+0000 7fa206200700 1 -- 172.21.15.59:0/2098939698 wait complete. 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.830+0000 7fa206200700 1 Processor -- start 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.830+0000 7fa206200700 1 -- start start 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.830+0000 7fa206200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.830+0000 7fa206200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010c6e0 con 0x7fa200108270 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.831+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.831+0000 7fa1ff400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39402/0 (socket says 172.21.15.59:39402) 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.831+0000 7fa1ff400700 1 -- 172.21.15.59:0/422541534 learned_addr learned my addr 172.21.15.59:0/422541534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.831+0000 7fa1ff400700 1 -- 172.21.15.59:0/422541534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20007e030 con 0x7fa200108270 2024-04-02T17:02:06.928 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.831+0000 7fa1ff400700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0011b60 tx=0x7fa1f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.832+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0003970 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.832+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f0003b10 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.832+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0024610 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.832+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20007c220 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.832+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20007c760 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.833+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa1f001a9f0 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.833+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200063c20 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.834+0000 7fa1f7e00700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 0x7fa1e403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.834+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1f0011e00 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.834+0000 7fa1fea00700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 0x7fa1e403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.835+0000 7fa1fea00700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 0x7fa1e403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:05.839+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1f0016260 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.135+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa20007c9e0 con 0x7fa200108270 2024-04-02T17:02:06.929 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.300+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa1f0019e10 con 0x7fa200108270 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.303+0000 7fa1f7e00700 1 -- 172.21.15.59:0/422541534 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa1f0024c40 con 0x7fa200108270 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 msgr2=0x7fa1e403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 0x7fa1e403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 msgr2=0x7fa20007daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0011b60 tx=0x7fa1f0011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 shutdown_connections 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fa1e40395b0 0x7fa1e403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 --2- 172.21.15.59:0/422541534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200108270 0x7fa20007daf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.306+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 >> 172.21.15.59:0/422541534 conn(0x7fa200103910 msgr2=0x7fa200104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.307+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 shutdown_connections 2024-04-02T17:02:06.930 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:06.307+0000 7fa206200700 1 -- 172.21.15.59:0/422541534 wait complete. 2024-04-02T17:02:07.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:07 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/422541534' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T17:02:07.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:07 smithi059 ceph-mon[31664]: mgrmap e5: x(active, since 5s) 2024-04-02T17:02:07.397 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:07 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:07.107+0000 7fb67be6a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T17:02:07.397 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:07 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:07.232+0000 7fb67be6a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T17:02:08.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1816946884' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T17:02:08.345 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.064+0000 7fb67be6a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T17:02:08.345 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.190+0000 7fb67be6a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T17:02:08.615 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.372+0000 7fb67be6a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T17:02:08.615 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.500+0000 7fb67be6a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T17:02:08.615 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.614+0000 7fb67be6a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T17:02:08.635 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.551+0000 7fcefd800700 1 Processor -- start 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.551+0000 7fcefd800700 1 -- start start 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.551+0000 7fcefd800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.551+0000 7fcefd800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcef807c7c0 con 0x7fcef807dd80 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.551+0000 7fcef6a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.552+0000 7fcef6a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39436/0 (socket says 172.21.15.59:39436) 2024-04-02T17:02:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.552+0000 7fcef6a00700 1 -- 172.21.15.59:0/2060591475 learned_addr learned my addr 172.21.15.59:0/2060591475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.552+0000 7fcef6a00700 1 -- 172.21.15.59:0/2060591475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef807c940 con 0x7fcef807dd80 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.552+0000 7fcef6a00700 1 --2- 172.21.15.59:0/2060591475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcee80098a0 tx=0x7fcee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f2f64ca4481184 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcef6000700 1 -- 172.21.15.59:0/2060591475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee8004160 con 0x7fcef807dd80 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcef6000700 1 -- 172.21.15.59:0/2060591475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcee8004300 con 0x7fcef807dd80 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcef6000700 1 -- 172.21.15.59:0/2060591475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee8011b50 con 0x7fcef807dd80 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcefd800700 1 -- 172.21.15.59:0/2060591475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 msgr2=0x7fcef807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcefd800700 1 --2- 172.21.15.59:0/2060591475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcee80098a0 tx=0x7fcee8009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcefd800700 1 -- 172.21.15.59:0/2060591475 shutdown_connections 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.553+0000 7fcefd800700 1 --2- 172.21.15.59:0/2060591475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef807c280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.554+0000 7fcefd800700 1 -- 172.21.15.59:0/2060591475 >> 172.21.15.59:0/2060591475 conn(0x7fcef81038b0 msgr2=0x7fcef8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.554+0000 7fcefd800700 1 -- 172.21.15.59:0/2060591475 shutdown_connections 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.554+0000 7fcefd800700 1 -- 172.21.15.59:0/2060591475 wait complete. 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcefd800700 1 Processor -- start 2024-04-02T17:02:08.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcefd800700 1 -- start start 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcefd800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcefd800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcef810e880 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcef6a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcef6a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39450/0 (socket says 172.21.15.59:39450) 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.555+0000 7fcef6a00700 1 -- 172.21.15.59:0/1816946884 learned_addr learned my addr 172.21.15.59:0/1816946884 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fcef6a00700 1 -- 172.21.15.59:0/1816946884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef819e980 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fcef6a00700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fb0 tx=0x7fcee8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee80038b0 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcee8011e50 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef819ec10 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.556+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee8019420 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.557+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef81a1900 con 0x7fcef807dd80 2024-04-02T17:02:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.558+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef8063c20 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.558+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcee8003a50 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.558+0000 7fceef400700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 0x7fcedc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.558+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcee804d910 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.559+0000 7fceefe00700 1 -- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 msgr2=0x7fcedc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.559+0000 7fceefe00700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 0x7fcedc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.568+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcee804d330 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.760+0000 7fceefe00700 1 -- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 msgr2=0x7fcedc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.760+0000 7fceefe00700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 0x7fcedc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.849+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcef81a1ad0 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.850+0000 7fceef400700 1 -- 172.21.15.59:0/1816946884 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcee800fda0 con 0x7fcef807dd80 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 msgr2=0x7fcedc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 0x7fcedc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 msgr2=0x7fcef819e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fb0 tx=0x7fcee8011bf0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 shutdown_connections 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7fcedc0395b0 0x7fcedc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 --2- 172.21.15.59:0/1816946884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef807dd80 0x7fcef819e440 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 >> 172.21.15.59:0/1816946884 conn(0x7fcef81038b0 msgr2=0x7fcef8104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.853+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 shutdown_connections 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:07.854+0000 7fcefd800700 1 -- 172.21.15.59:0/1816946884 wait complete. 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-04-02T17:02:08.640 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 5... 2024-04-02T17:02:09.161 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:08 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:08.964+0000 7fb67be6a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T17:02:09.161 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:09.112+0000 7fb67be6a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T17:02:09.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:09.230+0000 7fb67be6a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T17:02:09.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:09.355+0000 7fb67be6a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T17:02:10.051 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:09.579+0000 7fb67be6a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T17:02:11.215 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:10 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:10.889+0000 7fb67be6a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T17:02:11.215 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:11 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:11.069+0000 7fb67be6a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T17:02:11.798 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:11 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:11.504+0000 7fb67be6a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T17:02:11.798 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:11 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:11.626+0000 7fb67be6a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T17:02:12.143 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:11 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:11.890+0000 7fb67be6a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:02:12.143 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:12 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:12.141+0000 7fb67be6a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T17:02:12.799 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:12 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:12.530+0000 7fb67be6a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:02:13.170 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:12 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:12.854+0000 7fb67be6a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T17:02:13.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:13 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:13.169+0000 7fb67be6a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T17:02:14.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:14 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:14.239+0000 7fb67be6a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T17:02:15.298 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:14 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:14.961+0000 7fb67be6a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T17:02:15.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:15.095+0000 7fb67be6a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T17:02:15.798 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:15.386+0000 7fb67be6a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: Active manager daemon x restarted 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: Activating manager daemon x 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: mgrmap e6: x(active, starting, since 0.00516819s) 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T17:02:15.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:15 smithi059 ceph-mon[31664]: Manager daemon x is now available 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: Found migration_current of "None". Setting to last migration. 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: mgrmap e7: x(active, since 1.01606s) 2024-04-02T17:02:16.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:16 smithi059 ceph-mon[31664]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.160+0000 7f4130e00700 1 Processor -- start 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.160+0000 7f4130e00700 1 -- start start 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.161+0000 7f4130e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.161+0000 7f4130e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1079e0 con 0x7f412c106ff0 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.161+0000 7f4129e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.161+0000 7f4129e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39466/0 (socket says 172.21.15.59:39466) 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.161+0000 7f4129e00700 1 -- 172.21.15.59:0/216287652 learned_addr learned my addr 172.21.15.59:0/216287652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:17.143 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.162+0000 7f4129e00700 1 -- 172.21.15.59:0/216287652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c1081f0 con 0x7f412c106ff0 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.162+0000 7f4129e00700 1 --2- 172.21.15.59:0/216287652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098a0 tx=0x7f411c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251a45e8ab5ed810 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.162+0000 7f4128a00700 1 -- 172.21.15.59:0/216287652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c004160 con 0x7f412c106ff0 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.162+0000 7f4128a00700 1 -- 172.21.15.59:0/216287652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f411c004300 con 0x7f412c106ff0 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.162+0000 7f4128a00700 1 -- 172.21.15.59:0/216287652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c011b50 con 0x7f412c106ff0 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 -- 172.21.15.59:0/216287652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 msgr2=0x7f412c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 --2- 172.21.15.59:0/216287652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098a0 tx=0x7f411c009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 -- 172.21.15.59:0/216287652 shutdown_connections 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 --2- 172.21.15.59:0/216287652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c107410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 -- 172.21.15.59:0/216287652 >> 172.21.15.59:0/216287652 conn(0x7f412c1025d0 msgr2=0x7f412c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 -- 172.21.15.59:0/216287652 shutdown_connections 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.163+0000 7f4130e00700 1 -- 172.21.15.59:0/216287652 wait complete. 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.164+0000 7f4130e00700 1 Processor -- start 2024-04-02T17:02:17.144 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.164+0000 7f4130e00700 1 -- start start 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4130e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4130e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10b5a0 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4129e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4129e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39476/0 (socket says 172.21.15.59:39476) 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4129e00700 1 -- 172.21.15.59:0/425718478 learned_addr learned my addr 172.21.15.59:0/425718478 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.165+0000 7f4129e00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c19c6d0 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4129e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f411c011850 tx=0x7f411c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c003af0 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f411c020070 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c0195a0 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c19c960 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.166+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c19f650 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.167+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f411c019850 con 0x7f412c106ff0 2024-04-02T17:02:17.145 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.168+0000 7f4122a00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.168+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f411003c1d0 con 0x7f4110039600 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.168+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f411c011e00 con 0x7f412c106ff0 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.168+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.168+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.369+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.369+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.770+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:09.770+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:10.571+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:10.571+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T17:02:17.146 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:12.173+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:12.173+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:15.378+0000 7f4129400700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3477889503 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:15.378+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:15.394+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f411c01a430 con 0x7f412c106ff0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:15.394+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 msgr2=0x7f411003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:15.394+0000 7f4122a00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.399+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f411c00f2e0 con 0x7f412c106ff0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.400+0000 7f4122a00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 0x7f411003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.400+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f411c04c070 con 0x7f411003cef0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.402+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 0x7f411003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.404+0000 7f4129400700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 0x7f411003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4118003a10 tx=0x7f41180092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.406+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f411c04c070 con 0x7f411003cef0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.418+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f412c063c20 con 0x7f411003cef0 2024-04-02T17:02:17.147 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.419+0000 7f4122a00700 1 -- 172.21.15.59:0/425718478 <== mgr.14120 v2:172.21.15.59:6800/2274685378 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f412c063c20 con 0x7f411003cef0 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 msgr2=0x7f411003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 0x7f411003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4118003a10 tx=0x7f41180092b0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 msgr2=0x7f412c19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f411c011850 tx=0x7f411c011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 shutdown_connections 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f411003cef0 0x7f411003f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c106ff0 0x7f412c19c190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 --2- 172.21.15.59:0/425718478 >> [v2:172.21.15.59:6800/3477889503,v1:172.21.15.59:6801/3477889503] conn(0x7f4110039600 0x7f411003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 >> 172.21.15.59:0/425718478 conn(0x7f412c1025d0 msgr2=0x7f412c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 shutdown_connections 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:16.420+0000 7f4130e00700 1 -- 172.21.15.59:0/425718478 wait complete. 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 5 is available 2024-04-02T17:02:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:Setting orchestrator backend to cephadm... 2024-04-02T17:02:17.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:17 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:17.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:17 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:17.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:17 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:18.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.656+0000 7fdce2000700 1 Processor -- start 2024-04-02T17:02:18.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.656+0000 7fdce2000700 1 -- start start 2024-04-02T17:02:18.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.656+0000 7fdce2000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:18.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.657+0000 7fdce2000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc07b740 con 0x7fdcdc07cd10 2024-04-02T17:02:18.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.657+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:18.560 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.657+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52384/0 (socket says 172.21.15.59:52384) 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.657+0000 7fdcdb400700 1 -- 172.21.15.59:0/2150486673 learned_addr learned my addr 172.21.15.59:0/2150486673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.657+0000 7fdcdb400700 1 -- 172.21.15.59:0/2150486673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc07b8c0 con 0x7fdcdc07cd10 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.658+0000 7fdcdb400700 1 --2- 172.21.15.59:0/2150486673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed59a9197d76b378 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.658+0000 7fdcda000700 1 -- 172.21.15.59:0/2150486673 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc004160 con 0x7fdcdc07cd10 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.658+0000 7fdcda000700 1 -- 172.21.15.59:0/2150486673 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdccc004300 con 0x7fdcdc07cd10 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.658+0000 7fdcda000700 1 -- 172.21.15.59:0/2150486673 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc011b50 con 0x7fdcdc07cd10 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 -- 172.21.15.59:0/2150486673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 msgr2=0x7fdcdc07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 --2- 172.21.15.59:0/2150486673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 -- 172.21.15.59:0/2150486673 shutdown_connections 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 --2- 172.21.15.59:0/2150486673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc07b170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 -- 172.21.15.59:0/2150486673 >> 172.21.15.59:0/2150486673 conn(0x7fdcdc103640 msgr2=0x7fdcdc105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 -- 172.21.15.59:0/2150486673 shutdown_connections 2024-04-02T17:02:18.561 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.659+0000 7fdce2000700 1 -- 172.21.15.59:0/2150486673 wait complete. 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.660+0000 7fdce2000700 1 Processor -- start 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.660+0000 7fdce2000700 1 -- start start 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.660+0000 7fdce2000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.660+0000 7fdce2000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10e6f0 con 0x7fdcdc07cd10 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.660+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcdb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52392/0 (socket says 172.21.15.59:52392) 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcdb400700 1 -- 172.21.15.59:0/346762992 learned_addr learned my addr 172.21.15.59:0/346762992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcdb400700 1 -- 172.21.15.59:0/346762992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc19e8a0 con 0x7fdcdc07cd10 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcdb400700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdccc011850 tx=0x7fdccc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc003af0 con 0x7fdcdc07cd10 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdccc020070 con 0x7fdcdc07cd10 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc0195a0 con 0x7fdcdc07cd10 2024-04-02T17:02:18.562 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.661+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc19eb30 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.662+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a1820 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.663+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fdccc019850 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.663+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc063c20 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.663+0000 7fdcd3e00700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 0x7fdcc003b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.663+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdccc011e00 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.663+0000 7fdcdaa00700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 0x7fdcc003b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.664+0000 7fdcdaa00700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 0x7fdcc003b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.670+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdccc00e260 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.857+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdccc01a430 con 0x7fdcdc07cd10 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.920+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fdcdc104a50 con 0x7fdcc00394e0 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.936+0000 7fdcd3e00700 1 -- 172.21.15.59:0/346762992 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdcdc104a50 con 0x7fdcc00394e0 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 msgr2=0x7fdcc003b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 0x7fdcc003b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:18.563 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 msgr2=0x7fdcdc19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdccc011850 tx=0x7fdccc011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 shutdown_connections 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fdcc00394e0 0x7fdcc003b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 --2- 172.21.15.59:0/346762992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcdc07cd10 0x7fdcdc19e360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 >> 172.21.15.59:0/346762992 conn(0x7fdcdc103640 msgr2=0x7fdcdc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 shutdown_connections 2024-04-02T17:02:18.564 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:17.939+0000 7fdce2000700 1 -- 172.21.15.59:0/346762992 wait complete. 2024-04-02T17:02:19.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:17] ENGINE Bus STARTING 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:17] ENGINE Serving on http://172.21.15.59:8765 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:17] ENGINE Serving on https://172.21.15.59:7150 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:17] ENGINE Bus STARTED 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: mgrmap e8: x(active, since 2s) 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:19.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:18 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:19.934 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:19 smithi059 ceph-mon[31664]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:19.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.133+0000 7fa5b1600700 1 Processor -- start 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.133+0000 7fa5b1600700 1 -- start start 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.133+0000 7fa5b1600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.133+0000 7fa5b1600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10f8a0 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.134+0000 7fa5aaa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.134+0000 7fa5aaa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52400/0 (socket says 172.21.15.59:52400) 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.134+0000 7fa5aaa00700 1 -- 172.21.15.59:0/366399136 learned_addr learned my addr 172.21.15.59:0/366399136 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.134+0000 7fa5aaa00700 1 -- 172.21.15.59:0/366399136 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac10fa20 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.134+0000 7fa5aaa00700 1 --2- 172.21.15.59:0/366399136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098a0 tx=0x7fa59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba77737b085695 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5aa000700 1 -- 172.21.15.59:0/366399136 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa59c004160 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5aa000700 1 -- 172.21.15.59:0/366399136 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa59c004300 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5aa000700 1 -- 172.21.15.59:0/366399136 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa59c011b50 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5b1600700 1 -- 172.21.15.59:0/366399136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 msgr2=0x7fa5ac10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5b1600700 1 --2- 172.21.15.59:0/366399136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098a0 tx=0x7fa59c009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5b1600700 1 -- 172.21.15.59:0/366399136 shutdown_connections 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5b1600700 1 --2- 172.21.15.59:0/366399136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac10f2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.135+0000 7fa5b1600700 1 -- 172.21.15.59:0/366399136 >> 172.21.15.59:0/366399136 conn(0x7fa5ac07d940 msgr2=0x7fa5ac07fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.136+0000 7fa5b1600700 1 -- 172.21.15.59:0/366399136 shutdown_connections 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.136+0000 7fa5b1600700 1 -- 172.21.15.59:0/366399136 wait complete. 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.137+0000 7fa5b1600700 1 Processor -- start 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.137+0000 7fa5b1600700 1 -- start start 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.137+0000 7fa5b1600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.137+0000 7fa5b1600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac112b80 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5aaa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:19.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5aaa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52412/0 (socket says 172.21.15.59:52412) 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5aaa00700 1 -- 172.21.15.59:0/1354605393 learned_addr learned my addr 172.21.15.59:0/1354605393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5aaa00700 1 -- 172.21.15.59:0/1354605393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac1a2cc0 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5aaa00700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa59c011b60 tx=0x7fa59c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.138+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa59c003b20 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.139+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa59c003cc0 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.139+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa59c024840 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.139+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1a2fb0 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.139+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1a5c40 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.140+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac063c20 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.140+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa59c022020 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.953 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.140+0000 7fa5a3e00700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 0x7fa59003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:19.953 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.141+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa59c011e00 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.141+0000 7fa5a2000700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 0x7fa59003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.141+0000 7fa5a2000700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 0x7fa59003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa59400ad30 tx=0x7fa5940093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.146+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa59c00b3e0 con 0x7fa5ac10eeb0 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.377+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa5ac07ebb0 con 0x7fa590039600 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.379+0000 7fa5a3e00700 1 -- 172.21.15.59:0/1354605393 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa5ac07ebb0 con 0x7fa590039600 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 msgr2=0x7fa59003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 0x7fa59003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa59400ad30 tx=0x7fa5940093f0 comp rx=0 tx=0).stop 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 msgr2=0x7fa5ac1a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa59c011b60 tx=0x7fa59c011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 shutdown_connections 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fa590039600 0x7fa59003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 --2- 172.21.15.59:0/1354605393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5ac10eeb0 0x7fa5ac1a2780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 >> 172.21.15.59:0/1354605393 conn(0x7fa5ac07d940 msgr2=0x7fa5ac07e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:19.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 shutdown_connections 2024-04-02T17:02:19.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:19.383+0000 7fa5b1600700 1 -- 172.21.15.59:0/1354605393 wait complete. 2024-04-02T17:02:19.955 INFO:teuthology.orchestra.run.smithi059.stdout:Generating ssh key... 2024-04-02T17:02:21.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: Generating public/private rsa key pair. 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: Your identification has been saved in /tmp/tmph094jalb/key. 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: Your public key has been saved in /tmp/tmph094jalb/key.pub. 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: The key fingerprint is: 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: SHA256:RaRy0K2NcheMNV8C8TDfUv6dthHAqcOWOlOBqrg5vOo ceph-8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: The key's randomart image is: 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: +---[RSA 3072]----+ 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | .. *@oo.+ | 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | .o++O O. | 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | . ++o.O o. | 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | .++.oB . .+| 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | . .oS.+ . +o| 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | . . + . o| 2024-04-02T17:02:21.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | . o o . | 2024-04-02T17:02:21.551 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: | = | 2024-04-02T17:02:21.551 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: |.E..o | 2024-04-02T17:02:21.551 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: +----[SHA256]-----+ 2024-04-02T17:02:21.958 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.553+0000 7fe965400700 1 Processor -- start 2024-04-02T17:02:21.958 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.553+0000 7fe965400700 1 -- start start 2024-04-02T17:02:21.958 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.554+0000 7fe965400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.554+0000 7fe965400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe960108ce0 con 0x7fe9601082f0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.554+0000 7fe95e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.554+0000 7fe95e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52424/0 (socket says 172.21.15.59:52424) 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.554+0000 7fe95e800700 1 -- 172.21.15.59:0/1001661877 learned_addr learned my addr 172.21.15.59:0/1001661877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.555+0000 7fe95e800700 1 -- 172.21.15.59:0/1001661877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe960109450 con 0x7fe9601082f0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.555+0000 7fe95e800700 1 --2- 172.21.15.59:0/1001661877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9500098a0 tx=0x7fe950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febe5ba28beebd8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.555+0000 7fe95de00700 1 -- 172.21.15.59:0/1001661877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe950004160 con 0x7fe9601082f0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.555+0000 7fe95de00700 1 -- 172.21.15.59:0/1001661877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe950004300 con 0x7fe9601082f0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.555+0000 7fe95de00700 1 -- 172.21.15.59:0/1001661877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe950011b50 con 0x7fe9601082f0 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 -- 172.21.15.59:0/1001661877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 msgr2=0x7fe960108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 --2- 172.21.15.59:0/1001661877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9500098a0 tx=0x7fe950009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 -- 172.21.15.59:0/1001661877 shutdown_connections 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 --2- 172.21.15.59:0/1001661877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe960108710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:21.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 -- 172.21.15.59:0/1001661877 >> 172.21.15.59:0/1001661877 conn(0x7fe9601038b0 msgr2=0x7fe960105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.556+0000 7fe965400700 1 -- 172.21.15.59:0/1001661877 shutdown_connections 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.557+0000 7fe965400700 1 -- 172.21.15.59:0/1001661877 wait complete. 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.558+0000 7fe965400700 1 Processor -- start 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.558+0000 7fe965400700 1 -- start start 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.558+0000 7fe965400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.558+0000 7fe965400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe96010c800 con 0x7fe9601082f0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.559+0000 7fe95e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.559+0000 7fe95e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52440/0 (socket says 172.21.15.59:52440) 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.559+0000 7fe95e800700 1 -- 172.21.15.59:0/787142657 learned_addr learned my addr 172.21.15.59:0/787142657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.559+0000 7fe95e800700 1 -- 172.21.15.59:0/787142657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96019e960 con 0x7fe9601082f0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe95e800700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe950003b20 con 0x7fe9601082f0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe950003cc0 con 0x7fe9601082f0 2024-04-02T17:02:21.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe950024910 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96019ebf0 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.560+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a18e0 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.561+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe960063c20 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.561+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe950022020 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.562+0000 7fe957e00700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 0x7fe94403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.562+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe950011e00 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.562+0000 7fe955e00700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 0x7fe94403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.563+0000 7fe955e00700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 0x7fe94403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe948006fd0 tx=0x7fe948006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.568+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe95001ad80 con 0x7fe9601082f0 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:20.796+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe960104a50 con 0x7fe944039600 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.280+0000 7fe957e00700 1 -- 172.21.15.59:0/787142657 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe960104a50 con 0x7fe944039600 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 msgr2=0x7fe94403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 0x7fe94403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe948006fd0 tx=0x7fe948006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:21.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 msgr2=0x7fe96019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe950011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 shutdown_connections 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe944039600 0x7fe94403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 --2- 172.21.15.59:0/787142657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9601082f0 0x7fe96019e420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 >> 172.21.15.59:0/787142657 conn(0x7fe9601038b0 msgr2=0x7fe960104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 shutdown_connections 2024-04-02T17:02:21.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:21.283+0000 7fe965400700 1 -- 172.21.15.59:0/787142657 wait complete. 2024-04-02T17:02:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:22 smithi059 ceph-mon[31664]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:22 smithi059 ceph-mon[31664]: Generating ssh key... 2024-04-02T17:02:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:22 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:22 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.560+0000 7f8bc0a00700 1 Processor -- start 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bc0a00700 1 -- start start 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bc0a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bc0a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc063890 con 0x7f8bbc1089e0 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bbb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bbb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43860/0 (socket says 172.21.15.59:43860) 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.561+0000 7f8bbb400700 1 -- 172.21.15.59:0/3945457581 learned_addr learned my addr 172.21.15.59:0/3945457581 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:23.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.562+0000 7f8bbb400700 1 -- 172.21.15.59:0/3945457581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc10b310 con 0x7f8bbc1089e0 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.562+0000 7f8bbb400700 1 --2- 172.21.15.59:0/3945457581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009e80 tx=0x7f8bac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cfbd1a9dd4917a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.562+0000 7f8bba000700 1 -- 172.21.15.59:0/3945457581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bac017070 con 0x7f8bbc1089e0 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.562+0000 7f8bba000700 1 -- 172.21.15.59:0/3945457581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bac00b8c0 con 0x7f8bbc1089e0 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.562+0000 7f8bba000700 1 -- 172.21.15.59:0/3945457581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bac0045c0 con 0x7f8bbc1089e0 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.563+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3945457581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 msgr2=0x7f8bbc10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.563+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3945457581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009e80 tx=0x7f8bac00b330 comp rx=0 tx=0).stop 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.563+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3945457581 shutdown_connections 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.563+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3945457581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc10add0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.563+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3945457581 >> 172.21.15.59:0/3945457581 conn(0x7f8bbc1023b0 msgr2=0x7f8bbc1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.564+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3945457581 shutdown_connections 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.564+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3945457581 wait complete. 2024-04-02T17:02:23.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bc0a00700 1 Processor -- start 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bc0a00700 1 -- start start 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bc0a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bc0a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc10c2d0 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bbb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bbb400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43862/0 (socket says 172.21.15.59:43862) 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.565+0000 7f8bbb400700 1 -- 172.21.15.59:0/3454563005 learned_addr learned my addr 172.21.15.59:0/3454563005 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bbb400700 1 -- 172.21.15.59:0/3454563005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc19e640 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bbb400700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0072a0 tx=0x7f8bac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bac020020 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bac003b50 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bac017040 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc19e8d0 con 0x7f8bbc1089e0 2024-04-02T17:02:23.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.566+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a15c0 con 0x7f8bbc1089e0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.568+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8bac023020 con 0x7f8bbc1089e0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.568+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bbc063c20 con 0x7f8bbc1089e0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.568+0000 7f8bb3e00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 0x7f8ba003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.568+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8bac00f050 con 0x7f8bbc1089e0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.569+0000 7f8bbaa00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 0x7f8ba003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.569+0000 7f8bbaa00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 0x7f8ba003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8006fd0 tx=0x7f8ba8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.574+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8bac02a020 con 0x7f8bbc1089e0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.805+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8bbc103560 con 0x7f8ba00395b0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.807+0000 7f8bb3e00700 1 -- 172.21.15.59:0/3454563005 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8bbc103560 con 0x7f8ba00395b0 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 msgr2=0x7f8ba003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 0x7f8ba003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8006fd0 tx=0x7f8ba8006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 msgr2=0x7f8bbc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0072a0 tx=0x7f8bac00bd50 comp rx=0 tx=0).stop 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 shutdown_connections 2024-04-02T17:02:23.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f8ba00395b0 0x7f8ba003ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:23.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 --2- 172.21.15.59:0/3454563005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bbc1089e0 0x7f8bbc19e100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:23.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 >> 172.21.15.59:0/3454563005 conn(0x7f8bbc1023b0 msgr2=0x7f8bbc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:23.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 shutdown_connections 2024-04-02T17:02:23.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:22.810+0000 7f8bc0a00700 1 -- 172.21.15.59:0/3454563005 wait complete. 2024-04-02T17:02:23.469 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T17:02:23.470 INFO:teuthology.orchestra.run.smithi059.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T17:02:23.470 INFO:teuthology.orchestra.run.smithi059.stdout:Adding host smithi059... 2024-04-02T17:02:23.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:23 smithi059 ceph-mon[31664]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:24.615 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:24 smithi059 ceph-mon[31664]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:26.731 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:26 smithi059 ceph-mon[31664]: Deploying cephadm binary to smithi059 2024-04-02T17:02:31.807 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Added host 'smithi059' with addr '172.21.15.59' 2024-04-02T17:02:31.807 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.037+0000 7f5fc3a00700 1 Processor -- start 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.037+0000 7f5fc3a00700 1 -- start start 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.038+0000 7f5fc3a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.038+0000 7f5fc3a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc063910 con 0x7f5fbc106930 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.038+0000 7f5fc1000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.039+0000 7f5fc1000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43864/0 (socket says 172.21.15.59:43864) 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.039+0000 7f5fc1000700 1 -- 172.21.15.59:0/3334827518 learned_addr learned my addr 172.21.15.59:0/3334827518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.039+0000 7f5fc1000700 1 -- 172.21.15.59:0/3334827518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc107290 con 0x7f5fbc106930 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.039+0000 7f5fc1000700 1 --2- 172.21.15.59:0/3334827518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098a0 tx=0x7f5fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2dee629c44519f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.040+0000 7f5fbb400700 1 -- 172.21.15.59:0/3334827518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0004160 con 0x7f5fbc106930 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.040+0000 7f5fbb400700 1 -- 172.21.15.59:0/3334827518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fb0004300 con 0x7f5fbc106930 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.040+0000 7f5fbb400700 1 -- 172.21.15.59:0/3334827518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0011b50 con 0x7f5fbc106930 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.040+0000 7f5fc3a00700 1 -- 172.21.15.59:0/3334827518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 msgr2=0x7f5fbc106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.040+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/3334827518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098a0 tx=0x7f5fb0009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:31.808 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.041+0000 7f5fc3a00700 1 -- 172.21.15.59:0/3334827518 shutdown_connections 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.041+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/3334827518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc106d50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.041+0000 7f5fc3a00700 1 -- 172.21.15.59:0/3334827518 >> 172.21.15.59:0/3334827518 conn(0x7f5fbc1025d0 msgr2=0x7f5fbc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.041+0000 7f5fc3a00700 1 -- 172.21.15.59:0/3334827518 shutdown_connections 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.041+0000 7f5fc3a00700 1 -- 172.21.15.59:0/3334827518 wait complete. 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.042+0000 7f5fc3a00700 1 Processor -- start 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.042+0000 7f5fc3a00700 1 -- start start 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.042+0000 7f5fc3a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.042+0000 7f5fc3a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc108320 con 0x7f5fbc106930 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.043+0000 7f5fc1000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.043+0000 7f5fc1000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43868/0 (socket says 172.21.15.59:43868) 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.043+0000 7f5fc1000700 1 -- 172.21.15.59:0/4014018750 learned_addr learned my addr 172.21.15.59:0/4014018750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.043+0000 7f5fc1000700 1 -- 172.21.15.59:0/4014018750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc19e8b0 con 0x7f5fbc106930 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.043+0000 7f5fc1000700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0011b60 tx=0x7f5fb0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:31.809 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.044+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0003970 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.044+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fb0003b10 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.044+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc19eb40 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.044+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0024610 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.044+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc1a1830 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.045+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc063c20 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.045+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5fb001a9f0 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.046+0000 7f5fb9600700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 0x7f5fa403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.046+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5fb0011e00 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.046+0000 7f5fbbe00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 0x7f5fa403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.047+0000 7f5fbbe00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 0x7f5fa403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006fd0 tx=0x7f5fa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.051+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5fb000b3e0 con 0x7f5fbc106930 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:24.286+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}) -- 0x7f5fbc0008d0 con 0x7f5fa40395b0 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.108+0000 7f5fb9600700 1 -- 172.21.15.59:0/4014018750 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5fbc0008d0 con 0x7f5fa40395b0 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.112+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 msgr2=0x7f5fa403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:31.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.112+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 0x7f5fa403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006fd0 tx=0x7f5fa8006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.112+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 msgr2=0x7f5fbc19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.112+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0011b60 tx=0x7f5fb0011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 shutdown_connections 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5fa40395b0 0x7f5fa403ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 --2- 172.21.15.59:0/4014018750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc106930 0x7f5fbc19e370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 >> 172.21.15.59:0/4014018750 conn(0x7f5fbc1025d0 msgr2=0x7f5fbc194fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 shutdown_connections 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:31.113+0000 7f5fc3a00700 1 -- 172.21.15.59:0/4014018750 wait complete. 2024-04-02T17:02:31.811 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mon service... 2024-04-02T17:02:32.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:32 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:32.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:32 smithi059 ceph-mon[31664]: Added host smithi059 2024-04-02T17:02:32.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:32 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:33.729 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T17:02:33.729 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.460+0000 7f563ae00700 1 Processor -- start 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.460+0000 7f563ae00700 1 -- start start 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.461+0000 7f563ae00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.461+0000 7f563ae00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56341079c0 con 0x7f5634106fd0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.461+0000 7f5633e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.461+0000 7f5633e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51642/0 (socket says 172.21.15.59:51642) 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.461+0000 7f5633e00700 1 -- 172.21.15.59:0/2362638491 learned_addr learned my addr 172.21.15.59:0/2362638491 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.462+0000 7f5633e00700 1 -- 172.21.15.59:0/2362638491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56341081d0 con 0x7f5634106fd0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.462+0000 7f5633e00700 1 --2- 172.21.15.59:0/2362638491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56240098a0 tx=0x7f5624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec5416e35818b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.462+0000 7f5632a00700 1 -- 172.21.15.59:0/2362638491 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624004160 con 0x7f5634106fd0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.462+0000 7f5632a00700 1 -- 172.21.15.59:0/2362638491 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5624004300 con 0x7f5634106fd0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f5632a00700 1 -- 172.21.15.59:0/2362638491 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624011b50 con 0x7f5634106fd0 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 -- 172.21.15.59:0/2362638491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 msgr2=0x7f56341073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:33.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 --2- 172.21.15.59:0/2362638491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56240098a0 tx=0x7f5624009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 -- 172.21.15.59:0/2362638491 shutdown_connections 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 --2- 172.21.15.59:0/2362638491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f56341073f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 -- 172.21.15.59:0/2362638491 >> 172.21.15.59:0/2362638491 conn(0x7f56341025d0 msgr2=0x7f56341049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.463+0000 7f563ae00700 1 -- 172.21.15.59:0/2362638491 shutdown_connections 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.464+0000 7f563ae00700 1 -- 172.21.15.59:0/2362638491 wait complete. 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.465+0000 7f563ae00700 1 Processor -- start 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.465+0000 7f563ae00700 1 -- start start 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.465+0000 7f563ae00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.465+0000 7f563ae00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f563410b580 con 0x7f5634106fd0 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.466+0000 7f5633e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.466+0000 7f5633e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51646/0 (socket says 172.21.15.59:51646) 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.466+0000 7f5633e00700 1 -- 172.21.15.59:0/3578542897 learned_addr learned my addr 172.21.15.59:0/3578542897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.466+0000 7f5633e00700 1 -- 172.21.15.59:0/3578542897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563419e900 con 0x7f5634106fd0 2024-04-02T17:02:33.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.466+0000 7f5633e00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5624011850 tx=0x7f5624011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.467+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624003af0 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.467+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5624020070 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.467+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563419eb90 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.467+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56240195a0 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.467+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a1880 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.468+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5624019850 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.468+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634063c20 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.469+0000 7f5630c00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 0x7f561803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.469+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5624011e00 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.469+0000 7f5633400700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 0x7f561803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.470+0000 7f5633400700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 0x7f561803bb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5620006fd0 tx=0x7f5620006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.475+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f562400e260 con 0x7f5634106fd0 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.744+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f56340008d0 con 0x7f5618039660 2024-04-02T17:02:33.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.751+0000 7f5630c00700 1 -- 172.21.15.59:0/3578542897 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f56340008d0 con 0x7f5618039660 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 msgr2=0x7f561803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 0x7f561803bb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5620006fd0 tx=0x7f5620006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 msgr2=0x7f563419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5624011850 tx=0x7f5624011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 shutdown_connections 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f5618039660 0x7f561803bb20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 --2- 172.21.15.59:0/3578542897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634106fd0 0x7f563419e3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.753+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 >> 172.21.15.59:0/3578542897 conn(0x7f56341025d0 msgr2=0x7f5634103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.754+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 shutdown_connections 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:32.754+0000 7f563ae00700 1 -- 172.21.15.59:0/3578542897 wait complete. 2024-04-02T17:02:33.733 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mgr service... 2024-04-02T17:02:34.024 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:33 smithi059 ceph-mon[31664]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:34.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:33 smithi059 ceph-mon[31664]: Saving service mon spec with placement count:5 2024-04-02T17:02:34.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:33 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: Saving service mgr spec with placement count:2 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:35.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:35 smithi059 ceph-mon[31664]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:02:35.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T17:02:35.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.934+0000 7fe236000700 1 Processor -- start 2024-04-02T17:02:35.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.934+0000 7fe236000700 1 -- start start 2024-04-02T17:02:35.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.934+0000 7fe236000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:35.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.934+0000 7fe236000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe230063890 con 0x7fe2301089e0 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.935+0000 7fe234c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.935+0000 7fe234c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51656/0 (socket says 172.21.15.59:51656) 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.935+0000 7fe234c00700 1 -- 172.21.15.59:0/3669923677 learned_addr learned my addr 172.21.15.59:0/3669923677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.936+0000 7fe234c00700 1 -- 172.21.15.59:0/3669923677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23010b310 con 0x7fe2301089e0 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.936+0000 7fe234c00700 1 --2- 172.21.15.59:0/3669923677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2240098a0 tx=0x7fe224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e1f9066dc3c791 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.936+0000 7fe22f400700 1 -- 172.21.15.59:0/3669923677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe224004160 con 0x7fe2301089e0 2024-04-02T17:02:35.831 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.936+0000 7fe22f400700 1 -- 172.21.15.59:0/3669923677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe224004300 con 0x7fe2301089e0 2024-04-02T17:02:35.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.936+0000 7fe22f400700 1 -- 172.21.15.59:0/3669923677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe224011b20 con 0x7fe2301089e0 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 -- 172.21.15.59:0/3669923677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 msgr2=0x7fe23010add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 --2- 172.21.15.59:0/3669923677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2240098a0 tx=0x7fe224009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 -- 172.21.15.59:0/3669923677 shutdown_connections 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 --2- 172.21.15.59:0/3669923677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23010add0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 -- 172.21.15.59:0/3669923677 >> 172.21.15.59:0/3669923677 conn(0x7fe2301023b0 msgr2=0x7fe2301047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 -- 172.21.15.59:0/3669923677 shutdown_connections 2024-04-02T17:02:35.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.937+0000 7fe236000700 1 -- 172.21.15.59:0/3669923677 wait complete. 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.938+0000 7fe236000700 1 Processor -- start 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe236000700 1 -- start start 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe236000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe236000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010c310 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe234c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe234c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51658/0 (socket says 172.21.15.59:51658) 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.939+0000 7fe234c00700 1 -- 172.21.15.59:0/2938498009 learned_addr learned my addr 172.21.15.59:0/2938498009 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe234c00700 1 -- 172.21.15.59:0/2938498009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23019c410 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe234c00700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2240117e0 tx=0x7fe224011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe224003ba0 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22401a910 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2240248f0 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23019c6a0 con 0x7fe2301089e0 2024-04-02T17:02:35.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.940+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23019cb00 con 0x7fe2301089e0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.942+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe230063c20 con 0x7fe2301089e0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.942+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe224022020 con 0x7fe2301089e0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.942+0000 7fe22d600700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 0x7fe21803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.943+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe22404dbd0 con 0x7fe2301089e0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.943+0000 7fe22fe00700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 0x7fe21803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.944+0000 7fe22fe00700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 0x7fe21803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe21c006fd0 tx=0x7fe21c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:34.948+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe22400b3e0 con 0x7fe2301089e0 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.193+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe2301035b0 con 0x7fe218039600 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.199+0000 7fe22d600700 1 -- 172.21.15.59:0/2938498009 <== mgr.14120 v2:172.21.15.59:6800/2274685378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe2301035b0 con 0x7fe218039600 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.201+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 msgr2=0x7fe21803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.201+0000 7fe236000700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 0x7fe21803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe21c006fd0 tx=0x7fe21c006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.201+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 msgr2=0x7fe23019bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.201+0000 7fe236000700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2240117e0 tx=0x7fe224011ba0 comp rx=0 tx=0).stop 2024-04-02T17:02:35.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 shutdown_connections 2024-04-02T17:02:35.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fe218039600 0x7fe21803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:35.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 --2- 172.21.15.59:0/2938498009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2301089e0 0x7fe23019bed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:35.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 >> 172.21.15.59:0/2938498009 conn(0x7fe2301023b0 msgr2=0x7fe230102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:35.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 shutdown_connections 2024-04-02T17:02:35.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:35.202+0000 7fe236000700 1 -- 172.21.15.59:0/2938498009 wait complete. 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.408+0000 7fbdf7c00700 1 Processor -- start 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.408+0000 7fbdf7c00700 1 -- start start 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.408+0000 7fbdf7c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.408+0000 7fbdf7c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0108d20 con 0x7fbdf0108330 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.408+0000 7fbdf5200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.409+0000 7fbdf5200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51670/0 (socket says 172.21.15.59:51670) 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.409+0000 7fbdf5200700 1 -- 172.21.15.59:0/3959672867 learned_addr learned my addr 172.21.15.59:0/3959672867 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:37.446 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.409+0000 7fbdf5200700 1 -- 172.21.15.59:0/3959672867 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf0109490 con 0x7fbdf0108330 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.409+0000 7fbdf5200700 1 --2- 172.21.15.59:0/3959672867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbde40098a0 tx=0x7fbde4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed32ced1908f6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.410+0000 7fbdef400700 1 -- 172.21.15.59:0/3959672867 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4004160 con 0x7fbdf0108330 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.410+0000 7fbdef400700 1 -- 172.21.15.59:0/3959672867 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde4004300 con 0x7fbdf0108330 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.410+0000 7fbdef400700 1 -- 172.21.15.59:0/3959672867 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4011b20 con 0x7fbdf0108330 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3959672867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 msgr2=0x7fbdf0108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3959672867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbde40098a0 tx=0x7fbde4009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3959672867 shutdown_connections 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3959672867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf0108750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3959672867 >> 172.21.15.59:0/3959672867 conn(0x7fbdf0103950 msgr2=0x7fbdf0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3959672867 shutdown_connections 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.411+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3959672867 wait complete. 2024-04-02T17:02:37.447 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.412+0000 7fbdf7c00700 1 Processor -- start 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.412+0000 7fbdf7c00700 1 -- start start 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.412+0000 7fbdf7c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.412+0000 7fbdf7c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf010c840 con 0x7fbdf0108330 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.413+0000 7fbdf5200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.413+0000 7fbdf5200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51674/0 (socket says 172.21.15.59:51674) 2024-04-02T17:02:37.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.413+0000 7fbdf5200700 1 -- 172.21.15.59:0/3031438123 learned_addr learned my addr 172.21.15.59:0/3031438123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:37.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.413+0000 7fbdf5200700 1 -- 172.21.15.59:0/3031438123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf019e920 con 0x7fbdf0108330 2024-04-02T17:02:37.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.413+0000 7fbdf5200700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde40117e0 tx=0x7fbde4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:37.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.414+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4003bf0 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.414+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbde401a910 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.414+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf019ebb0 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.414+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde4024970 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.414+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf019f010 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.415+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbde4022020 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.415+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf0063c20 con 0x7fbdf0108330 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.415+0000 7fbded600700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 0x7fbdd803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:37.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.416+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbde404dbd0 con 0x7fbdf0108330 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.416+0000 7fbdefe00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 0x7fbdd803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.416+0000 7fbdefe00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 0x7fbdd803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006fd0 tx=0x7fbddc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.421+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbde404d520 con 0x7fbdf0108330 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.672+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbdf01a19e0 con 0x7fbdf0108330 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.676+0000 7fbded600700 1 -- 172.21.15.59:0/3031438123 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbde402d910 con 0x7fbdf0108330 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.679+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 msgr2=0x7fbdd803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:37.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.679+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 0x7fbdd803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006fd0 tx=0x7fbddc006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.679+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 msgr2=0x7fbdf019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.679+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbde40117e0 tx=0x7fbde4011ba0 comp rx=0 tx=0).stop 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 shutdown_connections 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fbdd8039600 0x7fbdd803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 --2- 172.21.15.59:0/3031438123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdf0108330 0x7fbdf019e3e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 >> 172.21.15.59:0/3031438123 conn(0x7fbdf0103950 msgr2=0x7fbdf01043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 shutdown_connections 2024-04-02T17:02:37.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:36.680+0000 7fbdf7c00700 1 -- 172.21.15.59:0/3031438123 wait complete. 2024-04-02T17:02:37.971 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3031438123' entity='client.admin' 2024-04-02T17:02:37.971 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:37 smithi059 ceph-mon[31664]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:02:39.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:39 smithi059 ceph-mon[31664]: from='mgr.14120 172.21.15.59:0/344095751' entity='mgr.x' 2024-04-02T17:02:39.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:39 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2064264524' entity='client.admin' 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.290+0000 7fc586000700 1 Processor -- start 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.290+0000 7fc586000700 1 -- start start 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.290+0000 7fc586000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.290+0000 7fc586000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010aed0 con 0x7fc58010a4e0 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.290+0000 7fc57f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:39.160 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.291+0000 7fc57f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51682/0 (socket says 172.21.15.59:51682) 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.291+0000 7fc57f400700 1 -- 172.21.15.59:0/3150725672 learned_addr learned my addr 172.21.15.59:0/3150725672 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.291+0000 7fc57f400700 1 -- 172.21.15.59:0/3150725672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58010b690 con 0x7fc58010a4e0 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.291+0000 7fc57f400700 1 --2- 172.21.15.59:0/3150725672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5700098a0 tx=0x7fc570009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211a6c1b16f5b60f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.292+0000 7fc57e000700 1 -- 172.21.15.59:0/3150725672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570004160 con 0x7fc58010a4e0 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.292+0000 7fc57e000700 1 -- 172.21.15.59:0/3150725672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc570004300 con 0x7fc58010a4e0 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.292+0000 7fc57e000700 1 -- 172.21.15.59:0/3150725672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570011b20 con 0x7fc58010a4e0 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 -- 172.21.15.59:0/3150725672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 msgr2=0x7fc58010a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 --2- 172.21.15.59:0/3150725672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5700098a0 tx=0x7fc570009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 -- 172.21.15.59:0/3150725672 shutdown_connections 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 --2- 172.21.15.59:0/3150725672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58010a900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 -- 172.21.15.59:0/3150725672 >> 172.21.15.59:0/3150725672 conn(0x7fc58007d930 msgr2=0x7fc580108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 -- 172.21.15.59:0/3150725672 shutdown_connections 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.293+0000 7fc586000700 1 -- 172.21.15.59:0/3150725672 wait complete. 2024-04-02T17:02:39.161 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.294+0000 7fc586000700 1 Processor -- start 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.294+0000 7fc586000700 1 -- start start 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc586000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc586000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc58010ea40 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc57f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc57f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51692/0 (socket says 172.21.15.59:51692) 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc57f400700 1 -- 172.21.15.59:0/2064264524 learned_addr learned my addr 172.21.15.59:0/2064264524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.295+0000 7fc57f400700 1 -- 172.21.15.59:0/2064264524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58019e900 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc57f400700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5700117e0 tx=0x7fc570011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570003bf0 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc57001a910 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58019eb90 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5700249a0 con 0x7fc58010a4e0 2024-04-02T17:02:39.162 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.296+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58019eff0 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.297+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc580063c20 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.298+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc570022020 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.298+0000 7fc577e00700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 0x7fc56403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.298+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc57004da90 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.298+0000 7fc57ea00700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 0x7fc56403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.299+0000 7fc57ea00700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 0x7fc56403ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc56c006fd0 tx=0x7fc56c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.304+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc570011e20 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.530+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc5801a19c0 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.535+0000 7fc577e00700 1 -- 172.21.15.59:0/2064264524 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc57004d3e0 con 0x7fc58010a4e0 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.538+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 msgr2=0x7fc56403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.538+0000 7fc586000700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 0x7fc56403ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc56c006fd0 tx=0x7fc56c006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.538+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 msgr2=0x7fc58019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:39.163 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.538+0000 7fc586000700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5700117e0 tx=0x7fc570011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 shutdown_connections 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fc564039560 0x7fc56403ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 --2- 172.21.15.59:0/2064264524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc58010a4e0 0x7fc58019e3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 >> 172.21.15.59:0/2064264524 conn(0x7fc58007d930 msgr2=0x7fc580107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 shutdown_connections 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:38.539+0000 7fc586000700 1 -- 172.21.15.59:0/2064264524 wait complete. 2024-04-02T17:02:39.164 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling the dashboard module... 2024-04-02T17:02:40.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:40 smithi059 ceph-mon[31664]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:02:40.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4102327476' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T17:02:40.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: ignoring --setuser ceph since I am not root 2024-04-02T17:02:40.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: ignoring --setgroup ceph since I am not root 2024-04-02T17:02:40.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.728+0000 7f97c9a00700 1 Processor -- start 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.729+0000 7f97c9a00700 1 -- start start 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.729+0000 7f97c9a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.729+0000 7f97c9a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97c410ad90 con 0x7f97c410a3a0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.730+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.730+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51706/0 (socket says 172.21.15.59:51706) 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.730+0000 7f97c2a00700 1 -- 172.21.15.59:0/655146135 learned_addr learned my addr 172.21.15.59:0/655146135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.730+0000 7f97c2a00700 1 -- 172.21.15.59:0/655146135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c410b550 con 0x7f97c410a3a0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.730+0000 7f97c2a00700 1 --2- 172.21.15.59:0/655146135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098a0 tx=0x7f97b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324e8d42bcfed545 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.731+0000 7f97c1600700 1 -- 172.21.15.59:0/655146135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b4004160 con 0x7f97c410a3a0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.731+0000 7f97c1600700 1 -- 172.21.15.59:0/655146135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4004300 con 0x7f97c410a3a0 2024-04-02T17:02:40.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.731+0000 7f97c1600700 1 -- 172.21.15.59:0/655146135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b4011b20 con 0x7f97c410a3a0 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 -- 172.21.15.59:0/655146135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 msgr2=0x7f97c410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 --2- 172.21.15.59:0/655146135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098a0 tx=0x7f97b4009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 -- 172.21.15.59:0/655146135 shutdown_connections 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 --2- 172.21.15.59:0/655146135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c410a7c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 -- 172.21.15.59:0/655146135 >> 172.21.15.59:0/655146135 conn(0x7f97c4105a20 msgr2=0x7f97c4107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 -- 172.21.15.59:0/655146135 shutdown_connections 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.732+0000 7f97c9a00700 1 -- 172.21.15.59:0/655146135 wait complete. 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.733+0000 7f97c9a00700 1 Processor -- start 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.733+0000 7f97c9a00700 1 -- start start 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c9a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c9a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97c410e900 con 0x7f97c410a3a0 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:40.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51720/0 (socket says 172.21.15.59:51720) 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c2a00700 1 -- 172.21.15.59:0/4102327476 learned_addr learned my addr 172.21.15.59:0/4102327476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.734+0000 7f97c2a00700 1 -- 172.21.15.59:0/4102327476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c419e840 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97c2a00700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c00 tx=0x7f97b4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b4003b60 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b401a910 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c419ead0 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b4024930 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.735+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c419ef30 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.736+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c4063c20 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.736+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f97b4022020 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.737+0000 7f97bb400700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 0x7f97a803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.737+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f97b404d8a0 con 0x7f97c410a3a0 2024-04-02T17:02:40.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.737+0000 7f97c2000700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 0x7f97a803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.738+0000 7f97c2000700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 0x7f97a803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:39.743+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f97b402ddb0 con 0x7f97c410a3a0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.043+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f97c41a19c0 con 0x7f97c410a3a0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.078+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f97b400b3e0 con 0x7f97c410a3a0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.081+0000 7f97bb400700 1 -- 172.21.15.59:0/4102327476 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f97b4024ad0 con 0x7f97c410a3a0 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.081+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 msgr2=0x7f97a803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.081+0000 7f97c9a00700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 0x7f97a803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.081+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 msgr2=0x7f97c419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.081+0000 7f97c9a00700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c00 tx=0x7f97b4011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.082+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 shutdown_connections 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.082+0000 7f97c9a00700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f97a8039600 0x7f97a803bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.082+0000 7f97c9a00700 1 --2- 172.21.15.59:0/4102327476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c410a3a0 0x7f97c419e300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:40.779 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.082+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 >> 172.21.15.59:0/4102327476 conn(0x7f97c4105a20 msgr2=0x7f97c4106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:40.780 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.082+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 shutdown_connections 2024-04-02T17:02:40.780 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:40.083+0000 7f97c9a00700 1 -- 172.21.15.59:0/4102327476 wait complete. 2024-04-02T17:02:41.043 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:40.931+0000 7f4e6ff28200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T17:02:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4102327476' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T17:02:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:41 smithi059 ceph-mon[31664]: mgrmap e9: x(active, since 24s) 2024-04-02T17:02:41.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:41.058+0000 7f4e6ff28200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T17:02:42.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/327107298' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T17:02:42.229 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:41.918+0000 7f4e6ff28200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T17:02:42.229 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.043+0000 7f4e6ff28200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T17:02:42.229 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.228+0000 7f4e6ff28200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T17:02:42.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:42.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.328+0000 7fd161400700 1 Processor -- start 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.328+0000 7fd161400700 1 -- start start 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.328+0000 7fd161400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.328+0000 7fd161400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c108800 con 0x7fd15c107ea0 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.329+0000 7fd15be00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.329+0000 7fd15be00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34282/0 (socket says 172.21.15.59:34282) 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.329+0000 7fd15be00700 1 -- 172.21.15.59:0/2596271053 learned_addr learned my addr 172.21.15.59:0/2596271053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.329+0000 7fd15be00700 1 -- 172.21.15.59:0/2596271053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c109090 con 0x7fd15c107ea0 2024-04-02T17:02:42.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.330+0000 7fd15be00700 1 --2- 172.21.15.59:0/2596271053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0098a0 tx=0x7fd14c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffff51d1a6b0372 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.330+0000 7fd15aa00700 1 -- 172.21.15.59:0/2596271053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd14c004160 con 0x7fd15c107ea0 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.330+0000 7fd15aa00700 1 -- 172.21.15.59:0/2596271053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c004300 con 0x7fd15c107ea0 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.330+0000 7fd15aa00700 1 -- 172.21.15.59:0/2596271053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd14c011b20 con 0x7fd15c107ea0 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 -- 172.21.15.59:0/2596271053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 msgr2=0x7fd15c1082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 --2- 172.21.15.59:0/2596271053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0098a0 tx=0x7fd14c009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 -- 172.21.15.59:0/2596271053 shutdown_connections 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 --2- 172.21.15.59:0/2596271053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c1082c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 -- 172.21.15.59:0/2596271053 >> 172.21.15.59:0/2596271053 conn(0x7fd15c1034d0 msgr2=0x7fd15c1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 -- 172.21.15.59:0/2596271053 shutdown_connections 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.331+0000 7fd161400700 1 -- 172.21.15.59:0/2596271053 wait complete. 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.332+0000 7fd161400700 1 Processor -- start 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.332+0000 7fd161400700 1 -- start start 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd161400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd161400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c10c350 con 0x7fd15c107ea0 2024-04-02T17:02:42.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd15be00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd15be00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34286/0 (socket says 172.21.15.59:34286) 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd15be00700 1 -- 172.21.15.59:0/327107298 learned_addr learned my addr 172.21.15.59:0/327107298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.333+0000 7fd15be00700 1 -- 172.21.15.59:0/327107298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c19e6a0 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd15be00700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0117e0 tx=0x7fd14c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd14c003bf0 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c01a910 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c19e930 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd14c024970 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.334+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c19ed90 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.335+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fd14c022020 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.335+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c063c20 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.336+0000 7fd158c00700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 0x7fd14003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.336+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd14c04df70 con 0x7fd15c107ea0 2024-04-02T17:02:42.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.336+0000 7fd15b400700 1 -- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 msgr2=0x7fd14003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.336+0000 7fd15b400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 0x7fd14003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.345+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd14c02d910 con 0x7fd15c107ea0 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.537+0000 7fd15b400700 1 -- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 msgr2=0x7fd14003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.537+0000 7fd15b400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 0x7fd14003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.633+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd15c1a1620 con 0x7fd15c107ea0 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.634+0000 7fd158c00700 1 -- 172.21.15.59:0/327107298 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd14c02d3f0 con 0x7fd15c107ea0 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 msgr2=0x7fd14003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 0x7fd14003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 msgr2=0x7fd15c19e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0117e0 tx=0x7fd14c011ba0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 shutdown_connections 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7fd140039600 0x7fd14003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 --2- 172.21.15.59:0/327107298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd15c107ea0 0x7fd15c19e160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:42.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.637+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 >> 172.21.15.59:0/327107298 conn(0x7fd15c1034d0 msgr2=0x7fd15c104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:42.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.638+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 shutdown_connections 2024-04-02T17:02:42.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:41.638+0000 7fd161400700 1 -- 172.21.15.59:0/327107298 wait complete. 2024-04-02T17:02:42.295 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-04-02T17:02:42.295 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 9... 2024-04-02T17:02:42.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.361+0000 7f4e6ff28200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T17:02:42.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.478+0000 7f4e6ff28200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T17:02:42.898 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.827+0000 7f4e6ff28200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T17:02:43.219 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:42.975+0000 7f4e6ff28200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T17:02:43.219 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:43.091+0000 7f4e6ff28200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T17:02:43.220 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:43.219+0000 7f4e6ff28200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T17:02:43.508 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:43.444+0000 7f4e6ff28200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T17:02:45.256 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:44.801+0000 7f4e6ff28200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T17:02:45.256 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:44.982+0000 7f4e6ff28200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T17:02:45.538 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:45.416+0000 7f4e6ff28200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T17:02:45.539 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:45.538+0000 7f4e6ff28200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T17:02:46.059 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:45.806+0000 7f4e6ff28200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:02:46.454 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:46.059+0000 7f4e6ff28200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T17:02:46.454 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:46.453+0000 7f4e6ff28200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:02:47.048 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:46.780+0000 7f4e6ff28200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T17:02:47.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:47.099+0000 7f4e6ff28200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T17:02:48.548 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:48.182+0000 7f4e6ff28200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T17:02:49.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:48.913+0000 7f4e6ff28200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T17:02:49.299 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:49.051+0000 7f4e6ff28200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T17:02:49.798 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:02:49.347+0000 7f4e6ff28200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: Active manager daemon x restarted 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: Activating manager daemon x 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: mgrmap e10: x(active, starting, since 0.00519066s) 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T17:02:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T17:02:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:49 smithi059 ceph-mon[31664]: Manager daemon x is now available 2024-04-02T17:02:50.681 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:02:50.681 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:49] ENGINE Bus STARTING 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:49] ENGINE Serving on https://172.21.15.59:7150 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:50] ENGINE Serving on http://172.21.15.59:8765 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: [02/Apr/2024:17:02:50] ENGINE Bus STARTED 2024-04-02T17:02:50.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:50 smithi059 ceph-mon[31664]: mgrmap e11: x(active, since 1.00928s) 2024-04-02T17:02:50.959 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-04-02T17:02:50.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T17:02:50.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T17:02:50.960 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-04-02T17:02:50.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.898+0000 7f6eff600700 1 Processor -- start 2024-04-02T17:02:50.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.898+0000 7f6eff600700 1 -- start start 2024-04-02T17:02:50.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.899+0000 7f6eff600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:50.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.899+0000 7f6eff600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef807b720 con 0x7f6ef807ccf0 2024-04-02T17:02:50.961 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.899+0000 7f6efcc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:50.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.899+0000 7f6efcc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34294/0 (socket says 172.21.15.59:34294) 2024-04-02T17:02:50.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.899+0000 7f6efcc00700 1 -- 172.21.15.59:0/3246992137 learned_addr learned my addr 172.21.15.59:0/3246992137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:50.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.900+0000 7f6efcc00700 1 -- 172.21.15.59:0/3246992137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef807b8a0 con 0x7f6ef807ccf0 2024-04-02T17:02:50.962 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.900+0000 7f6efcc00700 1 --2- 172.21.15.59:0/3246992137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0098a0 tx=0x7f6eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17a55bbc73f741e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:50.963 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6ef7400700 1 -- 172.21.15.59:0/3246992137 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec004160 con 0x7f6ef807ccf0 2024-04-02T17:02:50.963 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6ef7400700 1 -- 172.21.15.59:0/3246992137 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec004300 con 0x7f6ef807ccf0 2024-04-02T17:02:50.963 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6ef7400700 1 -- 172.21.15.59:0/3246992137 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec011b20 con 0x7f6ef807ccf0 2024-04-02T17:02:50.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 -- 172.21.15.59:0/3246992137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 msgr2=0x7f6ef807b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:50.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 --2- 172.21.15.59:0/3246992137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0098a0 tx=0x7f6eec009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 -- 172.21.15.59:0/3246992137 shutdown_connections 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 --2- 172.21.15.59:0/3246992137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef807b150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 -- 172.21.15.59:0/3246992137 >> 172.21.15.59:0/3246992137 conn(0x7f6ef8103640 msgr2=0x7f6ef8105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.901+0000 7f6eff600700 1 -- 172.21.15.59:0/3246992137 shutdown_connections 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.902+0000 7f6eff600700 1 -- 172.21.15.59:0/3246992137 wait complete. 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6eff600700 1 Processor -- start 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6eff600700 1 -- start start 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6eff600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6eff600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef810e760 con 0x7f6ef807ccf0 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6efcc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6efcc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34306/0 (socket says 172.21.15.59:34306) 2024-04-02T17:02:50.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.903+0000 7f6efcc00700 1 -- 172.21.15.59:0/1713223572 learned_addr learned my addr 172.21.15.59:0/1713223572 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6efcc00700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef819e800 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6efcc00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6eec011b60 tx=0x7f6eec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec003b60 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec01a910 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef819ea90 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.904+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec024930 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.905+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef819eef0 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6eec022020 con 0x7f6ef807ccf0 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef5600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee003c230 con 0x7f6ee0039660 2024-04-02T17:02:50.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6eec04d9f0 con 0x7f6ef807ccf0 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:42.906+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:43.107+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:43.107+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:43.508+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:43.508+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:44.309+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:44.309+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T17:02:50.969 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:45.911+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.972 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:45.911+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T17:02:50.972 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:49.115+0000 7f6ef7e00700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2274685378 2024-04-02T17:02:50.972 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:49.115+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-02T17:02:50.972 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:49.355+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6eec02d3f0 con 0x7f6ef807ccf0 2024-04-02T17:02:50.972 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:49.355+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 msgr2=0x7f6ee003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:49.355+0000 7f6ef5600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.356+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6eec029c50 con 0x7f6ef807ccf0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.357+0000 7f6ef5600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 0x7f6ee003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.357+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee003c230 con 0x7f6ee003cff0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.357+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 0x7f6ee003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.360+0000 7f6ef7e00700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 0x7f6ee003f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4003a10 tx=0x7f6ee4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.361+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f6ee003c230 con 0x7f6ee003cff0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.370+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6ef8063c20 con 0x7f6ee003cff0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.370+0000 7f6ef5600700 1 -- 172.21.15.59:0/1713223572 <== mgr.14152 v2:172.21.15.59:6800/1634390325 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6ef8063c20 con 0x7f6ee003cff0 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 msgr2=0x7f6ee003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 0x7f6ee003f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4003a10 tx=0x7f6ee4009270 comp rx=0 tx=0).stop 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 msgr2=0x7f6ef819e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6eec011b60 tx=0x7f6eec011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:50.973 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 shutdown_connections 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ee003cff0 0x7f6ee003f3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef807ccf0 0x7f6ef819e2c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 --2- 172.21.15.59:0/1713223572 >> [v2:172.21.15.59:6800/2274685378,v1:172.21.15.59:6801/2274685378] conn(0x7f6ee0039660 0x7f6ee003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 >> 172.21.15.59:0/1713223572 conn(0x7f6ef8103640 msgr2=0x7f6ef8104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 shutdown_connections 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:50.371+0000 7f6eff600700 1 -- 172.21.15.59:0/1713223572 wait complete. 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 9 is available 2024-04-02T17:02:50.974 INFO:teuthology.orchestra.run.smithi059.stdout:Generating a dashboard self-signed certificate... 2024-04-02T17:02:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:51 smithi059 ceph-mon[31664]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T17:02:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:51 smithi059 ceph-mon[31664]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T17:02:52.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:52 smithi059 ceph-mon[31664]: mgrmap e12: x(active, since 2s) 2024-04-02T17:02:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:02:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:52 smithi059 ceph-mon[31664]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:02:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:02:52.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T17:02:52.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.950+0000 7f20e7e00700 1 Processor -- start 2024-04-02T17:02:52.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.950+0000 7f20e7e00700 1 -- start start 2024-04-02T17:02:52.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.951+0000 7f20e7e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:52.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.951+0000 7f20e7e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20e007c890 con 0x7f20e007de10 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.952+0000 7f20e5400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.952+0000 7f20e5400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38074/0 (socket says 172.21.15.59:38074) 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.952+0000 7f20e5400700 1 -- 172.21.15.59:0/905687308 learned_addr learned my addr 172.21.15.59:0/905687308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.952+0000 7f20e5400700 1 -- 172.21.15.59:0/905687308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e007ca10 con 0x7f20e007de10 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.952+0000 7f20e5400700 1 --2- 172.21.15.59:0/905687308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d40098a0 tx=0x7f20d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323fde77b54e2be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.953+0000 7f20dfe00700 1 -- 172.21.15.59:0/905687308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d4004160 con 0x7f20e007de10 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.953+0000 7f20dfe00700 1 -- 172.21.15.59:0/905687308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4004300 con 0x7f20e007de10 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.953+0000 7f20dfe00700 1 -- 172.21.15.59:0/905687308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d4011b20 con 0x7f20e007de10 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 -- 172.21.15.59:0/905687308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 msgr2=0x7f20e007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:52.921 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 --2- 172.21.15.59:0/905687308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d40098a0 tx=0x7f20d4009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 -- 172.21.15.59:0/905687308 shutdown_connections 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 --2- 172.21.15.59:0/905687308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e007c2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 -- 172.21.15.59:0/905687308 >> 172.21.15.59:0/905687308 conn(0x7f20e0103950 msgr2=0x7f20e0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 -- 172.21.15.59:0/905687308 shutdown_connections 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.954+0000 7f20e7e00700 1 -- 172.21.15.59:0/905687308 wait complete. 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.955+0000 7f20e7e00700 1 Processor -- start 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.955+0000 7f20e7e00700 1 -- start start 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e7e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e7e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20e010e8f0 con 0x7f20e007de10 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e5400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e5400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38082/0 (socket says 172.21.15.59:38082) 2024-04-02T17:02:52.925 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e5400700 1 -- 172.21.15.59:0/3512553889 learned_addr learned my addr 172.21.15.59:0/3512553889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:52.925 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.956+0000 7f20e5400700 1 -- 172.21.15.59:0/3512553889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e019e980 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.957+0000 7f20e5400700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20d4011b60 tx=0x7f20d4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.957+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d4003b60 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.957+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d401a910 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.957+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e019ec10 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.957+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d4024930 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.958+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e019f070 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.959+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e0063c20 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.959+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f20d4022020 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.960+0000 7f20de000700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 0x7f20c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.960+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f20d404da70 con 0x7f20e007de10 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.960+0000 7f20e4a00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 0x7f20c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:52.926 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.961+0000 7f20e4a00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 0x7f20c803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006fd0 tx=0x7f20cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:51.966+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f20d4024c70 con 0x7f20e007de10 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.210+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f20e0104b90 con 0x7f20c8039600 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.323+0000 7f20de000700 1 -- 172.21.15.59:0/3512553889 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f20e0104b90 con 0x7f20c8039600 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.326+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 msgr2=0x7f20c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.326+0000 7f20e7e00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 0x7f20c803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006fd0 tx=0x7f20cc006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.326+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 msgr2=0x7f20e019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.326+0000 7f20e7e00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20d4011b60 tx=0x7f20d4011c40 comp rx=0 tx=0).stop 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 shutdown_connections 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f20c8039600 0x7f20c803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 --2- 172.21.15.59:0/3512553889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e007de10 0x7f20e019e440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 >> 172.21.15.59:0/3512553889 conn(0x7f20e0103950 msgr2=0x7f20e0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 shutdown_connections 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:52.327+0000 7f20e7e00700 1 -- 172.21.15.59:0/3512553889 wait complete. 2024-04-02T17:02:52.927 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial admin user... 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$n5OFOyXLBJ.4crgeeaGQj.w0zPuoCQpfOqYT2XMynuI4V9EGW9kg6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712077374, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.502+0000 7f9453a00700 1 Processor -- start 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.502+0000 7f9453a00700 1 -- start start 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.503+0000 7f9453a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.503+0000 7f9453a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f944c108ad0 con 0x7f944c1080e0 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.503+0000 7f9451000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.503+0000 7f9451000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38086/0 (socket says 172.21.15.59:38086) 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.503+0000 7f9451000700 1 -- 172.21.15.59:0/107264706 learned_addr learned my addr 172.21.15.59:0/107264706 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:54.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.504+0000 7f9451000700 1 -- 172.21.15.59:0/107264706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1092e0 con 0x7f944c1080e0 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.504+0000 7f9451000700 1 --2- 172.21.15.59:0/107264706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94400098a0 tx=0x7f9440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ec9b0bbfbe6a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f944b400700 1 -- 172.21.15.59:0/107264706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9440004160 con 0x7f944c1080e0 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f944b400700 1 -- 172.21.15.59:0/107264706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9440004300 con 0x7f944c1080e0 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f944b400700 1 -- 172.21.15.59:0/107264706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9440011b20 con 0x7f944c1080e0 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f9453a00700 1 -- 172.21.15.59:0/107264706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 msgr2=0x7f944c108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f9453a00700 1 --2- 172.21.15.59:0/107264706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94400098a0 tx=0x7f9440009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f9453a00700 1 -- 172.21.15.59:0/107264706 shutdown_connections 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f9453a00700 1 --2- 172.21.15.59:0/107264706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c108500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.505+0000 7f9453a00700 1 -- 172.21.15.59:0/107264706 >> 172.21.15.59:0/107264706 conn(0x7f944c103680 msgr2=0x7f944c105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.506+0000 7f9453a00700 1 -- 172.21.15.59:0/107264706 shutdown_connections 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.506+0000 7f9453a00700 1 -- 172.21.15.59:0/107264706 wait complete. 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.507+0000 7f9453a00700 1 Processor -- start 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.507+0000 7f9453a00700 1 -- start start 2024-04-02T17:02:54.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.507+0000 7f9453a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.507+0000 7f9453a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f944c10c690 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.508+0000 7f9451000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.508+0000 7f9451000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38100/0 (socket says 172.21.15.59:38100) 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.508+0000 7f9451000700 1 -- 172.21.15.59:0/2011843458 learned_addr learned my addr 172.21.15.59:0/2011843458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.508+0000 7f9451000700 1 -- 172.21.15.59:0/2011843458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c19e870 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.508+0000 7f9451000700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f9440011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.509+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9440003b60 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.509+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944001a910 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.509+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9440024930 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.509+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c19eb00 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.509+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c19ef60 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.510+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9440022020 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.510+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c063c20 con 0x7f944c1080e0 2024-04-02T17:02:54.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.510+0000 7f9449600700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 0x7f943403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.510+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f944004daa0 con 0x7f944c1080e0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.511+0000 7f944be00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 0x7f943403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.511+0000 7f944be00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 0x7f943403bb70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9438006fd0 tx=0x7f9438006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.516+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f944000b3e0 con 0x7f944c1080e0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:53.775+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f944c1a1970 con 0x7f94340396b0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.041+0000 7f9449600700 1 -- 172.21.15.59:0/2011843458 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f944c1a1970 con 0x7f94340396b0 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 msgr2=0x7f943403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 0x7f943403bb70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9438006fd0 tx=0x7f9438006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 msgr2=0x7f944c19e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f9440011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 shutdown_connections 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f94340396b0 0x7f943403bb70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:54.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 --2- 172.21.15.59:0/2011843458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944c1080e0 0x7f944c19e330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:54.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.044+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 >> 172.21.15.59:0/2011843458 conn(0x7f944c103680 msgr2=0x7f944c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:54.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.045+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 shutdown_connections 2024-04-02T17:02:54.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:54.045+0000 7f9453a00700 1 -- 172.21.15.59:0/2011843458 wait complete. 2024-04-02T17:02:54.751 INFO:teuthology.orchestra.run.smithi059.stdout:Fetching dashboard port number... 2024-04-02T17:02:55.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:55 smithi059 ceph-mon[31664]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:02:55.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:02:55.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:02:56.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1175137162' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T17:02:57.006 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T17:02:57.006 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.354+0000 7fc0b1400700 1 Processor -- start 2024-04-02T17:02:57.006 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.354+0000 7fc0b1400700 1 -- start start 2024-04-02T17:02:57.006 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.355+0000 7fc0b1400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:57.007 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.355+0000 7fc0b1400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac07b780 con 0x7fc0ac07cd50 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.355+0000 7fc0aa800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.355+0000 7fc0aa800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38102/0 (socket says 172.21.15.59:38102) 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.355+0000 7fc0aa800700 1 -- 172.21.15.59:0/3272232684 learned_addr learned my addr 172.21.15.59:0/3272232684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.356+0000 7fc0aa800700 1 -- 172.21.15.59:0/3272232684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac07b900 con 0x7fc0ac07cd50 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.356+0000 7fc0aa800700 1 --2- 172.21.15.59:0/3272232684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098a0 tx=0x7fc094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a116ded544c66ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.356+0000 7fc0a9400700 1 -- 172.21.15.59:0/3272232684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc094004160 con 0x7fc0ac07cd50 2024-04-02T17:02:57.008 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.356+0000 7fc0a9400700 1 -- 172.21.15.59:0/3272232684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc094004300 con 0x7fc0ac07cd50 2024-04-02T17:02:57.018 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.356+0000 7fc0a9400700 1 -- 172.21.15.59:0/3272232684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc094011b20 con 0x7fc0ac07cd50 2024-04-02T17:02:57.018 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.357+0000 7fc0b1400700 1 -- 172.21.15.59:0/3272232684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 msgr2=0x7fc0ac07b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:57.018 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.357+0000 7fc0b1400700 1 --2- 172.21.15.59:0/3272232684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098a0 tx=0x7fc094009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.357+0000 7fc0b1400700 1 -- 172.21.15.59:0/3272232684 shutdown_connections 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.357+0000 7fc0b1400700 1 --2- 172.21.15.59:0/3272232684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac07b1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.357+0000 7fc0b1400700 1 -- 172.21.15.59:0/3272232684 >> 172.21.15.59:0/3272232684 conn(0x7fc0ac103640 msgr2=0x7fc0ac105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.358+0000 7fc0b1400700 1 -- 172.21.15.59:0/3272232684 shutdown_connections 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.358+0000 7fc0b1400700 1 -- 172.21.15.59:0/3272232684 wait complete. 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.359+0000 7fc0b1400700 1 Processor -- start 2024-04-02T17:02:57.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.359+0000 7fc0b1400700 1 -- start start 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.359+0000 7fc0b1400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.359+0000 7fc0b1400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac10e750 con 0x7fc0ac07cd50 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.360+0000 7fc0aa800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.360+0000 7fc0aa800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38104/0 (socket says 172.21.15.59:38104) 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.360+0000 7fc0aa800700 1 -- 172.21.15.59:0/1175137162 learned_addr learned my addr 172.21.15.59:0/1175137162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.360+0000 7fc0aa800700 1 -- 172.21.15.59:0/1175137162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac10b870 con 0x7fc0ac07cd50 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.361+0000 7fc0aa800700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.361+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc094003b60 con 0x7fc0ac07cd50 2024-04-02T17:02:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.361+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc094003d00 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.361+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0940248a0 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.361+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac10bb00 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.362+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac108040 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.363+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac10bcd0 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.364+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc094022020 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.364+0000 7fc0a3400700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 0x7fc09003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.364+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc09404d890 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.365+0000 7fc0a9e00700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 0x7fc09003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.366+0000 7fc0a9e00700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 0x7fc09003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.370+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc0ac10bcd0 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.598+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc0ac108330 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.599+0000 7fc0a3400700 1 -- 172.21.15.59:0/1175137162 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc09402c560 con 0x7fc0ac07cd50 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 msgr2=0x7fc09003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:57.021 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 0x7fc09003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 msgr2=0x7fc0ac10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 shutdown_connections 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc090039660 0x7fc09003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 --2- 172.21.15.59:0/1175137162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac07cd50 0x7fc0ac10b330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 >> 172.21.15.59:0/1175137162 conn(0x7fc0ac103640 msgr2=0x7fc0ac104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 shutdown_connections 2024-04-02T17:02:57.022 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:55.602+0000 7fc0b1400700 1 -- 172.21.15.59:0/1175137162 wait complete. 2024-04-02T17:02:57.046 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T17:02:57.046 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-04-02T17:02:57.073 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T17:02:57.073 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-04-02T17:02:57.074 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-04-02T17:02:57.074 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T17:02:57.082 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph Dashboard is now available at: 2024-04-02T17:02:57.082 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:02:57.083 INFO:teuthology.orchestra.run.smithi059.stdout: URL: https://smithi059.front.sepia.ceph.com:8443/ 2024-04-02T17:02:57.083 INFO:teuthology.orchestra.run.smithi059.stdout: User: admin 2024-04-02T17:02:57.083 INFO:teuthology.orchestra.run.smithi059.stdout: Password: qudeel5hkd 2024-04-02T17:02:57.083 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:02:57.084 INFO:teuthology.orchestra.run.smithi059.stdout:Saving cluster configuration to /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config directory 2024-04-02T17:02:57.086 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling autotune for osd_memory_target 2024-04-02T17:02:57.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:02:57 smithi059 ceph-mon[31664]: mgrmap e13: x(active, since 6s) 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.658+0000 7f22cbe00700 1 Processor -- start 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.658+0000 7f22cbe00700 1 -- start start 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.659+0000 7f22cbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.659+0000 7f22cbe00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc1089f0 con 0x7f22cc108090 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.659+0000 7f22caa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.659+0000 7f22caa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38114/0 (socket says 172.21.15.59:38114) 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.659+0000 7f22caa00700 1 -- 172.21.15.59:0/512448026 learned_addr learned my addr 172.21.15.59:0/512448026 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.660+0000 7f22caa00700 1 -- 172.21.15.59:0/512448026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc108b70 con 0x7f22cc108090 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.660+0000 7f22caa00700 1 --2- 172.21.15.59:0/512448026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0098a0 tx=0x7f22bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11a39a81d196c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.661+0000 7f22c9600700 1 -- 172.21.15.59:0/512448026 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22bc004160 con 0x7f22cc108090 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.661+0000 7f22c9600700 1 -- 172.21.15.59:0/512448026 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22bc004300 con 0x7f22cc108090 2024-04-02T17:02:58.465 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.661+0000 7f22c9600700 1 -- 172.21.15.59:0/512448026 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22bc011b20 con 0x7f22cc108090 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.661+0000 7f22cbe00700 1 -- 172.21.15.59:0/512448026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 msgr2=0x7f22cc1084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.661+0000 7f22cbe00700 1 --2- 172.21.15.59:0/512448026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0098a0 tx=0x7f22bc009bb0 comp rx=0 tx=0).stop 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.662+0000 7f22cbe00700 1 -- 172.21.15.59:0/512448026 shutdown_connections 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.662+0000 7f22cbe00700 1 --2- 172.21.15.59:0/512448026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc1084b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.662+0000 7f22cbe00700 1 -- 172.21.15.59:0/512448026 >> 172.21.15.59:0/512448026 conn(0x7f22cc1036f0 msgr2=0x7f22cc105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.662+0000 7f22cbe00700 1 -- 172.21.15.59:0/512448026 shutdown_connections 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.662+0000 7f22cbe00700 1 -- 172.21.15.59:0/512448026 wait complete. 2024-04-02T17:02:58.466 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.663+0000 7f22cbe00700 1 Processor -- start 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.663+0000 7f22cbe00700 1 -- start start 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.664+0000 7f22cbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.664+0000 7f22cbe00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc10c500 con 0x7f22cc108090 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.664+0000 7f22caa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.664+0000 7f22caa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38116/0 (socket says 172.21.15.59:38116) 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.664+0000 7f22caa00700 1 -- 172.21.15.59:0/137020184 learned_addr learned my addr 172.21.15.59:0/137020184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22caa00700 1 -- 172.21.15.59:0/137020184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc07e030 con 0x7f22cc108090 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22caa00700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0117e0 tx=0x7f22bc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:58.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22bc003bf0 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22bc01a910 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc07c220 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22bc024970 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.665+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc07c6e0 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.667+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f22bc022020 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.667+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc063c20 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.667+0000 7f22c3400700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 0x7f22b003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.667+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f22bc04dd00 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.667+0000 7f22ca000700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 0x7f22b003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.668+0000 7f22ca000700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 0x7f22b003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.673+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f22bc011e20 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.900+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f22cc07c960 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.902+0000 7f22c3400700 1 -- 172.21.15.59:0/137020184 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f22bc00b3e0 con 0x7f22cc108090 2024-04-02T17:02:58.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.904+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 msgr2=0x7f22b003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.904+0000 7f22cbe00700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 0x7f22b003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8006e40 comp rx=0 tx=0).stop 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.904+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 msgr2=0x7f22cc07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.904+0000 7f22cbe00700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0117e0 tx=0x7f22bc011c10 comp rx=0 tx=0).stop 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 shutdown_connections 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22b0039650 0x7f22b003bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 --2- 172.21.15.59:0/137020184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc108090 0x7f22cc07daf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 >> 172.21.15.59:0/137020184 conn(0x7f22cc1036f0 msgr2=0x7f22cc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 shutdown_connections 2024-04-02T17:02:58.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:57.905+0000 7f22cbe00700 1 -- 172.21.15.59:0/137020184 wait complete. 2024-04-02T17:03:00.068 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.041+0000 7fe5c5000700 1 Processor -- start 2024-04-02T17:03:00.068 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.041+0000 7fe5c5000700 1 -- start start 2024-04-02T17:03:00.068 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.042+0000 7fe5c5000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:00.068 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.042+0000 7fe5c5000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0108a40 con 0x7fe5c0108050 2024-04-02T17:03:00.069 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.042+0000 7fe5bf400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:00.069 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.042+0000 7fe5bf400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38132/0 (socket says 172.21.15.59:38132) 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.042+0000 7fe5bf400700 1 -- 172.21.15.59:0/3688640112 learned_addr learned my addr 172.21.15.59:0/3688640112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.043+0000 7fe5bf400700 1 -- 172.21.15.59:0/3688640112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c0109250 con 0x7fe5c0108050 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.043+0000 7fe5bf400700 1 --2- 172.21.15.59:0/3688640112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098a0 tx=0x7fe5b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a9dc6ce2e93e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.043+0000 7fe5be000700 1 -- 172.21.15.59:0/3688640112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b0004160 con 0x7fe5c0108050 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.043+0000 7fe5be000700 1 -- 172.21.15.59:0/3688640112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0004300 con 0x7fe5c0108050 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.043+0000 7fe5be000700 1 -- 172.21.15.59:0/3688640112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b0011b20 con 0x7fe5c0108050 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 -- 172.21.15.59:0/3688640112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 msgr2=0x7fe5c0108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:00.070 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3688640112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098a0 tx=0x7fe5b0009bb0 comp rx=0 tx=0).stop 2024-04-02T17:03:00.074 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 -- 172.21.15.59:0/3688640112 shutdown_connections 2024-04-02T17:03:00.074 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3688640112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c0108470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:00.074 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 -- 172.21.15.59:0/3688640112 >> 172.21.15.59:0/3688640112 conn(0x7fe5c01036f0 msgr2=0x7fe5c0105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 -- 172.21.15.59:0/3688640112 shutdown_connections 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.044+0000 7fe5c5000700 1 -- 172.21.15.59:0/3688640112 wait complete. 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.045+0000 7fe5c5000700 1 Processor -- start 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5c5000700 1 -- start start 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5c5000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5c5000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c010c600 con 0x7fe5c0108050 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5bf400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5bf400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38142/0 (socket says 172.21.15.59:38142) 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.046+0000 7fe5bf400700 1 -- 172.21.15.59:0/3325558398 learned_addr learned my addr 172.21.15.59:0/3325558398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5bf400700 1 -- 172.21.15.59:0/3325558398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c019c540 con 0x7fe5c0108050 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5bf400700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00117e0 tx=0x7fe5b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b0003bf0 con 0x7fe5c0108050 2024-04-02T17:03:00.075 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b001a910 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5b0024970 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c019c7d0 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.047+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c019cc30 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.048+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c0063c20 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.049+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe5b0022020 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.049+0000 7fe5b7e00700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 0x7fe5a403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.049+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe5b004dda0 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.049+0000 7fe5bea00700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 0x7fe5a403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.050+0000 7fe5bea00700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 0x7fe5a403bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac006fd0 tx=0x7fe5ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.054+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe5b00212c0 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.403+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe5c019f890 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.407+0000 7fe5b7e00700 1 -- 172.21.15.59:0/3325558398 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fe5b001ab70 con 0x7fe5c0108050 2024-04-02T17:03:00.076 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.410+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 msgr2=0x7fe5a403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:00.077 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.410+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 0x7fe5a403bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac006fd0 tx=0x7fe5ac006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:00.077 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.410+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 msgr2=0x7fe5c019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:00.077 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.410+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00117e0 tx=0x7fe5b0011ba0 comp rx=0 tx=0).stop 2024-04-02T17:03:00.077 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 shutdown_connections 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe5a40396a0 0x7fe5a403bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 --2- 172.21.15.59:0/3325558398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0108050 0x7fe5c019c000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 >> 172.21.15.59:0/3325558398 conn(0x7fe5c01036f0 msgr2=0x7fe5c0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 shutdown_connections 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-04-02T17:02:59.411+0000 7fe5c5000700 1 -- 172.21.15.59:0/3325558398 wait complete. 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T17:03:00.078 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: ceph telemetry on 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout:For more information see: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:00.079 INFO:teuthology.orchestra.run.smithi059.stdout:Bootstrap complete. 2024-04-02T17:03:00.203 INFO:tasks.cephadm:Fetching config... 2024-04-02T17:03:00.203 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:03:00.203 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T17:03:00.253 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T17:03:00.253 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:03:00.253 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T17:03:00.336 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T17:03:00.337 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:03:00.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-02T17:03:00.468 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T17:03:00.469 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:03:00.469 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T17:03:00.526 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T17:03:00.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T17:03:00.647 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:00 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3325558398' entity='client.admin' 2024-04-02T17:03:00.650 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:00.719 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T17:03:00.768 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:00.783 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T17:03:00.831 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC558KmeaUUj9PyswXadHKWGE7kUhfBKWUaf9U3OSTPpyxW/S0S+E5Ar1ZOkGzeG17UyeZZfYm2ogAvxa7/5JtaF65NC+Pyp6P3dqSi+L75dEi3wKrS+bfLE6WpYP9aUepXAY1h4mgb1t0k4h0sGyk+uL6YZCsPhHSJuE1N1QvrTH7mze26jg0Pi1MnBLoL9BD7OK5NBWFMHSFc9ZYT2RkKYPHmMINrnxN9usX1mbBnlJe7iZ+xp4peaRkHVQWIkHeG92ZLT1/dp5qlS0BFKRuG0vINnqO4FXQd+cE4kWuu0PAcbsk0GZ4Z7NMs9jGKh1S5pIZ31YQcLOLca/kFTgQjEoXV5Vnta5qtCZCGojgbcR/nkDoba5fFx8RnkMA7mmKv/oOHWPyz+LRt2qZQYyeIemxoqGvksV4sSBnzz1WOdoFdsPW750dr+Z4tFOLQ7YnzZ47blkSvCGwG8peVWjW7E1tGpxnkQTgZozbGJY/678eoAfr5b7H8lWkVUNpJDAk= ceph-8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:00.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T17:03:01.272 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 -- 172.21.15.59:0/2310176551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 msgr2=0x7ff990104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 --2- 172.21.15.59:0/2310176551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff990104490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff984009910 tx=0x7ff984009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 -- 172.21.15.59:0/2310176551 shutdown_connections 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 --2- 172.21.15.59:0/2310176551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff990104490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 -- 172.21.15.59:0/2310176551 >> 172.21.15.59:0/2310176551 conn(0x7ff9900ff630 msgr2=0x7ff990101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 -- 172.21.15.59:0/2310176551 shutdown_connections 2024-04-02T17:03:03.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.471+0000 7ff996400700 1 -- 172.21.15.59:0/2310176551 wait complete. 2024-04-02T17:03:03.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.472+0000 7ff996400700 1 Processor -- start 2024-04-02T17:03:03.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.472+0000 7ff996400700 1 -- start start 2024-04-02T17:03:03.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff996400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:03.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff996400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010a100 con 0x7ff990104070 2024-04-02T17:03:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff995000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff995000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47652/0 (socket says 172.21.15.59:47652) 2024-04-02T17:03:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff995000700 1 -- 172.21.15.59:0/3413881360 learned_addr learned my addr 172.21.15.59:0/3413881360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.473+0000 7ff995000700 1 -- 172.21.15.59:0/3413881360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99019e090 con 0x7ff990104070 2024-04-02T17:03:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff995000700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff984005850 tx=0x7ff984005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:03.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff984026070 con 0x7ff990104070 2024-04-02T17:03:03.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9840044a0 con 0x7ff990104070 2024-04-02T17:03:03.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99019e320 con 0x7ff990104070 2024-04-02T17:03:03.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff984003ab0 con 0x7ff990104070 2024-04-02T17:03:03.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.474+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a1010 con 0x7ff990104070 2024-04-02T17:03:03.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.475+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99010a880 con 0x7ff990104070 2024-04-02T17:03:03.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.475+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff984029070 con 0x7ff990104070 2024-04-02T17:03:03.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.476+0000 7ff98d600700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 0x7ff97803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.476+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff98401e070 con 0x7ff990104070 2024-04-02T17:03:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.476+0000 7ff98fe00700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 0x7ff97803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.477+0000 7ff98fe00700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 0x7ff97803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:03.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.482+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff984013070 con 0x7ff990104070 2024-04-02T17:03:03.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.710+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff99002de40 con 0x7ff990104070 2024-04-02T17:03:03.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.715+0000 7ff98d600700 1 -- 172.21.15.59:0/3413881360 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff9840300c0 con 0x7ff990104070 2024-04-02T17:03:03.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 msgr2=0x7ff97803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:03.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 0x7ff97803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:03.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 msgr2=0x7ff99019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:03.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff984005850 tx=0x7ff984005030 comp rx=0 tx=0).stop 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 shutdown_connections 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff978039650 0x7ff97803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 --2- 172.21.15.59:0/3413881360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff990104070 0x7ff99019db50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.719+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 >> 172.21.15.59:0/3413881360 conn(0x7ff9900ff630 msgr2=0x7ff9901000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.720+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 shutdown_connections 2024-04-02T17:03:03.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:03.720+0000 7ff996400700 1 -- 172.21.15.59:0/3413881360 wait complete. 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3413881360' entity='client.admin' 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:04.420 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T17:03:04.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T17:03:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 -- 172.21.15.59:0/872701740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 msgr2=0x7fa200102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 --2- 172.21.15.59:0/872701740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200102460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009910 tx=0x7fa1f4009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 -- 172.21.15.59:0/872701740 shutdown_connections 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 --2- 172.21.15.59:0/872701740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200102460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 -- 172.21.15.59:0/872701740 >> 172.21.15.59:0/872701740 conn(0x7fa2000fd5e0 msgr2=0x7fa2000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.788+0000 7fa208200700 1 -- 172.21.15.59:0/872701740 shutdown_connections 2024-04-02T17:03:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.789+0000 7fa208200700 1 -- 172.21.15.59:0/872701740 wait complete. 2024-04-02T17:03:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.790+0000 7fa208200700 1 Processor -- start 2024-04-02T17:03:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.790+0000 7fa208200700 1 -- start start 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa208200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa208200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2001080d0 con 0x7fa200102040 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa205800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa205800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47662/0 (socket says 172.21.15.59:47662) 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa205800700 1 -- 172.21.15.59:0/2455924289 learned_addr learned my addr 172.21.15.59:0/2455924289 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa205800700 1 -- 172.21.15.59:0/2455924289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200197c60 con 0x7fa200102040 2024-04-02T17:03:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.791+0000 7fa205800700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005850 tx=0x7fa1f4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.792+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f4026070 con 0x7fa200102040 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.792+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f40044a0 con 0x7fa200102040 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.792+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f4003ab0 con 0x7fa200102040 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.792+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa200197ef0 con 0x7fa200102040 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.792+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20019abe0 con 0x7fa200102040 2024-04-02T17:03:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.793+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa1f4029070 con 0x7fa200102040 2024-04-02T17:03:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.793+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200108850 con 0x7fa200102040 2024-04-02T17:03:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.794+0000 7fa1fe000700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 0x7fa1e803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.794+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1f401e070 con 0x7fa200102040 2024-04-02T17:03:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.794+0000 7fa204e00700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 0x7fa1e803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.795+0000 7fa204e00700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 0x7fa1e803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:06.799+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1f4013070 con 0x7fa200102040 2024-04-02T17:03:07.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.036+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa20002de40 con 0x7fa1e80396b0 2024-04-02T17:03:07.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.044+0000 7fa1fe000700 1 -- 172.21.15.59:0/2455924289 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa20002de40 con 0x7fa1e80396b0 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 msgr2=0x7fa1e803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 0x7fa1e803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006fd0 tx=0x7fa1ec006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 msgr2=0x7fa200197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005850 tx=0x7fa1f4005030 comp rx=0 tx=0).stop 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 shutdown_connections 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1e80396b0 0x7fa1e803bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 --2- 172.21.15.59:0/2455924289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa200102040 0x7fa200197720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 >> 172.21.15.59:0/2455924289 conn(0x7fa2000fd5e0 msgr2=0x7fa2000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 shutdown_connections 2024-04-02T17:03:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:07.047+0000 7fa208200700 1 -- 172.21.15.59:0/2455924289 wait complete. 2024-04-02T17:03:07.790 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-04-02T17:03:07.791 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:03:07.791 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-04-02T17:03:07.808 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:03:07.808 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:07.865 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-04-02T17:03:07.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi154 2024-04-02T17:03:08.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:03:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:08 smithi059 ceph-mon[31664]: Updating smithi059:/etc/ceph/ceph.conf 2024-04-02T17:03:08.309 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:09 smithi059 ceph-mon[31664]: Updating smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:10.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 -- 172.21.15.59:0/820272400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 msgr2=0x7f97dc074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:10.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 --2- 172.21.15.59:0/820272400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc074e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009910 tx=0x7f97cc009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:10.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 -- 172.21.15.59:0/820272400 shutdown_connections 2024-04-02T17:03:10.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 --2- 172.21.15.59:0/820272400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc074e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:10.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 -- 172.21.15.59:0/820272400 >> 172.21.15.59:0/820272400 conn(0x7f97dc0fdb00 msgr2=0x7f97dc0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:10.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 -- 172.21.15.59:0/820272400 shutdown_connections 2024-04-02T17:03:10.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.099+0000 7f97e3800700 1 -- 172.21.15.59:0/820272400 wait complete. 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.100+0000 7f97e3800700 1 Processor -- start 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.100+0000 7f97e3800700 1 -- start start 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.101+0000 7f97e3800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.101+0000 7f97e3800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc10e880 con 0x7f97dc074a20 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.101+0000 7f97e0e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.101+0000 7f97e0e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47668/0 (socket says 172.21.15.59:47668) 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.101+0000 7f97e0e00700 1 -- 172.21.15.59:0/2864931859 learned_addr learned my addr 172.21.15.59:0/2864931859 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:10.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97e0e00700 1 -- 172.21.15.59:0/2864931859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc19a280 con 0x7f97dc074a20 2024-04-02T17:03:10.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97e0e00700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97cc006010 tx=0x7f97cc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:10.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc026070 con 0x7f97dc074a20 2024-04-02T17:03:10.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc00bc30 con 0x7f97dc074a20 2024-04-02T17:03:10.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc19a510 con 0x7f97dc074a20 2024-04-02T17:03:10.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.102+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc0039a0 con 0x7f97dc074a20 2024-04-02T17:03:10.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.103+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc19a970 con 0x7f97dc074a20 2024-04-02T17:03:10.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.104+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc0760d0 con 0x7f97dc074a20 2024-04-02T17:03:10.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.104+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f97cc029070 con 0x7f97dc074a20 2024-04-02T17:03:10.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.104+0000 7f97da000700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 0x7f97c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:10.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.105+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f97cc01e070 con 0x7f97dc074a20 2024-04-02T17:03:10.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.105+0000 7f97d3e00700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 0x7f97c003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:10.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.106+0000 7f97d3e00700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 0x7f97c003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:10.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.110+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f97cc013070 con 0x7f97dc074a20 2024-04-02T17:03:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:10 smithi059 ceph-mon[31664]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:10 smithi059 ceph-mon[31664]: Updating smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T17:03:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:10.350+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}) -- 0x7f97dc19b110 con 0x7f97c0039650 2024-04-02T17:03:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:11 smithi059 ceph-mon[31664]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:12 smithi059 ceph-mon[31664]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:03:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:12 smithi059 ceph-mon[31664]: Deploying cephadm binary to smithi154 2024-04-02T17:03:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:13 smithi059 ceph-mon[31664]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.353+0000 7f97da000700 1 -- 172.21.15.59:0/2864931859 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f97dc19b110 con 0x7f97c0039650 2024-04-02T17:03:15.354 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-04-02T17:03:15.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:15 smithi059 ceph-mon[31664]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.356+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 msgr2=0x7f97c003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.356+0000 7f97e3800700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 0x7f97c003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 msgr2=0x7f97dc199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97cc006010 tx=0x7f97cc005030 comp rx=0 tx=0).stop 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 shutdown_connections 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97c0039650 0x7f97c003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 --2- 172.21.15.59:0/2864931859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97dc074a20 0x7f97dc199d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 >> 172.21.15.59:0/2864931859 conn(0x7f97dc0fdb00 msgr2=0x7f97dc106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 shutdown_connections 2024-04-02T17:03:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:15.357+0000 7f97e3800700 1 -- 172.21.15.59:0/2864931859 wait complete. 2024-04-02T17:03:16.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T17:03:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:16 smithi059 ceph-mon[31664]: Added host smithi154 2024-04-02T17:03:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:16 smithi059 ceph-mon[31664]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:16.696 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:17.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 -- 172.21.15.59:0/3506181780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 msgr2=0x7ff60810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 --2- 172.21.15.59:0/3506181780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60810ea60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009910 tx=0x7ff5f8009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 -- 172.21.15.59:0/3506181780 shutdown_connections 2024-04-02T17:03:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 --2- 172.21.15.59:0/3506181780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60810ea60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:18.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 -- 172.21.15.59:0/3506181780 >> 172.21.15.59:0/3506181780 conn(0x7ff608077d90 msgr2=0x7ff60807a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:18.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.534+0000 7ff60e200700 1 -- 172.21.15.59:0/3506181780 shutdown_connections 2024-04-02T17:03:18.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.535+0000 7ff60e200700 1 -- 172.21.15.59:0/3506181780 wait complete. 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.535+0000 7ff60e200700 1 Processor -- start 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff60e200700 1 -- start start 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff60e200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff60e200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff608112c30 con 0x7ff60810e680 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff607400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff607400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51474/0 (socket says 172.21.15.59:51474) 2024-04-02T17:03:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.536+0000 7ff607400700 1 -- 172.21.15.59:0/723145259 learned_addr learned my addr 172.21.15.59:0/723145259 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff607400700 1 -- 172.21.15.59:0/723145259 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60819e5b0 con 0x7ff60810e680 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff607400700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8004f80 tx=0x7ff5f8005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f8026070 con 0x7ff60810e680 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f800b6f0 con 0x7ff60810e680 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f8003910 con 0x7ff60810e680 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.537+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60819e8a0 con 0x7ff60810e680 2024-04-02T17:03:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.538+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a2940 con 0x7ff60810e680 2024-04-02T17:03:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.539+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff5f8029070 con 0x7ff60810e680 2024-04-02T17:03:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.539+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608109d30 con 0x7ff60810e680 2024-04-02T17:03:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.539+0000 7ff5ffe00700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 0x7ff5ec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.539+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5f801e070 con 0x7ff60810e680 2024-04-02T17:03:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.539+0000 7ff606a00700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 0x7ff5ec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.540+0000 7ff606a00700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 0x7ff5ec03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006fd0 tx=0x7ff5f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:18.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.545+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff5f8013070 con 0x7ff60810e680 2024-04-02T17:03:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.782+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff60819f3d0 con 0x7ff5ec0396b0 2024-04-02T17:03:18.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.785+0000 7ff5ffe00700 1 -- 172.21.15.59:0/723145259 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff60819f3d0 con 0x7ff5ec0396b0 2024-04-02T17:03:18.786 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:18.786 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.787+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 msgr2=0x7ff5ec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.787+0000 7ff60e200700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 0x7ff5ec03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006fd0 tx=0x7ff5f4006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 msgr2=0x7ff60819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8004f80 tx=0x7ff5f8005060 comp rx=0 tx=0).stop 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 shutdown_connections 2024-04-02T17:03:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5ec0396b0 0x7ff5ec03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 --2- 172.21.15.59:0/723145259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff60810e680 0x7ff60819e070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 >> 172.21.15.59:0/723145259 conn(0x7ff608077d90 msgr2=0x7ff608079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 shutdown_connections 2024-04-02T17:03:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:18.788+0000 7ff60e200700 1 -- 172.21.15.59:0/723145259 wait complete. 2024-04-02T17:03:19.432 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:19 smithi059 ceph-mon[31664]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:19.432 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:19.493 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-04-02T17:03:19.493 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:03:19.493 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-04-02T17:03:19.515 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:03:19.515 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:19.573 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-04-02T17:03:19.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi164 2024-04-02T17:03:20.029 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:20.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:20 smithi059 ceph-mon[31664]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:03:21.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:21.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:21.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.830+0000 7fd44aa00700 1 -- 172.21.15.59:0/4275165605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 msgr2=0x7fd444102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.830+0000 7fd44aa00700 1 --2- 172.21.15.59:0/4275165605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd444102600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd434009910 tx=0x7fd434009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.831+0000 7fd44aa00700 1 -- 172.21.15.59:0/4275165605 shutdown_connections 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.831+0000 7fd44aa00700 1 --2- 172.21.15.59:0/4275165605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd444102600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.831+0000 7fd44aa00700 1 -- 172.21.15.59:0/4275165605 >> 172.21.15.59:0/4275165605 conn(0x7fd4440fdad0 msgr2=0x7fd4440ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.831+0000 7fd44aa00700 1 -- 172.21.15.59:0/4275165605 shutdown_connections 2024-04-02T17:03:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.831+0000 7fd44aa00700 1 -- 172.21.15.59:0/4275165605 wait complete. 2024-04-02T17:03:21.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.832+0000 7fd44aa00700 1 Processor -- start 2024-04-02T17:03:21.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd44aa00700 1 -- start start 2024-04-02T17:03:21.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd44aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:21.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd44aa00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd44410c5f0 con 0x7fd444102220 2024-04-02T17:03:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd443e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd443e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59016/0 (socket says 172.21.15.59:59016) 2024-04-02T17:03:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.833+0000 7fd443e00700 1 -- 172.21.15.59:0/2392107602 learned_addr learned my addr 172.21.15.59:0/2392107602 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd443e00700 1 -- 172.21.15.59:0/2392107602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44419dcb0 con 0x7fd444102220 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd443e00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd434006010 tx=0x7fd434005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd434026070 con 0x7fd444102220 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43400bbd0 con 0x7fd444102220 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd434003910 con 0x7fd444102220 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.834+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44419dfa0 con 0x7fd444102220 2024-04-02T17:03:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.835+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd444197930 con 0x7fd444102220 2024-04-02T17:03:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.836+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd434029070 con 0x7fd444102220 2024-04-02T17:03:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.836+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd444103700 con 0x7fd444102220 2024-04-02T17:03:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.836+0000 7fd440c00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 0x7fd42803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.836+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd43401e070 con 0x7fd444102220 2024-04-02T17:03:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.836+0000 7fd443400700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 0x7fd42803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.837+0000 7fd443400700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 0x7fd42803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:21.845+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd434013070 con 0x7fd444102220 2024-04-02T17:03:22.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:22.089+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7fd444198ed0 con 0x7fd4280396b0 2024-04-02T17:03:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:22 smithi059 ceph-mon[31664]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: Updating smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: Updating smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:24.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:24 smithi059 ceph-mon[31664]: Deploying cephadm binary to smithi164 2024-04-02T17:03:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:25 smithi059 ceph-mon[31664]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.092+0000 7fd440c00700 1 -- 172.21.15.59:0/2392107602 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd444198ed0 con 0x7fd4280396b0 2024-04-02T17:03:27.095 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-04-02T17:03:27.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 msgr2=0x7fd42803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:27.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 0x7fd42803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 msgr2=0x7fd44419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd434006010 tx=0x7fd434005030 comp rx=0 tx=0).stop 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 shutdown_connections 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4280396b0 0x7fd42803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.097+0000 7fd44aa00700 1 --2- 172.21.15.59:0/2392107602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd444102220 0x7fd44419d770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.098+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 >> 172.21.15.59:0/2392107602 conn(0x7fd4440fdad0 msgr2=0x7fd444103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.098+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 shutdown_connections 2024-04-02T17:03:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:27.098+0000 7fd44aa00700 1 -- 172.21.15.59:0/2392107602 wait complete. 2024-04-02T17:03:27.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:27 smithi059 ceph-mon[31664]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:27.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:27.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T17:03:28.327 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:28.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:28 smithi059 ceph-mon[31664]: Added host smithi164 2024-04-02T17:03:28.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:28.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:29.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:29 smithi059 ceph-mon[31664]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:29.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:30.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 -- 172.21.15.59:0/2992161920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 msgr2=0x7f487c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:30.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 --2- 172.21.15.59:0/2992161920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c103560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f486c009910 tx=0x7f486c009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:30.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 -- 172.21.15.59:0/2992161920 shutdown_connections 2024-04-02T17:03:30.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 --2- 172.21.15.59:0/2992161920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c103560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:30.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 -- 172.21.15.59:0/2992161920 >> 172.21.15.59:0/2992161920 conn(0x7f487c077b30 msgr2=0x7f487c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:30.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 -- 172.21.15.59:0/2992161920 shutdown_connections 2024-04-02T17:03:30.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.176+0000 7f4881c00700 1 -- 172.21.15.59:0/2992161920 wait complete. 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.177+0000 7f4881c00700 1 Processor -- start 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f4881c00700 1 -- start start 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f4881c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f4881c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f487c10c670 con 0x7f487c103180 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f487aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f487aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59032/0 (socket says 172.21.15.59:59032) 2024-04-02T17:03:30.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.178+0000 7f487aa00700 1 -- 172.21.15.59:0/36042348 learned_addr learned my addr 172.21.15.59:0/36042348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:30.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.179+0000 7f487aa00700 1 -- 172.21.15.59:0/36042348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c19a2b0 con 0x7f487c103180 2024-04-02T17:03:30.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.179+0000 7f487aa00700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486c006010 tx=0x7f486c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:30.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.179+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f486c026070 con 0x7f487c103180 2024-04-02T17:03:30.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.179+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486c00bc30 con 0x7f487c103180 2024-04-02T17:03:30.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.180+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f486c0039a0 con 0x7f487c103180 2024-04-02T17:03:30.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.180+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c19a540 con 0x7f487c103180 2024-04-02T17:03:30.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.180+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c19a9a0 con 0x7f487c103180 2024-04-02T17:03:30.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.181+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f486c029070 con 0x7f487c103180 2024-04-02T17:03:30.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.182+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c06a2c0 con 0x7f487c103180 2024-04-02T17:03:30.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.182+0000 7f4873400700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 0x7f486003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:30.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.182+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f486c01e070 con 0x7f487c103180 2024-04-02T17:03:30.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.182+0000 7f487a000700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 0x7f486003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:30.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.183+0000 7f487a000700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 0x7f486003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4868006fd0 tx=0x7f4868006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:30.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.188+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f486c013070 con 0x7f487c103180 2024-04-02T17:03:30.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.426+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f487c19b0d0 con 0x7f4860039700 2024-04-02T17:03:30.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.428+0000 7f4873400700 1 -- 172.21.15.59:0/36042348 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f487c19b0d0 con 0x7f4860039700 2024-04-02T17:03:30.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:03:30.428 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-04-02T17:03:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 msgr2=0x7f486003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 0x7f486003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4868006fd0 tx=0x7f4868006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 msgr2=0x7f487c199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486c006010 tx=0x7f486c005030 comp rx=0 tx=0).stop 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 shutdown_connections 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4860039700 0x7f486003bbc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 --2- 172.21.15.59:0/36042348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f487c103180 0x7f487c199d70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 >> 172.21.15.59:0/36042348 conn(0x7f487c077b30 msgr2=0x7f487c108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 shutdown_connections 2024-04-02T17:03:30.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:30.431+0000 7f4881c00700 1 -- 172.21.15.59:0/36042348 wait complete. 2024-04-02T17:03:30.971 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:30 smithi059 ceph-mon[31664]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:30.972 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:31.039 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T17:03:31.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T17:03:31.457 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:31 smithi059 ceph-mon[31664]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:03:33.191 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:32 smithi059 ceph-mon[31664]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:33.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 -- 172.21.15.59:0/2655286240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 msgr2=0x7efe4810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 --2- 172.21.15.59:0/2655286240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4810ea60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efe38009910 tx=0x7efe38009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 -- 172.21.15.59:0/2655286240 shutdown_connections 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 --2- 172.21.15.59:0/2655286240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4810ea60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 -- 172.21.15.59:0/2655286240 >> 172.21.15.59:0/2655286240 conn(0x7efe48077d90 msgr2=0x7efe4807a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 -- 172.21.15.59:0/2655286240 shutdown_connections 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.363+0000 7efe4dc00700 1 -- 172.21.15.59:0/2655286240 wait complete. 2024-04-02T17:03:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.365+0000 7efe4dc00700 1 Processor -- start 2024-04-02T17:03:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.365+0000 7efe4dc00700 1 -- start start 2024-04-02T17:03:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.365+0000 7efe4dc00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.365+0000 7efe4dc00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe48112c30 con 0x7efe4810e680 2024-04-02T17:03:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.366+0000 7efe46a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.366+0000 7efe46a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51308/0 (socket says 172.21.15.59:51308) 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.366+0000 7efe46a00700 1 -- 172.21.15.59:0/3497010400 learned_addr learned my addr 172.21.15.59:0/3497010400 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.366+0000 7efe46a00700 1 -- 172.21.15.59:0/3497010400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4819e590 con 0x7efe4810e680 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.366+0000 7efe46a00700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe380092e0 tx=0x7efe38005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.367+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe38026070 con 0x7efe4810e680 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.367+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3800b6f0 con 0x7efe4810e680 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.367+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe380038e0 con 0x7efe4810e680 2024-04-02T17:03:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.367+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4819e880 con 0x7efe4810e680 2024-04-02T17:03:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.367+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481a2920 con 0x7efe4810e680 2024-04-02T17:03:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.368+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efe38029070 con 0x7efe4810e680 2024-04-02T17:03:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.368+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe48109d30 con 0x7efe4810e680 2024-04-02T17:03:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.369+0000 7efe3f400700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 0x7efe2c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.369+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe3801e070 con 0x7efe4810e680 2024-04-02T17:03:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.369+0000 7efe46000700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 0x7efe2c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.370+0000 7efe46000700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 0x7efe2c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe34006fd0 tx=0x7efe34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.374+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efe38013070 con 0x7efe4810e680 2024-04-02T17:03:33.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.612+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7efe4819f3b0 con 0x7efe4810e680 2024-04-02T17:03:33.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.956+0000 7efe3f400700 1 -- 172.21.15.59:0/3497010400 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7efe3802bd20 con 0x7efe4810e680 2024-04-02T17:03:33.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.960+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 msgr2=0x7efe2c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:33.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.960+0000 7efe4dc00700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 0x7efe2c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe34006fd0 tx=0x7efe34006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.960+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 msgr2=0x7efe4819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.960+0000 7efe4dc00700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe380092e0 tx=0x7efe38005030 comp rx=0 tx=0).stop 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 shutdown_connections 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe2c0396b0 0x7efe2c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 --2- 172.21.15.59:0/3497010400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe4810e680 0x7efe4819e050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 >> 172.21.15.59:0/3497010400 conn(0x7efe48077d90 msgr2=0x7efe48078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 shutdown_connections 2024-04-02T17:03:33.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:33.961+0000 7efe4dc00700 1 -- 172.21.15.59:0/3497010400 wait complete. 2024-04-02T17:03:33.963 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-02T17:03:34.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:33 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3497010400' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T17:03:34.690 INFO:tasks.cephadm:Adding mon.a on smithi059 2024-04-02T17:03:34.690 INFO:tasks.cephadm:Adding mon.b on smithi154 2024-04-02T17:03:34.690 INFO:tasks.cephadm:Adding mon.c on smithi164 2024-04-02T17:03:34.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi059:172.21.15.59=a;smithi154:172.21.15.154=b;smithi164:172.21.15.164=c' 2024-04-02T17:03:34.917 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:35.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: Updating smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:35.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:35.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:03:35.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: Updating smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T17:03:35.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3497010400' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T17:03:35.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:35.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:35.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:35.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:36.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- 172.21.15.164:0/735834405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 msgr2=0x7f3a18102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:36.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 --2- 172.21.15.164:0/735834405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a18102600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009910 tx=0x7f3a08009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:36.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- 172.21.15.164:0/735834405 shutdown_connections 2024-04-02T17:03:36.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 --2- 172.21.15.164:0/735834405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a18102600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:36.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- 172.21.15.164:0/735834405 >> 172.21.15.164:0/735834405 conn(0x7f3a180fdad0 msgr2=0x7f3a180ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- 172.21.15.164:0/735834405 shutdown_connections 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- 172.21.15.164:0/735834405 wait complete. 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 Processor -- start 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.361+0000 7f3a204a5700 1 -- start start 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.362+0000 7f3a204a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.362+0000 7f3a204a5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810c5f0 con 0x7f3a18102220 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.362+0000 7f3a1e241700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.362+0000 7f3a1e241700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.164:46742/0 (socket says 172.21.15.164:46742) 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.362+0000 7f3a1e241700 1 -- 172.21.15.164:0/4019590470 learned_addr learned my addr 172.21.15.164:0/4019590470 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.363+0000 7f3a1e241700 1 -- 172.21.15.164:0/4019590470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a180767e0 con 0x7f3a18102220 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.363+0000 7f3a1e241700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006010 tx=0x7f3a08005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.364+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a08026070 con 0x7f3a18102220 2024-04-02T17:03:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.364+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a18074e90 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.364+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a18075350 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.364+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0800bbd0 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.364+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a080038e0 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.365+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a18103700 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.365+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3a08029070 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.365+0000 7f3a0f7fe700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 0x7f3a0403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.365+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a0801e070 con 0x7f3a18102220 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.366+0000 7f3a1da40700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 0x7f3a0403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.367+0000 7f3a1da40700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 0x7f3a0403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a14006fd0 tx=0x7f3a14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.381+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a08013070 con 0x7f3a18102220 2024-04-02T17:03:36.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.593+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi154:172.21.15.154=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}) -- 0x7f3a180755e0 con 0x7f3a04039650 2024-04-02T17:03:36.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.608+0000 7f3a0f7fe700 1 -- 172.21.15.164:0/4019590470 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3a180755e0 con 0x7f3a04039650 2024-04-02T17:03:36.609 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2024-04-02T17:03:36.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 msgr2=0x7f3a0403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 0x7f3a0403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a14006fd0 tx=0x7f3a14006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 msgr2=0x7f3a180748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006010 tx=0x7f3a08005030 comp rx=0 tx=0).stop 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 shutdown_connections 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a04039650 0x7f3a0403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 --2- 172.21.15.164:0/4019590470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a18102220 0x7f3a180748f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 >> 172.21.15.164:0/4019590470 conn(0x7f3a180fdad0 msgr2=0x7f3a18103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 shutdown_connections 2024-04-02T17:03:36.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:36.610+0000 7f3a204a5700 1 -- 172.21.15.164:0/4019590470 wait complete. 2024-04-02T17:03:37.198 DEBUG:teuthology.orchestra.run.smithi154:mon.b> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.b.service 2024-04-02T17:03:37.201 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.c.service 2024-04-02T17:03:37.203 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T17:03:37.204 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:03:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:38.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:37 smithi059 ceph-mon[31664]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi154:172.21.15.154=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:03:38.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:37 smithi059 ceph-mon[31664]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi154:172.21.15.154=b;smithi164:172.21.15.164=c;count:3 2024-04-02T17:03:38.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:37 smithi059 ceph-mon[31664]: Deploying daemon mon.c on smithi164 2024-04-02T17:03:39.220 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:03:39.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:38 smithi059 ceph-mon[31664]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:40.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fada8fc4700 1 -- 172.21.15.164:0/151094192 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad9c007b20 con 0x7fada41071f0 2024-04-02T17:03:40.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 -- 172.21.15.164:0/151094192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 msgr2=0x7fada41075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:40.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 --2- 172.21.15.164:0/151094192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada41075d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00b0a0 tx=0x7fad9c00b3b0 comp rx=0 tx=0).stop 2024-04-02T17:03:40.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 -- 172.21.15.164:0/151094192 shutdown_connections 2024-04-02T17:03:40.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 --2- 172.21.15.164:0/151094192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada41075d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:40.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 -- 172.21.15.164:0/151094192 >> 172.21.15.164:0/151094192 conn(0x7fada40fbdf0 msgr2=0x7fada40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:40.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.979+0000 7fadac22a700 1 -- 172.21.15.164:0/151094192 shutdown_connections 2024-04-02T17:03:40.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.980+0000 7fadac22a700 1 -- 172.21.15.164:0/151094192 wait complete. 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.980+0000 7fadac22a700 1 Processor -- start 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.980+0000 7fadac22a700 1 -- start start 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.980+0000 7fadac22a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.980+0000 7fadac22a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fada410c750 con 0x7fada41071f0 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.981+0000 7fada9fc6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.981+0000 7fada9fc6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.164:46770/0 (socket says 172.21.15.164:46770) 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.981+0000 7fada9fc6700 1 -- 172.21.15.164:0/2301898451 learned_addr learned my addr 172.21.15.164:0/2301898451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:03:40.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fada9fc6700 1 -- 172.21.15.164:0/2301898451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada419dd20 con 0x7fada41071f0 2024-04-02T17:03:40.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fada9fc6700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad9c015040 tx=0x7fad9c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad9c017040 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada419dfb0 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41979a0 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9c022070 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.982+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad9c027ae0 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.983+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad88005330 con 0x7fada41071f0 2024-04-02T17:03:40.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.984+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fad9c027460 con 0x7fada41071f0 2024-04-02T17:03:40.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.984+0000 7fad9affd700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 0x7fad9003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:40.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.984+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad9c00f050 con 0x7fada41071f0 2024-04-02T17:03:40.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.985+0000 7fada97c5700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 0x7fad9003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:40.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:40.986+0000 7fada97c5700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 0x7fad9003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad94006fd0 tx=0x7fad94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:41.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.000+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fad9c007e20 con 0x7fada41071f0 2024-04-02T17:03:41.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.292+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fad88005620 con 0x7fada41071f0 2024-04-02T17:03:41.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.293+0000 7fad9affd700 1 -- 172.21.15.164:0/2301898451 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fad9c01b5b0 con 0x7fada41071f0 2024-04-02T17:03:41.294 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:03:41.294 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","modified":"2024-04-02T17:01:29.288578Z","created":"2024-04-02T17:01:29.288578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 msgr2=0x7fad9003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 0x7fad9003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad94006fd0 tx=0x7fad94006e40 comp rx=0 tx=0).stop 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 msgr2=0x7fada419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad9c015040 tx=0x7fad9c003ce0 comp rx=0 tx=0).stop 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 shutdown_connections 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fad90039650 0x7fad9003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 --2- 172.21.15.164:0/2301898451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada41071f0 0x7fada419d7e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 >> 172.21.15.164:0/2301898451 conn(0x7fada40fbdf0 msgr2=0x7fada41090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 shutdown_connections 2024-04-02T17:03:41.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:41.295+0000 7fadac22a700 1 -- 172.21.15.164:0/2301898451 wait complete. 2024-04-02T17:03:41.297 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2024-04-02T17:03:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:40 smithi059 ceph-mon[31664]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:42.960 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T17:03:42.960 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T17:03:43.200 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: mon.a calling monitor election 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: mon.c calling monitor election 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: monmap epoch 2 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: last_changed 2024-04-02T17:03:41.385169+0000 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: min_mon_release 19 (squid) 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: election_strategy: 1 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:46.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: fsmap 2024-04-02T17:03:46.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:46.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: mgrmap e13: x(active, since 57s) 2024-04-02T17:03:46.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: overall HEALTH_OK 2024-04-02T17:03:46.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:46.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:46.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:47.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 -- 172.21.15.164:0/2489645865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab380035c0 msgr2=0x7fab38005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:47.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 --2- 172.21.15.164:0/2489645865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab380035c0 0x7fab38005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab4c06ba30 tx=0x7fab4800feb0 comp rx=0 tx=0).stop 2024-04-02T17:03:47.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 -- 172.21.15.164:0/2489645865 shutdown_connections 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 --2- 172.21.15.164:0/2489645865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab380035c0 0x7fab38005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 -- 172.21.15.164:0/2489645865 >> 172.21.15.164:0/2489645865 conn(0x7fab4c0776a0 msgr2=0x7fab4c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 -- 172.21.15.164:0/2489645865 shutdown_connections 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.707+0000 7fab5338c700 1 -- 172.21.15.164:0/2489645865 wait complete. 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 Processor -- start 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 -- start start 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 0x7fab4c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10a1a0 con 0x7fab4c19dda0 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab5338c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10a020 con 0x7fab380035c0 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.708+0000 7fab51128700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38020/0 (socket says 172.21.15.164:38020) 2024-04-02T17:03:47.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 -- 172.21.15.164:0/1376058927 learned_addr learned my addr 172.21.15.164:0/1376058927 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:03:47.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 -- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 msgr2=0x7fab4c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T17:03:47.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 -- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 msgr2=0x7fab4c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T17:03:47.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T17:03:47.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab51128700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:03:47.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab50927700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 0x7fab4c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab50927700 1 -- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 msgr2=0x7fab4c19d860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab50927700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.709+0000 7fab50927700 1 -- 172.21.15.164:0/1376058927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c19e3e0 con 0x7fab4c19dda0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab50927700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 0x7fab4c197a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00b4d0 tx=0x7fab3c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab3c011630 con 0x7fab4c19dda0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c197f60 con 0x7fab4c19dda0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c198570 con 0x7fab4c19dda0 2024-04-02T17:03:47.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3c011cb0 con 0x7fab4c19dda0 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.710+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab3c0193b0 con 0x7fab4c19dda0 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.711+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab38002100 con 0x7fab4c19dda0 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.712+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fab3c018020 con 0x7fab4c19dda0 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.712+0000 7fab427fc700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 0x7fab3003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.712+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fab3c04c180 con 0x7fab4c19dda0 2024-04-02T17:03:47.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.713+0000 7fab51128700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 0x7fab3003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:47.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.714+0000 7fab51128700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 0x7fab3003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab4800fa60 tx=0x7fab48005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:47.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:47.727+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab3c01f020 con 0x7fab4c19dda0 2024-04-02T17:03:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:03:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:47 smithi059 ceph-mon[31664]: Deploying daemon mon.b on smithi154 2024-04-02T17:03:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:03:47.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:47.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:47 smithi164 ceph-mon[36499]: Deploying daemon mon.b on smithi154 2024-04-02T17:03:47.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:48.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.010+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fab38002bf0 con 0x7fab4c19dda0 2024-04-02T17:03:48.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.011+0000 7fab427fc700 1 -- 172.21.15.164:0/1376058927 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fab3c018570 con 0x7fab4c19dda0 2024-04-02T17:03:48.012 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:03:48.012 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","modified":"2024-04-02T17:03:41.385169Z","created":"2024-04-02T17:01:29.288578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T17:03:48.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 msgr2=0x7fab3003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:48.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 0x7fab3003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab4800fa60 tx=0x7fab48005960 comp rx=0 tx=0).stop 2024-04-02T17:03:48.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 msgr2=0x7fab4c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:48.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 0x7fab4c197a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00b4d0 tx=0x7fab3c00b890 comp rx=0 tx=0).stop 2024-04-02T17:03:48.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 shutdown_connections 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab380035c0 0x7fab4c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab30039440 0x7fab3003b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 --2- 172.21.15.164:0/1376058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab4c19dda0 0x7fab4c197a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 >> 172.21.15.164:0/1376058927 conn(0x7fab4c0776a0 msgr2=0x7fab4c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 shutdown_connections 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:48.014+0000 7fab5338c700 1 -- 172.21.15.164:0/1376058927 wait complete. 2024-04-02T17:03:48.016 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2024-04-02T17:03:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:48 smithi164 ceph-mon[36499]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:48 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/1376058927' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T17:03:48.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:48 smithi059 ceph-mon[31664]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:48.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:48 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/1376058927' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T17:03:49.619 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T17:03:49.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T17:03:49.863 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:03:50.361 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:50 smithi154 ceph-mon[36479]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T17:03:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:50 smithi154 bash[36460]: 14d487407afd3893ce7bdfd0a5bf3e49c47371f0a5d0a54e4ca7c424a69f5b46 2024-04-02T17:03:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:50 smithi154 systemd[1]: Started Ceph mon.b for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:03:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 -- 172.21.15.164:0/3940045375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300035c0 msgr2=0x7f2830005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 --2- 172.21.15.164:0/3940045375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300035c0 0x7f2830005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2848102000 tx=0x7f283800fe60 comp rx=0 tx=0).stop 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 -- 172.21.15.164:0/3940045375 shutdown_connections 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 --2- 172.21.15.164:0/3940045375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300035c0 0x7f2830005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 -- 172.21.15.164:0/3940045375 >> 172.21.15.164:0/3940045375 conn(0x7f28480fdbc0 msgr2=0x7f28480fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.683+0000 7f284c951700 1 -- 172.21.15.164:0/3940045375 shutdown_connections 2024-04-02T17:03:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.684+0000 7f284c951700 1 -- 172.21.15.164:0/3940045375 wait complete. 2024-04-02T17:03:55.684 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: mon.c calling monitor election 2024-04-02T17:03:55.684 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:03:55.684 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: mon.a calling monitor election 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: monmap epoch 3 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:55.685 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: min_mon_release 19 (squid) 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: election_strategy: 1 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: fsmap 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: mgrmap e13: x(active, since 66s) 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: overall HEALTH_OK 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.686 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.685+0000 7f284c951700 1 Processor -- start 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.685+0000 7f284c951700 1 -- start start 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.685+0000 7f284c951700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.685+0000 7f284c951700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 0x7f284819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284c951700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284c951700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f284810a110 con 0x7f28481982a0 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284c951700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2848109f90 con 0x7f28300035c0 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284c951700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f284810a290 con 0x7f284819cb70 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33576/0 (socket says 172.21.15.164:33576) 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 -- 172.21.15.164:0/37590041 learned_addr learned my addr 172.21.15.164:0/37590041 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 msgr2=0x7f284819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 msgr2=0x7f284819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T17:03:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284659c700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2846d9d700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f2845d9b700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 0x7f284819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284659c700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 msgr2=0x7f2848197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284659c700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 msgr2=0x7f2848197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284659c700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.686+0000 7f284659c700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 msgr2=0x7f284819d020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 msgr2=0x7f2848197d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 -- 172.21.15.164:0/37590041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28481a48e0 con 0x7f28481982a0 2024-04-02T17:03:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.687+0000 7f2845d9b700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 0x7f284819c630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f283c00b550 tx=0x7f283c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.688+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004230 con 0x7f28481982a0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.688+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2848074880 con 0x7f28481982a0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.688+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2848074e10 con 0x7f28481982a0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.688+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283c0043d0 con 0x7f28481982a0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.688+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c011c20 con 0x7f28481982a0 2024-04-02T17:03:55.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.689+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2830002140 con 0x7f28481982a0 2024-04-02T17:03:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.690+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f283c0036a0 con 0x7f28481982a0 2024-04-02T17:03:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.690+0000 7f28377fe700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 0x7f282803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.690+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f283c04c2c0 con 0x7f28481982a0 2024-04-02T17:03:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.691+0000 7f284659c700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 0x7f282803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.692+0000 7f284659c700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 0x7f282803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2838020040 tx=0x7f2838001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:55.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:55.711+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f283c021e00 con 0x7f28481982a0 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: mon.c calling monitor election 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: mon.a calling monitor election 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: monmap epoch 3 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: min_mon_release 19 (squid) 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: election_strategy: 1 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: fsmap 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: mgrmap e13: x(active, since 66s) 2024-04-02T17:03:55.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: overall HEALTH_OK 2024-04-02T17:03:55.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:55.801 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:56.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.005+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2830002c30 con 0x7f28481982a0 2024-04-02T17:03:56.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.006+0000 7f28377fe700 1 -- 172.21.15.164:0/37590041 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f283c003d30 con 0x7f28481982a0 2024-04-02T17:03:56.006 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-02T17:03:56.006 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","modified":"2024-04-02T17:03:50.359639Z","created":"2024-04-02T17:01:29.288578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T17:03:56.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 msgr2=0x7f282803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 0x7f282803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2838020040 tx=0x7f2838001e70 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 msgr2=0x7f284819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 0x7f284819c630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f283c00b550 tx=0x7f283c00b910 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 shutdown_connections 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284819cb70 0x7f284819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28300035c0 0x7f2848197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2828039ae0 0x7f282803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 --2- 172.21.15.164:0/37590041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28481982a0 0x7f284819c630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 >> 172.21.15.164:0/37590041 conn(0x7f28480fdbc0 msgr2=0x7f2848108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 shutdown_connections 2024-04-02T17:03:56.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:03:56.009+0000 7f284c951700 1 -- 172.21.15.164:0/37590041 wait complete. 2024-04-02T17:03:56.010 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2024-04-02T17:03:56.620 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:56 smithi164 ceph-mon[36499]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:56.620 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:56 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:56.620 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:56 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/37590041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T17:03:56.620 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:56 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:56.650 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T17:03:56.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T17:03:56.729 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:56 smithi059 ceph-mon[31664]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:56.729 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:56 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:56.729 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/37590041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T17:03:56.729 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:56 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:03:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: mon.c calling monitor election 2024-04-02T17:03:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:03:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: mon.a calling monitor election 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: monmap epoch 3 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: min_mon_release 19 (squid) 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: election_strategy: 1 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: fsmap 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: mgrmap e13: x(active, since 66s) 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: overall HEALTH_OK 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:57.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:03:57.653 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/37590041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T17:03:57.653 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:58.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mon.b calling monitor election 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mon.b calling monitor election 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mon.c calling monitor election 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mon.a calling monitor election 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: monmap epoch 3 2024-04-02T17:03:58.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: min_mon_release 19 (squid) 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: election_strategy: 1 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:58.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: fsmap 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: mgrmap e13: x(active, since 68s) 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: overall HEALTH_OK 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:58.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mon.b calling monitor election 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mon.b calling monitor election 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mon.c calling monitor election 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mon.a calling monitor election 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: monmap epoch 3 2024-04-02T17:03:58.354 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: min_mon_release 19 (squid) 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: election_strategy: 1 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: fsmap 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: mgrmap e13: x(active, since 68s) 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: overall HEALTH_OK 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.355 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.356 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:58.356 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mon.b calling monitor election 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mon.b calling monitor election 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mon.c calling monitor election 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mon.a calling monitor election 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T17:03:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: monmap epoch 3 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: last_changed 2024-04-02T17:03:50.359639+0000 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: created 2024-04-02T17:01:29.288578+0000 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: min_mon_release 19 (squid) 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: election_strategy: 1 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: fsmap 2024-04-02T17:03:58.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: mgrmap e13: x(active, since 68s) 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: overall HEALTH_OK 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:03:58.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.957+0000 7f45b2000700 1 -- 172.21.15.59:0/3050076894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac103180 msgr2=0x7f45ac103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.957+0000 7f45b2000700 1 --2- 172.21.15.59:0/3050076894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac103180 0x7f45ac103560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f459c009910 tx=0x7f459c009c20 comp rx=0 tx=0).stop 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.958+0000 7f45b2000700 1 -- 172.21.15.59:0/3050076894 shutdown_connections 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.958+0000 7f45b2000700 1 --2- 172.21.15.59:0/3050076894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac103180 0x7f45ac103560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.958+0000 7f45b2000700 1 -- 172.21.15.59:0/3050076894 >> 172.21.15.59:0/3050076894 conn(0x7f45ac077b30 msgr2=0x7f45ac0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.958+0000 7f45b2000700 1 -- 172.21.15.59:0/3050076894 shutdown_connections 2024-04-02T17:03:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.958+0000 7f45b2000700 1 -- 172.21.15.59:0/3050076894 wait complete. 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 Processor -- start 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 -- start start 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ac19a510 0x7f45ac19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac19ede0 0x7f45ac19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac10c620 con 0x7f45ac19ede0 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac10c4a0 con 0x7f45ac103180 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.959+0000 7f45b2000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac10c7a0 con 0x7f45ac19a510 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50830/0 (socket says 172.21.15.59:50830) 2024-04-02T17:03:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 -- 172.21.15.59:0/1386276338 learned_addr learned my addr 172.21.15.59:0/1386276338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45aaa00700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ac19a510 0x7f45ac19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45abe00700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac19ede0 0x7f45ac19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 -- 172.21.15.59:0/1386276338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ac19a510 msgr2=0x7f45ac19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ac19a510 0x7f45ac19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 -- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac19ede0 msgr2=0x7f45ac19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac19ede0 0x7f45ac19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 -- 172.21.15.59:0/1386276338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a6b50 con 0x7f45ac103180 2024-04-02T17:03:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.960+0000 7f45ab400700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f459c009fd0 tx=0x7f459c00bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.961+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c026070 con 0x7f45ac103180 2024-04-02T17:03:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.961+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459c005da0 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.961+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a6d20 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.961+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0038e0 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.961+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a71a0 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.962+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f459c029070 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.962+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac063c20 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.962+0000 7f45a3e00700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 0x7f459003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.962+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f459c04d6f0 con 0x7f45ac103180 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.963+0000 7f45aaa00700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 0x7f459003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:03:58.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.963+0000 7f45aaa00700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 0x7f459003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4598009ad0 tx=0x7f4598009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:03:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:58.968+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f459c013070 con 0x7f45ac103180 2024-04-02T17:03:59.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.226+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f45ac02dd30 con 0x7f45ac103180 2024-04-02T17:03:59.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:59.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi059:/etc/ceph/ceph.conf 2024-04-02T17:03:59.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-02T17:03:59.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: Updating smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.226+0000 7f45a3e00700 1 -- 172.21.15.59:0/1386276338 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f459c030020 con 0x7f45ac103180 2024-04-02T17:03:59.228 INFO:teuthology.orchestra.run.smithi059.stdout:# minimal ceph.conf for 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:59.228 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-02T17:03:59.228 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:03:59.228 INFO:teuthology.orchestra.run.smithi059.stdout: mon_host = [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-02T17:03:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.229+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 msgr2=0x7f459003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.229+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 0x7f459003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4598009ad0 tx=0x7f4598009210 comp rx=0 tx=0).stop 2024-04-02T17:03:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.229+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 msgr2=0x7f45ac199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:03:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.229+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f459c009fd0 tx=0x7f459c00bde0 comp rx=0 tx=0).stop 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 shutdown_connections 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4590039ae0 0x7f459003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45ac19ede0 0x7f45ac19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45ac103180 0x7f45ac199fd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 --2- 172.21.15.59:0/1386276338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ac19a510 0x7f45ac19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 >> 172.21.15.59:0/1386276338 conn(0x7f45ac077b30 msgr2=0x7f45ac109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 shutdown_connections 2024-04-02T17:03:59.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:03:59.230+0000 7f45b2000700 1 -- 172.21.15.59:0/1386276338 wait complete. 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi059:/etc/ceph/ceph.conf 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: Updating smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:59.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:03:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.506 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:03:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:03:59.360+0000 7f4e25200700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi059:/etc/ceph/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: Updating smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T17:03:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:03:59.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:03:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:03:59.948 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T17:03:59.948 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:03:59.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T17:04:00.027 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:04:00.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:04:00.150 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:04:00.150 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T17:04:00.182 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:04:00.182 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:04:00.250 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:04:00.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T17:04:00.286 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:04:00.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T17:04:00.352 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: Reconfiguring daemon mon.a on smithi059 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1386276338' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.353 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.356 INFO:tasks.cephadm:Adding mgr.x on smithi059 2024-04-02T17:04:00.356 INFO:tasks.cephadm:Adding mgr.y on smithi154 2024-04-02T17:04:00.356 INFO:tasks.cephadm:Adding mgr.z on smithi164 2024-04-02T17:04:00.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi059=x;smithi154=y;smithi164=z' 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:00.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:00.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: Reconfiguring daemon mon.a on smithi059 2024-04-02T17:04:00.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1386276338' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:00.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: Reconfiguring daemon mon.a on smithi059 2024-04-02T17:04:00.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1386276338' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:00.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:00.631 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:04:01.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:01 smithi059 ceph-mon[31664]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:01 smithi154 ceph-mon[36479]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:01 smithi164 ceph-mon[36499]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:02.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 -- 172.21.15.164:0/4187039904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c102c00 msgr2=0x7fc87c104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:02.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 --2- 172.21.15.164:0/4187039904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c102c00 0x7fc87c104ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc87800b0a0 tx=0x7fc87800b3b0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 -- 172.21.15.164:0/4187039904 shutdown_connections 2024-04-02T17:04:02.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 --2- 172.21.15.164:0/4187039904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c102c00 0x7fc87c104ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 -- 172.21.15.164:0/4187039904 >> 172.21.15.164:0/4187039904 conn(0x7fc87c0fc5d0 msgr2=0x7fc87c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 -- 172.21.15.164:0/4187039904 shutdown_connections 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.077+0000 7fc882fcf700 1 -- 172.21.15.164:0/4187039904 wait complete. 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 Processor -- start 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 -- start start 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87c100bb0 0x7fc87c101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 0x7fc87c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c108160 con 0x7fc87c101570 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c107fe0 con 0x7fc87c100bb0 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc882fcf700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1082e0 con 0x7fc87c102c00 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc880d6b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc880d6b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33590/0 (socket says 172.21.15.164:33590) 2024-04-02T17:04:02.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc880d6b700 1 -- 172.21.15.164:0/1370346096 learned_addr learned my addr 172.21.15.164:0/1370346096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.078+0000 7fc873fff700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87c100bb0 0x7fc87c101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 -- 172.21.15.164:0/1370346096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87c100bb0 msgr2=0x7fc87c101030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87c100bb0 0x7fc87c101030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc88156c700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 0x7fc87c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 -- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 msgr2=0x7fc87c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 0x7fc87c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 -- 172.21.15.164:0/1370346096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a1f50 con 0x7fc87c102c00 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc88156c700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 0x7fc87c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc880d6b700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc878015040 tx=0x7fc878009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878017040 con 0x7fc87c102c00 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a21e0 con 0x7fc87c102c00 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc87800be10 con 0x7fc87c102c00 2024-04-02T17:04:02.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878012a20 con 0x7fc87c102c00 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.079+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a27a0 con 0x7fc87c102c00 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.080+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc878012bc0 con 0x7fc87c102c00 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.080+0000 7fc871ffb700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 0x7fc86403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.080+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c063c20 con 0x7fc87c102c00 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.080+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc87800f020 con 0x7fc87c102c00 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.081+0000 7fc873fff700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 0x7fc86403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.082+0000 7fc873fff700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 0x7fc86403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ad0 tx=0x7fc86c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:02.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.086+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc878029020 con 0x7fc87c102c00 2024-04-02T17:04:02.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.296+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi154=y;smithi164=z", "target": ["mon-mgr", ""]}) -- 0x7fc87c062a30 con 0x7fc864039ae0 2024-04-02T17:04:02.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.310+0000 7fc871ffb700 1 -- 172.21.15.164:0/1370346096 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc87c062a30 con 0x7fc864039ae0 2024-04-02T17:04:02.310 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2024-04-02T17:04:02.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 msgr2=0x7fc86403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 0x7fc86403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ad0 tx=0x7fc86c009210 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 msgr2=0x7fc87c102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc878015040 tx=0x7fc878009740 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 shutdown_connections 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc864039ae0 0x7fc86403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc87c101570 0x7fc87c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc87c102c00 0x7fc87c102560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 --2- 172.21.15.164:0/1370346096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc87c100bb0 0x7fc87c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 >> 172.21.15.164:0/1370346096 conn(0x7fc87c0fc5d0 msgr2=0x7fc87c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 shutdown_connections 2024-04-02T17:04:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:04:02.313+0000 7fc882fcf700 1 -- 172.21.15.164:0/1370346096 wait complete. 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: Reconfiguring mon.b (monmap changed)... 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: Reconfiguring mon.b (monmap changed)... 2024-04-02T17:04:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: Reconfiguring mon.b (monmap changed)... 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:02.588 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:02.931 DEBUG:teuthology.orchestra.run.smithi154:mgr.y> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.y.service 2024-04-02T17:04:02.933 DEBUG:teuthology.orchestra.run.smithi164:mgr.z> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.z.service 2024-04-02T17:04:02.935 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T17:04:02.936 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:04:02.936 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T17:04:02.998 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T17:04:02.998 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-02 17:03:01.695001393 +0000 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-02 16:57:34.386711512 +0000 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-02 16:57:34.386711512 +0000 2024-04-02T17:04:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-02 16:57:34.386711512 +0000 2024-04-02T17:04:03.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T17:04:03.156 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-02T17:04:03.156 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-02T17:04:03.156 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000392136 s, 1.3 MB/s 2024-04-02T17:04:03.160 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T17:04:03.217 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-04-02T17:04:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-02 17:03:01.786000279 +0000 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-02 16:57:34.979704330 +0000 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-02 16:57:34.979704330 +0000 2024-04-02T17:04:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-02 16:57:34.979704330 +0000 2024-04-02T17:04:03.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T17:04:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-02T17:04:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-02T17:04:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000351792 s, 1.5 MB/s 2024-04-02T17:04:03.361 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: Reconfiguring daemon mon.b on smithi154 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:03.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:03.422 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-04-02T17:04:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-02 17:03:01.850999483 +0000 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-02 16:57:35.648696228 +0000 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-02 16:57:35.648696228 +0000 2024-04-02T17:04:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-02 16:57:35.648696228 +0000 2024-04-02T17:04:03.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T17:04:03.513 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: Reconfiguring daemon mon.b on smithi154 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:03.514 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: Reconfiguring daemon mon.b on smithi154 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T17:04:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:03.575 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-02T17:04:03.575 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-02T17:04:03.575 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000349994 s, 1.5 MB/s 2024-04-02T17:04:03.580 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T17:04:03.645 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-02 17:03:01.936998429 +0000 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-02 16:57:36.211689410 +0000 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-02 16:57:36.211689410 +0000 2024-04-02T17:04:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-02 16:57:36.211689410 +0000 2024-04-02T17:04:03.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T17:04:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-02T17:04:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-02T17:04:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000334636 s, 1.5 MB/s 2024-04-02T17:04:03.802 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T17:04:03.865 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-02T17:04:03.865 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T17:04:03.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T17:04:03.884 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-04-02T17:04:03.942 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-02 17:03:49.822400172 +0000 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-02 16:56:08.168891501 +0000 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-02 16:56:08.168891501 +0000 2024-04-02T17:04:03.943 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-02 16:56:08.168891501 +0000 2024-04-02T17:04:03.943 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T17:04:04.014 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-02T17:04:04.014 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-02T17:04:04.014 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000211388 s, 2.4 MB/s 2024-04-02T17:04:04.016 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T17:04:04.076 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-02 17:03:49.822400172 +0000 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-02 16:56:08.502882508 +0000 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-02 16:56:08.502882508 +0000 2024-04-02T17:04:04.135 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-02 16:56:08.502882508 +0000 2024-04-02T17:04:04.136 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T17:04:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-02T17:04:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-02T17:04:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000195158 s, 2.6 MB/s 2024-04-02T17:04:04.209 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T17:04:04.268 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-02 17:03:49.822400172 +0000 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-02 16:56:08.855873003 +0000 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-02 16:56:08.855873003 +0000 2024-04-02T17:04:04.328 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-02 16:56:08.855873003 +0000 2024-04-02T17:04:04.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T17:04:04.395 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi154=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: Saving service mgr spec with placement smithi059=x;smithi154=y;smithi164=z;count:3 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: Reconfiguring mon.c (monmap changed)... 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: Reconfiguring daemon mon.c on smithi164 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:04.396 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:04.397 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-02T17:04:04.398 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-02T17:04:04.398 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000262115 s, 2.0 MB/s 2024-04-02T17:04:04.399 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T17:04:04.457 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-02 17:03:49.823400146 +0000 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-02 16:56:09.185864117 +0000 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-02 16:56:09.185864117 +0000 2024-04-02T17:04:04.516 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-02 16:56:09.185864117 +0000 2024-04-02T17:04:04.517 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi154=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: Saving service mgr spec with placement smithi059=x;smithi154=y;smithi164=z;count:3 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: Reconfiguring mon.c (monmap changed)... 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: Reconfiguring daemon mon.c on smithi164 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi154=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: Saving service mgr spec with placement smithi059=x;smithi154=y;smithi164=z;count:3 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: Reconfiguring mon.c (monmap changed)... 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: Reconfiguring daemon mon.c on smithi164 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-02T17:04:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-02T17:04:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000187919 s, 2.7 MB/s 2024-04-02T17:04:04.586 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T17:04:04.644 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-02T17:04:04.645 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T17:04:04.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T17:04:04.665 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-02 17:03:40.232803813 +0000 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-02 16:56:19.926904073 +0000 2024-04-02T17:04:04.720 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-02 16:56:19.926904073 +0000 2024-04-02T17:04:04.721 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-02 16:56:19.926904073 +0000 2024-04-02T17:04:04.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T17:04:04.788 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-02T17:04:04.788 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-02T17:04:04.788 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000272016 s, 1.9 MB/s 2024-04-02T17:04:04.789 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T17:04:04.845 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:04.900 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-02 17:03:40.232803813 +0000 2024-04-02T17:04:04.901 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-02 16:56:20.262898280 +0000 2024-04-02T17:04:04.901 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-02 16:56:20.262898280 +0000 2024-04-02T17:04:04.901 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-02 16:56:20.262898280 +0000 2024-04-02T17:04:04.901 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T17:04:04.965 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-02T17:04:04.965 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-02T17:04:04.966 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000216496 s, 2.4 MB/s 2024-04-02T17:04:04.967 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T17:04:05.023 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-02 17:03:40.233803797 +0000 2024-04-02T17:04:05.078 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-02 16:56:20.783889297 +0000 2024-04-02T17:04:05.079 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-02 16:56:20.783889297 +0000 2024-04-02T17:04:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-02 16:56:20.783889297 +0000 2024-04-02T17:04:05.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T17:04:05.145 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-02T17:04:05.145 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-02T17:04:05.145 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000255569 s, 2.0 MB/s 2024-04-02T17:04:05.146 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T17:04:05.201 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T17:04:05.256 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-02 17:03:40.233803797 +0000 2024-04-02T17:04:05.257 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-02 16:56:21.152882935 +0000 2024-04-02T17:04:05.257 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-02 16:56:21.152882935 +0000 2024-04-02T17:04:05.257 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-02 16:56:21.152882935 +0000 2024-04-02T17:04:05.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T17:04:05.321 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-02T17:04:05.322 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-02T17:04:05.322 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000196117 s, 2.6 MB/s 2024-04-02T17:04:05.323 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T17:04:05.381 INFO:tasks.cephadm:Deploying osd.0 on smithi059 with /dev/vg_nvme/lv_4... 2024-04-02T17:04:05.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T17:04:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:05 smithi059 ceph-mon[31664]: Deploying daemon mgr.z on smithi164 2024-04-02T17:04:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:05 smithi164 ceph-mon[36499]: Deploying daemon mgr.z on smithi164 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:05 smithi154 ceph-mon[36479]: Deploying daemon mgr.z on smithi164 2024-04-02T17:04:05.844 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:04:06.323 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 systemd[1]: Starting Ceph mgr.z for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:04:06.603 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 podman[38366]: 2024-04-02 17:04:06.350661391 +0000 UTC m=+0.111148091 container create 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T17:04:06.603 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 podman[38366]: 2024-04-02 17:04:06.257057149 +0000 UTC m=+0.017543862 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:04:06.603 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 podman[38366]: 2024-04-02 17:04:06.553650201 +0000 UTC m=+0.314136911 container init 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T17:04:06.603 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 podman[38366]: 2024-04-02 17:04:06.55626853 +0000 UTC m=+0.316755240 container start 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-02T17:04:06.604 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 ceph-mgr[38385]: -- 172.21.15.164:0/3283727590 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a94f1ae5a0 con 0x55a94f182c00 2024-04-02T17:04:06.604 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 bash[38366]: 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 2024-04-02T17:04:07.019 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:06 smithi164 systemd[1]: Started Ceph mgr.z for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:07.322 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:07.106+0000 7f5c90b45200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T17:04:07.322 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:07.199+0000 7f5c90b45200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:07.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:07.470 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:07.470 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:07.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:08.177 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:07.843+0000 7f5c90b45200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T17:04:08.178 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:07.936+0000 7f5c90b45200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T17:04:08.178 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.079+0000 7f5c90b45200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T17:04:08.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:08 smithi154 ceph-mon[36479]: Deploying daemon mgr.y on smithi154 2024-04-02T17:04:08.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-mon[36499]: Deploying daemon mgr.y on smithi154 2024-04-02T17:04:08.461 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.178+0000 7f5c90b45200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T17:04:08.461 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.265+0000 7f5c90b45200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T17:04:08.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:08 smithi059 ceph-mon[31664]: Deploying daemon mgr.y on smithi154 2024-04-02T17:04:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T17:04:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T17:04:08.630 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-04-02T17:04:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-04-02T17:04:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0375866 s, 279 MB/s 2024-04-02T17:04:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-04-02T17:04:08.747 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.543+0000 7f5c90b45200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T17:04:08.747 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.657+0000 7f5c90b45200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T17:04:08.747 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.747+0000 7f5c90b45200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T17:04:08.835 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:08 smithi154 systemd[1]: Starting Ceph mgr.y for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:04:09.025 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:08.844+0000 7f5c90b45200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T17:04:09.025 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:09.025+0000 7f5c90b45200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T17:04:09.131 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:08 smithi154 podman[37838]: 2024-04-02 17:04:08.738601057 +0000 UTC m=+0.017323532 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:04:09.131 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:08 smithi154 podman[37838]: 2024-04-02 17:04:08.946791371 +0000 UTC m=+0.225513838 container create 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:04:09.131 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 podman[37838]: 2024-04-02 17:04:09.13236727 +0000 UTC m=+0.411089741 container init 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T17:04:09.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-04-02T17:04:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:09 smithi164 ceph-mon[36499]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:09 smithi154 ceph-mon[36479]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:09.400 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 podman[37838]: 2024-04-02 17:04:09.13504444 +0000 UTC m=+0.413766906 container start 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T17:04:09.400 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 ceph-mgr[37858]: -- 172.21.15.154:0/2725417511 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b814dc5a0 con 0x558b814b0c00 2024-04-02T17:04:09.400 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 bash[37838]: 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 2024-04-02T17:04:09.400 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 systemd[1]: Started Ceph mgr.y for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:04:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:09 smithi059 ceph-mon[31664]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:04:10.150 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:09.693+0000 7f3050df1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T17:04:10.150 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:09.790+0000 7f3050df1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T17:04:10.303 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:10.036+0000 7f5c90b45200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T17:04:10.303 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:10.184+0000 7f5c90b45200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:10.535+0000 7f5c90b45200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:10.570 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:10.472+0000 7f3050df1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T17:04:10.571 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.571 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.571 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.571 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.572 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:10.572 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:10.572 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:10.822 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:10.626+0000 7f5c90b45200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T17:04:10.897 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:10.572+0000 7f3050df1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T17:04:10.897 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:10.715+0000 7f3050df1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T17:04:10.898 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:10.810+0000 7f3050df1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T17:04:10.898 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:10.897+0000 7f3050df1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T17:04:11.311 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:10.825+0000 7f5c90b45200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:04:11.311 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:11.021+0000 7f5c90b45200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T17:04:11.459 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:11.169+0000 7f3050df1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T17:04:11.459 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:11.279+0000 7f3050df1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T17:04:11.459 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:11.364+0000 7f3050df1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T17:04:11.459 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:11.459+0000 7f3050df1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T17:04:11.572 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:11.311+0000 7f5c90b45200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 -- 172.21.15.59:0/917668242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201059e0 msgr2=0x7f7220107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 --2- 172.21.15.59:0/917668242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201059e0 0x7f7220107dd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f720c009990 tx=0x7f720c009ca0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 -- 172.21.15.59:0/917668242 shutdown_connections 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 --2- 172.21.15.59:0/917668242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f722010a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 --2- 172.21.15.59:0/917668242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201030b0 0x7f72201054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 --2- 172.21.15.59:0/917668242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201059e0 0x7f7220107dd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 -- 172.21.15.59:0/917668242 >> 172.21.15.59:0/917668242 conn(0x7f72200fcac0 msgr2=0x7f72200fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:04:11.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 -- 172.21.15.59:0/917668242 shutdown_connections 2024-04-02T17:04:11.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.631+0000 7f7227e00700 1 -- 172.21.15.59:0/917668242 wait complete. 2024-04-02T17:04:11.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.632+0000 7f7227e00700 1 Processor -- start 2024-04-02T17:04:11.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.632+0000 7f7227e00700 1 -- start start 2024-04-02T17:04:11.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201059e0 0x7f7220101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f72201015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f722010e260 con 0x7f7220108310 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f722010e0e0 con 0x7f72201030b0 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7227e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f722010e3e0 con 0x7f72201059e0 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59564/0 (socket says 172.21.15.59:59564) 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 -- 172.21.15.59:0/1487090851 learned_addr learned my addr 172.21.15.59:0/1487090851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7224a00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201059e0 0x7f7220101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f72201015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:11.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 -- 172.21.15.59:0/1487090851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201059e0 msgr2=0x7f7220101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201059e0 0x7f7220101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 -- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 msgr2=0x7f72201015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f72201015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225400700 1 -- 172.21.15.59:0/1487090851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7220101e50 con 0x7f72201030b0 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.633+0000 7f7225e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f72201015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f7225400700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f721400e9c0 tx=0x7f721400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721400cbd0 con 0x7f72201030b0 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721400cd70 con 0x7f72201030b0 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214010430 con 0x7f72201030b0 2024-04-02T17:04:11.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72201a2a90 con 0x7f72201030b0 2024-04-02T17:04:11.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.634+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72201a2fa0 con 0x7f72201030b0 2024-04-02T17:04:11.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.635+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f72140105d0 con 0x7f72201030b0 2024-04-02T17:04:11.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.636+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72200692b0 con 0x7f72201030b0 2024-04-02T17:04:11.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.636+0000 7f721e000700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 0x7f720803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:11.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.636+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f721404cd70 con 0x7f72201030b0 2024-04-02T17:04:11.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.636+0000 7f7224a00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 0x7f720803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:11.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.637+0000 7f7224a00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 0x7f720803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f720c00b7b0 tx=0x7f720c00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:11.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.641+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7214016b90 con 0x7f72201030b0 2024-04-02T17:04:11.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:11.883+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7220062a30 con 0x7f7208039890 2024-04-02T17:04:11.899 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:11.634+0000 7f3050df1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T17:04:11.927 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:11.574+0000 7f5c90b45200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T17:04:11.928 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:11.813+0000 7f5c90b45200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T17:04:12.222 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.222 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:12.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:12.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:12.900 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:12.597+0000 7f3050df1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T17:04:12.900 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:12.726+0000 7f3050df1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T17:04:12.994 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:12.620+0000 7f5c90b45200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T17:04:13.039 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:12 smithi059 ceph-mon[31664]: Reconfiguring daemon mgr.x on smithi059 2024-04-02T17:04:13.039 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:12 smithi059 ceph-mon[31664]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:13.040 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:12 smithi059 ceph-mon[31664]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:13.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:12 smithi164 ceph-mon[36499]: Reconfiguring daemon mgr.x on smithi059 2024-04-02T17:04:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:12 smithi164 ceph-mon[36499]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:13.292 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:12 smithi164 ceph-mon[36499]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:13.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:13.193+0000 7f5c90b45200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T17:04:13.292 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:13.292+0000 7f5c90b45200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:12 smithi154 ceph-mon[36479]: Reconfiguring daemon mgr.x on smithi059 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:12 smithi154 ceph-mon[36479]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:12 smithi154 ceph-mon[36479]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:13.060+0000 7f3050df1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:13.148+0000 7f3050df1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T17:04:13.342 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:13.341+0000 7f3050df1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:04:13.572 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:04:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z[38381]: 2024-04-02T17:04:13.512+0000 7f5c90b45200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T17:04:13.649 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:13.526+0000 7f3050df1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T17:04:14.069 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:13.809+0000 7f3050df1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T17:04:14.069 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:14.068+0000 7f3050df1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T17:04:14.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:14.246+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f7214021960 con 0x7f72201030b0 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:14.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T17:04:14.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:14.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: Standby manager daemon z started 2024-04-02T17:04:14.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T17:04:14.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:14 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:14.399 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:14.306+0000 7f3050df1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T17:04:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:14.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: Standby manager daemon z started 2024-04-02T17:04:14.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T17:04:14.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:14 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:14.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: Standby manager daemon z started 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T17:04:14.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:14 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.164:0/710841632' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-mon[36479]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:15.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-mon[36479]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-02T17:04:15.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T17:04:15.401 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:15.102+0000 7f3050df1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T17:04:15.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:15 smithi059 ceph-mon[31664]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:15.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:15 smithi059 ceph-mon[31664]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-02T17:04:15.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T17:04:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:15 smithi164 ceph-mon[36499]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:15.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:15 smithi164 ceph-mon[36499]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-02T17:04:15.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T17:04:15.981 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:15.672+0000 7f3050df1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T17:04:15.981 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:15.768+0000 7f3050df1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T17:04:15.981 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:04:15 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y[37854]: 2024-04-02T17:04:15.981+0000 7f3050df1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T17:04:16.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:16.260+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mon.2 v2:172.21.15.154:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f721401f340 con 0x7f72201030b0 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: Standby manager daemon y started 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1208947782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]': finished 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T17:04:16.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: Standby manager daemon y started 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1208947782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]': finished 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T17:04:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:16 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: Standby manager daemon y started 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='mgr.? 172.21.15.154:0/485123564' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1208947782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]: dispatch 2024-04-02T17:04:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb8100a6-a589-4808-b2c2-870b291c27de"}]': finished 2024-04-02T17:04:16.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T17:04:16.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:16 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:17.552 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:17 smithi154 ceph-mon[36479]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:17.552 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:17 smithi154 ceph-mon[36479]: mgrmap e15: x(active, since 86s), standbys: z, y 2024-04-02T17:04:17.552 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T17:04:17.552 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:17 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2089877608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:17 smithi164 ceph-mon[36499]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:17.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:17 smithi164 ceph-mon[36499]: mgrmap e15: x(active, since 86s), standbys: z, y 2024-04-02T17:04:17.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T17:04:17.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:17 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2089877608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:17 smithi059 ceph-mon[31664]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:17 smithi059 ceph-mon[31664]: mgrmap e15: x(active, since 86s), standbys: z, y 2024-04-02T17:04:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T17:04:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:17 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2089877608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:18 smithi164 ceph-mon[36499]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:18 smithi154 ceph-mon[36479]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:18 smithi059 ceph-mon[31664]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:20.789 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:20 smithi059 ceph-mon[31664]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:20.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:20 smithi164 ceph-mon[36499]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:20.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:20 smithi154 ceph-mon[36479]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:22.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:22 smithi059 ceph-mon[31664]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:22.825 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:22 smithi164 ceph-mon[36499]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:22.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:22 smithi154 ceph-mon[36479]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:24 smithi059 ceph-mon[31664]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:24 smithi164 ceph-mon[36499]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:24 smithi154 ceph-mon[36479]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:25.724 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T17:04:25.724 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:25.724 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:25 smithi164 ceph-mon[36499]: Deploying daemon osd.0 on smithi059 2024-04-02T17:04:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T17:04:25.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:25.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:25 smithi059 ceph-mon[31664]: Deploying daemon osd.0 on smithi059 2024-04-02T17:04:25.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T17:04:25.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:25.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:25 smithi154 ceph-mon[36479]: Deploying daemon osd.0 on smithi059 2024-04-02T17:04:26.768 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:26 smithi059 ceph-mon[31664]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:26.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:26 smithi164 ceph-mon[36499]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:26.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:26 smithi154 ceph-mon[36479]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:28 smithi154 ceph-mon[36479]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:28.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:28 smithi059 ceph-mon[31664]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:28.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:28 smithi164 ceph-mon[36499]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:30.537 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:30 smithi059 ceph-mon[31664]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:30.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:30 smithi164 ceph-mon[36499]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:30.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:30 smithi154 ceph-mon[36479]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:32.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:32 smithi059 ceph-mon[31664]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:32.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:32 smithi164 ceph-mon[36499]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:32.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:32 smithi154 ceph-mon[36479]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:33.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:33.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:33.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:33.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:35.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:34 smithi154 ceph-mon[36479]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:35.294 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:34 smithi059 ceph-mon[31664]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:34 smithi164 ceph-mon[36499]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:36.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:36.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:36.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.432+0000 7f721e000700 1 -- 172.21.15.59:0/1487090851 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7220062a30 con 0x7f7208039890 2024-04-02T17:04:37.433 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 0 on host 'smithi059' 2024-04-02T17:04:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 msgr2=0x7f720803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 0x7f720803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f720c00b7b0 tx=0x7f720c00b880 comp rx=0 tx=0).stop 2024-04-02T17:04:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 msgr2=0x7f7220102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f721400e9c0 tx=0x7f721400ecd0 comp rx=0 tx=0).stop 2024-04-02T17:04:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 shutdown_connections 2024-04-02T17:04:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7208039890 0x7f720803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7220108310 0x7f72201015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72201030b0 0x7f7220102a10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 --2- 172.21.15.59:0/1487090851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72201059e0 0x7f7220101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 >> 172.21.15.59:0/1487090851 conn(0x7f72200fcac0 msgr2=0x7f7220103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:04:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 shutdown_connections 2024-04-02T17:04:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:04:37.436+0000 7f7227e00700 1 -- 172.21.15.59:0/1487090851 wait complete. 2024-04-02T17:04:37.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:37 smithi059 ceph-mon[31664]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T17:04:37.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:37.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:37 smithi164 ceph-mon[36499]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T17:04:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:37 smithi154 ceph-mon[36479]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T17:04:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:38.087 DEBUG:teuthology.orchestra.run.smithi059:osd.0> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T17:04:38.091 INFO:tasks.cephadm:Deploying osd.1 on smithi154 with /dev/vg_nvme/lv_4... 2024-04-02T17:04:38.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T17:04:38.335 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:04:38.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-mon[31664]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:38.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-mon[31664]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T17:04:38.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-mon[31664]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T17:04:38.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-mon[31664]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:04:38.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:38.634 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0[44182]: 2024-04-02T17:04:38.534+0000 7f687d800700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T17:04:38.634 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:04:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0[44182]: 2024-04-02T17:04:38.540+0000 7f6878600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:04:38.760 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:38 smithi154 ceph-mon[36479]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:38.761 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:38 smithi154 ceph-mon[36479]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T17:04:38.761 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:38 smithi154 ceph-mon[36479]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T17:04:38.761 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:38 smithi154 ceph-mon[36479]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:04:38.761 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:38 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:38 smithi164 ceph-mon[36499]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:38 smithi164 ceph-mon[36499]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T17:04:38.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:38 smithi164 ceph-mon[36499]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T17:04:38.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:38 smithi164 ceph-mon[36499]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:04:38.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:38 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:39 smithi059 ceph-mon[31664]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:04:39.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:39 smithi059 ceph-mon[31664]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T17:04:39.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.777 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:39 smithi154 ceph-mon[36479]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:04:39.777 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:39 smithi154 ceph-mon[36479]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T17:04:39.777 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.777 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:39 smithi164 ceph-mon[36499]: from='osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:04:39.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:39 smithi164 ceph-mon[36499]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T17:04:39.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:39.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123547 s, 849 MB/s 2024-04-02T17:04:40.151 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] boot 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T17:04:40.704 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:40 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:40.731 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-04-02T17:04:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:04:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:04:40.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:40.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] boot 2024-04-02T17:04:40.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T17:04:40.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:40 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:04:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:04:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T17:04:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: osd.0 [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] boot 2024-04-02T17:04:40.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T17:04:40.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:40 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T17:04:40.976 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:04:41.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:41 smithi154 ceph-mon[36479]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T17:04:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:41 smithi059 ceph-mon[31664]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T17:04:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:41 smithi164 ceph-mon[36499]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T17:04:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 -- 172.21.15.154:0/691192331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98101520 msgr2=0x7f2c98101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 --2- 172.21.15.154:0/691192331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98101520 0x7f2c98101940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c880099c0 tx=0x7f2c88009cd0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 -- 172.21.15.154:0/691192331 shutdown_connections 2024-04-02T17:04:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 --2- 172.21.15.154:0/691192331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c981030c0 0x7f2c98109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 --2- 172.21.15.154:0/691192331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98101520 0x7f2c98101940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 --2- 172.21.15.154:0/691192331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98102720 0x7f2c98102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 -- 172.21.15.154:0/691192331 >> 172.21.15.154:0/691192331 conn(0x7f2c980fcae0 msgr2=0x7f2c980fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.398+0000 7f2ca076b700 1 -- 172.21.15.154:0/691192331 shutdown_connections 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 -- 172.21.15.154:0/691192331 wait complete. 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 Processor -- start 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 -- start start 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98101520 0x7f2c98105440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c981030c0 0x7f2c98104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d190 con 0x7f2c981030c0 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d010 con 0x7f2c98102720 2024-04-02T17:04:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2ca076b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810d310 con 0x7f2c98101520 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2c9dd06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2c9dd06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55586/0 (socket says 172.21.15.154:55586) 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2c9dd06700 1 -- 172.21.15.154:0/399355611 learned_addr learned my addr 172.21.15.154:0/399355611 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.399+0000 7f2c9e507700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98101520 0x7f2c98105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 -- 172.21.15.154:0/399355611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98101520 msgr2=0x7f2c98105440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98101520 0x7f2c98105440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 -- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c981030c0 msgr2=0x7f2c98104040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c981030c0 0x7f2c98104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 -- 172.21.15.154:0/399355611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c98104880 con 0x7f2c98102720 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c9dd06700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400d980 tx=0x7f2c9400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94017070 con 0x7f2c98102720 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a29f0 con 0x7f2c98102720 2024-04-02T17:04:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c94005020 con 0x7f2c98102720 2024-04-02T17:04:42.403 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94009bc0 con 0x7f2c98102720 2024-04-02T17:04:42.403 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.400+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a2fb0 con 0x7f2c98102720 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.401+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c980692b0 con 0x7f2c98102720 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.402+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2c940051d0 con 0x7f2c98102720 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.403+0000 7f2c8f7fe700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 0x7f2c840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.403+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2c940ce840 con 0x7f2c98102720 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.403+0000 7f2c9e507700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 0x7f2c840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:04:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.404+0000 7f2c9e507700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 0x7f2c840a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b580 tx=0x7f2c88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:04:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.407+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2c94020020 con 0x7f2c98102720 2024-04-02T17:04:42.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:04:42.616+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2c980feca0 con 0x7f2c840a3c30 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:42.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:04:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:04:42.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:42.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi059 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 17095M 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:43.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi059 2024-04-02T17:04:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 17095M 2024-04-02T17:04:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:43.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:43.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:43.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:43.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:43.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:43.869 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi059 2024-04-02T17:04:43.869 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 17095M 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:04:43.870 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:44.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:44 smithi164 ceph-mon[36499]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:44.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:44 smithi154 ceph-mon[36479]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:45.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:44 smithi059 ceph-mon[31664]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:45 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/1156734927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:45.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:45 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:45.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:45 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]': finished 2024-04-02T17:04:45.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:45 smithi164 ceph-mon[36499]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T17:04:45.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:45 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:04:45.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:45 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/1156734927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:45.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:45 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:45.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:45 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]': finished 2024-04-02T17:04:45.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:45 smithi154 ceph-mon[36479]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T17:04:45.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:45 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:04:46.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:45 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/1156734927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:46.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:45 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]: dispatch 2024-04-02T17:04:46.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:45 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d"}]': finished 2024-04-02T17:04:46.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:45 smithi059 ceph-mon[31664]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T17:04:46.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:45 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:04:46.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:46 smithi164 ceph-mon[36499]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:46.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:46 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/2659369037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:46 smithi154 ceph-mon[36479]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:46 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/2659369037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:47.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:46 smithi059 ceph-mon[31664]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:47.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:46 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/2659369037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:04:48.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:48 smithi164 ceph-mon[36499]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:48.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:48 smithi154 ceph-mon[36479]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:49.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:48 smithi059 ceph-mon[31664]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:50 smithi154 ceph-mon[36479]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:50.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:50 smithi164 ceph-mon[36499]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:51.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:50 smithi059 ceph-mon[31664]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:52.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:52 smithi154 ceph-mon[36479]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T17:04:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:53.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:52 smithi059 ceph-mon[31664]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T17:04:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:52 smithi164 ceph-mon[36499]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T17:04:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:04:53.855 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:53 smithi154 ceph-mon[36479]: Deploying daemon osd.1 on smithi154 2024-04-02T17:04:54.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:53 smithi059 ceph-mon[31664]: Deploying daemon osd.1 on smithi154 2024-04-02T17:04:54.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:53 smithi164 ceph-mon[36499]: Deploying daemon osd.1 on smithi154 2024-04-02T17:04:54.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:54 smithi154 ceph-mon[36479]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:55.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:54 smithi059 ceph-mon[31664]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:55.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:54 smithi164 ceph-mon[36499]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:56.838 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:56 smithi154 ceph-mon[36479]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:56.874 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:56 smithi164 ceph-mon[36499]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:57.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:56 smithi059 ceph-mon[31664]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:58 smithi059 ceph-mon[31664]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:04:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:59.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:58 smithi164 ceph-mon[36499]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:59.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:04:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:04:59.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:58 smithi154 ceph-mon[36479]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:04:59.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:04:59.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:04:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:00.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:00.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:00.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:00.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:00.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:00.875 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:00.947 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 1 on host 'smithi154' 2024-04-02T17:05:00.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.946+0000 7f2c8f7fe700 1 -- 172.21.15.154:0/399355611 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2c980feca0 con 0x7f2c840a3c30 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 msgr2=0x7f2c840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 0x7f2c840a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b580 tx=0x7f2c88005fb0 comp rx=0 tx=0).stop 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 msgr2=0x7f2c98103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400d980 tx=0x7f2c9400dc90 comp rx=0 tx=0).stop 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 shutdown_connections 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2c840a3c30 0x7f2c840a60f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:00.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c981030c0 0x7f2c98104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:00.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c98101520 0x7f2c98105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:00.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 --2- 172.21.15.154:0/399355611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c98102720 0x7f2c98103b00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:00.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 >> 172.21.15.154:0/399355611 conn(0x7f2c980fcae0 msgr2=0x7f2c980fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:05:00.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 shutdown_connections 2024-04-02T17:05:00.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:05:00.949+0000 7f2ca076b700 1 -- 172.21.15.154:0/399355611 wait complete. 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:01.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:01.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:01.493 DEBUG:teuthology.orchestra.run.smithi154:osd.1> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T17:05:01.496 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/vg_nvme/lv_4... 2024-04-02T17:05:01.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T17:05:01.648 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:05:01.611+0000 7fdab99c67c0 -1 osd.1 0 log_to_monitors true 2024-04-02T17:05:01.742 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:05:02.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.112 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:02.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:01 smithi164 ceph-mon[36499]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:02.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:01 smithi164 ceph-mon[36499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:01 smithi059 ceph-mon[31664]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:01 smithi059 ceph-mon[31664]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:02.387 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.387 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:02.387 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:02.388 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-mon[36479]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:02.388 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:01 smithi154 ceph-mon[36479]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:03.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:02 smithi164 ceph-mon[36499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T17:05:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T17:05:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:02 smithi059 ceph-mon[31664]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:03.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T17:05:03.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: from='osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T17:05:03.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:03.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:02 smithi154 ceph-mon[36479]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:05:03.676 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T17:05:03.676 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T17:05:03.676 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-02T17:05:03.677 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-02T17:05:03.677 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114302 s, 917 MB/s 2024-04-02T17:05:03.677 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-02T17:05:04.141 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-mon[36479]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:05:04.141 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-mon[36479]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T17:05:04.142 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.142 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-mon[36479]: from='osd.1 ' entity='osd.1' 2024-04-02T17:05:04.142 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.142 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:05:03.875+0000 7fdaaf82c700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T17:05:04.143 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:05:04.010+0000 7fdaa6e16700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:05:04.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:03 smithi164 ceph-mon[36499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:05:04.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:03 smithi164 ceph-mon[36499]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T17:05:04.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:03 smithi164 ceph-mon[36499]: from='osd.1 ' entity='osd.1' 2024-04-02T17:05:04.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-04-02T17:05:04.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:03 smithi059 ceph-mon[31664]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:05:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:03 smithi059 ceph-mon[31664]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T17:05:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:03 smithi059 ceph-mon[31664]: from='osd.1 ' entity='osd.1' 2024-04-02T17:05:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:04.526 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:05:05.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:05:05.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:05:05.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:05.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.333 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:05.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:05.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.334 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 -- 172.21.15.164:0/137153254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c108360 msgr2=0x7f882c10a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 --2- 172.21.15.164:0/137153254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c108360 0x7f882c10a770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f882400b0a0 tx=0x7f882400b3b0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 -- 172.21.15.164:0/137153254 shutdown_connections 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 --2- 172.21.15.164:0/137153254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c108360 0x7f882c10a770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 --2- 172.21.15.164:0/137153254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f882c103100 0x7f882c1054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 --2- 172.21.15.164:0/137153254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c105a30 0x7f882c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 -- 172.21.15.164:0/137153254 >> 172.21.15.164:0/137153254 conn(0x7f882c0fcab0 msgr2=0x7f882c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 -- 172.21.15.164:0/137153254 shutdown_connections 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 -- 172.21.15.164:0/137153254 wait complete. 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.054+0000 7f8830b7d700 1 Processor -- start 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 -- start start 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c103100 0x7f882c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f882c108360 0x7f882c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f882c10e240 con 0x7f882c108360 2024-04-02T17:05:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f882c10e0c0 con 0x7f882c103100 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8830b7d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f882c10e3c0 con 0x7f882c105a30 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37786/0 (socket says 172.21.15.164:37786) 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 -- 172.21.15.164:0/1982475111 learned_addr learned my addr 172.21.15.164:0/1982475111 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f882a59c700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c103100 0x7f882c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 -- 172.21.15.164:0/1982475111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c103100 msgr2=0x7f882c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c103100 0x7f882c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 -- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f882c108360 msgr2=0x7f882c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f882c108360 0x7f882c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 -- 172.21.15.164:0/1982475111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c1a0460 con 0x7f882c105a30 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8821d9b700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f881400c950 tx=0x7f881400cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:05:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.055+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814007ac0 con 0x7f882c105a30 2024-04-02T17:05:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.056+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8814004d10 con 0x7f882c105a30 2024-04-02T17:05:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.056+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c1a0750 con 0x7f882c105a30 2024-04-02T17:05:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.056+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881400f450 con 0x7f882c105a30 2024-04-02T17:05:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.056+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1a0c90 con 0x7f882c105a30 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.056+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c0692b0 con 0x7f882c105a30 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.058+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8814004750 con 0x7f882c105a30 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.058+0000 7f8823fff700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 0x7f88180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.058+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f88140cec80 con 0x7f882c105a30 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.059+0000 7f882a59c700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 0x7f88180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.060+0000 7f882a59c700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 0x7f88180a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:05:06.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.062+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8814014030 con 0x7f882c105a30 2024-04-02T17:05:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:06.268+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f882c062a30 con 0x7f88180a3bb0 2024-04-02T17:05:06.267 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:05 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi154 2024-04-02T17:05:06.267 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:05 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 17080M 2024-04-02T17:05:06.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:05 smithi164 ceph-mon[36499]: osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] boot 2024-04-02T17:05:06.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:05 smithi164 ceph-mon[36499]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T17:05:06.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:06.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:05 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi154 2024-04-02T17:05:06.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:05 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 17080M 2024-04-02T17:05:06.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:05 smithi059 ceph-mon[31664]: osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] boot 2024-04-02T17:05:06.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:05 smithi059 ceph-mon[31664]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T17:05:06.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:06.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:05 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi154 2024-04-02T17:05:06.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:05 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 17080M 2024-04-02T17:05:06.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:05 smithi154 ceph-mon[36479]: osd.1 [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] boot 2024-04-02T17:05:06.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:05 smithi154 ceph-mon[36479]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T17:05:06.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T17:05:07.233 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:06 smithi164 ceph-mon[36499]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:07.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:06 smithi164 ceph-mon[36499]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T17:05:07.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:07.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:07.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:07.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:06 smithi059 ceph-mon[31664]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:06 smithi059 ceph-mon[31664]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T17:05:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:07.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:06 smithi154 ceph-mon[36479]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T17:05:07.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:06 smithi154 ceph-mon[36479]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T17:05:07.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:07.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:07.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:08.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:08 smithi059 ceph-mon[31664]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:08.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:08 smithi164 ceph-mon[36499]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:08.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:08 smithi154 ceph-mon[36479]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/744276972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]': finished 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T17:05:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:09 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/744276972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]': finished 2024-04-02T17:05:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T17:05:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:09 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/744276972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]: dispatch 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fde8bb0-3569-424d-8271-a0f77a83f346"}]': finished 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T17:05:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:09 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:10 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/341547769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:10 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/341547769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:10.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:10 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/341547769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:11 smithi059 ceph-mon[31664]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:11.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:11 smithi164 ceph-mon[36499]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:11 smithi154 ceph-mon[36479]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:13.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:13 smithi164 ceph-mon[36499]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:13 smithi059 ceph-mon[31664]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:13.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:13 smithi154 ceph-mon[36479]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:15 smithi059 ceph-mon[31664]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:15.304 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:15 smithi164 ceph-mon[36499]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:15 smithi154 ceph-mon[36479]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:17 smithi059 ceph-mon[31664]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T17:05:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:17.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:17 smithi164 ceph-mon[36499]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:17.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T17:05:17.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:17 smithi154 ceph-mon[36479]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T17:05:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:18.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:18 smithi164 ceph-mon[36499]: Deploying daemon osd.2 on smithi164 2024-04-02T17:05:18.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:18 smithi059 ceph-mon[31664]: Deploying daemon osd.2 on smithi164 2024-04-02T17:05:18.301 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:18 smithi154 ceph-mon[36479]: Deploying daemon osd.2 on smithi164 2024-04-02T17:05:19.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:19 smithi164 ceph-mon[36499]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:19 smithi154 ceph-mon[36479]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:19 smithi059 ceph-mon[31664]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:21.119 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:21 smithi164 ceph-mon[36499]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:21.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:21 smithi154 ceph-mon[36479]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:21 smithi059 ceph-mon[31664]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:22.910 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:22 smithi164 ceph-mon[36499]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:22.910 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:22.910 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:22.910 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:23.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:22 smithi059 ceph-mon[31664]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:23.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:23.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:23.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:23.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:22 smithi154 ceph-mon[36479]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:23.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:23.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:23.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:25.004 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.060 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2024-04-02T17:05:25.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.059+0000 7f8823fff700 1 -- 172.21.15.164:0/1982475111 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f882c062a30 con 0x7f88180a3bb0 2024-04-02T17:05:25.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 msgr2=0x7f88180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:25.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 0x7f88180a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c008040 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 msgr2=0x7f882c198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f881400c950 tx=0x7f881400cd10 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 shutdown_connections 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88180a3bb0 0x7f88180a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f882c108360 0x7f882c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f882c103100 0x7f882c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 --2- 172.21.15.164:0/1982475111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f882c105a30 0x7f882c198d40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 >> 172.21.15.164:0/1982475111 conn(0x7f882c0fcab0 msgr2=0x7f882c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 shutdown_connections 2024-04-02T17:05:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:05:25.062+0000 7f8830b7d700 1 -- 172.21.15.164:0/1982475111 wait complete. 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:25.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:25.695 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T17:05:25.698 INFO:tasks.cephadm:Deploying osd.3 on smithi059 with /dev/vg_nvme/lv_3... 2024-04-02T17:05:25.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T17:05:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:26 smithi059 ceph-mon[31664]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:26 smithi059 ceph-mon[31664]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:26.174 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:05:26.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:26.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:26 smithi164 ceph-mon[36499]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:26.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:26 smithi164 ceph-mon[36499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:26.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:26.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:26.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:26 smithi154 ceph-mon[36479]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:26.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:26 smithi154 ceph-mon[36479]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T17:05:27.262 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:27.262 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T17:05:27.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T17:05:27.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:27.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:27.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:27 smithi059 ceph-mon[31664]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-mon[36499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:27.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T17:05:27.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T17:05:27.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:27.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: from='osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:27.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:27 smithi154 ceph-mon[36479]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:05:28.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-mon[36499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:05:28.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-mon[36499]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T17:05:28.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-mon[36499]: from='osd.2 ' entity='osd.2' 2024-04-02T17:05:28.270 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2[41668]: 2024-04-02T17:05:27.995+0000 7f6e4cec6700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T17:05:28.270 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2[41668]: 2024-04-02T17:05:28.175+0000 7f6e444b0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:05:28.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:28 smithi059 ceph-mon[31664]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:05:28.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:28 smithi059 ceph-mon[31664]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T17:05:28.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:28 smithi059 ceph-mon[31664]: from='osd.2 ' entity='osd.2' 2024-04-02T17:05:28.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:28 smithi154 ceph-mon[36479]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:05:28.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:28 smithi154 ceph-mon[36479]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T17:05:28.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:28.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:28 smithi154 ceph-mon[36479]: from='osd.2 ' entity='osd.2' 2024-04-02T17:05:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T17:05:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T17:05:28.682 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-04-02T17:05:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-04-02T17:05:28.682 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0389268 s, 269 MB/s 2024-04-02T17:05:28.683 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:29.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:29.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:29 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:29 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:29.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:29 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:05:30.242 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi164 2024-04-02T17:05:30.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 17080M 2024-04-02T17:05:30.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] boot 2024-04-02T17:05:30.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T17:05:30.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:30.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:30.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi164 2024-04-02T17:05:30.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 17080M 2024-04-02T17:05:30.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] boot 2024-04-02T17:05:30.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T17:05:30.403 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:30.403 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi164 2024-04-02T17:05:30.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 17080M 2024-04-02T17:05:30.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: osd.2 [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] boot 2024-04-02T17:05:30.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T17:05:30.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T17:05:30.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 ceph-mon[31664]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 ceph-mon[31664]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T17:05:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:31.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 ceph-mon[36479]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:31.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:31.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 ceph-mon[36479]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T17:05:31.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:31.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 ceph-mon[36499]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T17:05:31.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:31.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 ceph-mon[36499]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T17:05:31.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:05:31.606 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48419]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:31.606 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.606 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48419]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.606 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48419]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.607 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:31.607 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.607 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48423]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.607 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48423]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 -- 172.21.15.59:0/30436035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 msgr2=0x7fccd0107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 --2- 172.21.15.59:0/30436035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd0107dd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009990 tx=0x7fccc0009ca0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 -- 172.21.15.59:0/30436035 shutdown_connections 2024-04-02T17:05:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 --2- 172.21.15.59:0/30436035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd0108310 0x7fccd010a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 --2- 172.21.15.59:0/30436035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd01030b0 0x7fccd01054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 --2- 172.21.15.59:0/30436035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd0107dd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 -- 172.21.15.59:0/30436035 >> 172.21.15.59:0/30436035 conn(0x7fccd00fcac0 msgr2=0x7fccd00fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:05:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 -- 172.21.15.59:0/30436035 shutdown_connections 2024-04-02T17:05:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.689+0000 7fccd8800700 1 -- 172.21.15.59:0/30436035 wait complete. 2024-04-02T17:05:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.690+0000 7fccd8800700 1 Processor -- start 2024-04-02T17:05:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 -- start start 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 0x7fccd019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd0108310 0x7fccd01a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010e230 con 0x7fccd01030b0 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010e0b0 con 0x7fccd0108310 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd8800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010e3b0 con 0x7fccd01059e0 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd5e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 0x7fccd019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd5e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 0x7fccd019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49950/0 (socket says 172.21.15.59:49950) 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd5e00700 1 -- 172.21.15.59:0/2742141177 learned_addr learned my addr 172.21.15.59:0/2742141177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:05:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60556/0 (socket says 172.21.15.59:60556) 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.691+0000 7fccd6800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd0108310 0x7fccd01a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 -- 172.21.15.59:0/2742141177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd0108310 msgr2=0x7fccd01a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd0108310 0x7fccd01a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 -- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 msgr2=0x7fccd019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 0x7fccd019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 -- 172.21.15.59:0/2742141177 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd01a2610 con 0x7fccd01059e0 2024-04-02T17:05:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fccd5400700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccc00098e0 tx=0x7fccc000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:05:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.692+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0026070 con 0x7fccd01059e0 2024-04-02T17:05:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.693+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd01a28a0 con 0x7fccd01059e0 2024-04-02T17:05:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.693+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc000f460 con 0x7fccd01059e0 2024-04-02T17:05:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.693+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0021670 con 0x7fccd01059e0 2024-04-02T17:05:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.693+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a2cf0 con 0x7fccd01059e0 2024-04-02T17:05:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.694+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd00692b0 con 0x7fccd01059e0 2024-04-02T17:05:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.696+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fccc0029070 con 0x7fccd01059e0 2024-04-02T17:05:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.698+0000 7fcccaa00700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 0x7fccb80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:05:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.698+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fccc00d0230 con 0x7fccd01059e0 2024-04-02T17:05:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.698+0000 7fccd5e00700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 0x7fccb80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:05:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.699+0000 7fccd5e00700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 0x7fccb80a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009680 tx=0x7fccc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:05:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.700+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fccc0013070 con 0x7fccd01059e0 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45078]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45078]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.747 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.748 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45081]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.822 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45744]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45744]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45744]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45744]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45747]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45747]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45747]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:31.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45750]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45750]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:31.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45753]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:31 smithi164 sudo[45753]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:31.949+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fccd0062a30 con 0x7fccb80a3c30 2024-04-02T17:05:31.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:31.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48431]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48431]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:31.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:31.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:31.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48435]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:31.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:31 smithi059 sudo[48435]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45084]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:31 smithi154 sudo[45087]: pam_unix(sudo:session): session closed for user root 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T17:05:32.120 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.121 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:32.122 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:32.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:32.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:32.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:32.438+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fccc0009d10 con 0x7fccd01059e0 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T17:05:32.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:05:32.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:33.151 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:33 smithi059 ceph-mon[31664]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:33.151 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:33 smithi059 ceph-mon[31664]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:33.151 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:33 smithi059 ceph-mon[31664]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T17:05:33.151 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:33 smithi059 ceph-mon[31664]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T17:05:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:33 smithi154 ceph-mon[36479]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:33 smithi154 ceph-mon[36479]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:33 smithi154 ceph-mon[36479]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T17:05:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:33 smithi154 ceph-mon[36479]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T17:05:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:33 smithi164 ceph-mon[36499]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:33 smithi164 ceph-mon[36499]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:05:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:33 smithi164 ceph-mon[36499]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T17:05:33.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:33 smithi164 ceph-mon[36499]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T17:05:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:35 smithi154 ceph-mon[36479]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:35.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:35 smithi059 ceph-mon[31664]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:35.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:35 smithi164 ceph-mon[36499]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:36 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]: dispatch 2024-04-02T17:05:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:36 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]': finished 2024-04-02T17:05:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:36 smithi154 ceph-mon[36479]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T17:05:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:36 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]: dispatch 2024-04-02T17:05:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:36 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]': finished 2024-04-02T17:05:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:36 smithi059 ceph-mon[31664]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T17:05:36.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:36 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]: dispatch 2024-04-02T17:05:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:36 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4108085920' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73897ded-e965-4239-b88f-776b7af2eb73"}]': finished 2024-04-02T17:05:36.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:36 smithi164 ceph-mon[36499]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T17:05:36.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:37 smithi154 ceph-mon[36479]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:37 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2413171046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:37 smithi059 ceph-mon[31664]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2413171046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:37 smithi164 ceph-mon[36499]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:37 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2413171046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:05:39.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:39 smithi154 ceph-mon[36479]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:39 smithi059 ceph-mon[31664]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:39 smithi164 ceph-mon[36499]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:41 smithi059 ceph-mon[31664]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:41 smithi164 ceph-mon[36499]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:41 smithi154 ceph-mon[36479]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:43.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:43 smithi059 ceph-mon[31664]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:43 smithi164 ceph-mon[36499]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:43 smithi154 ceph-mon[36479]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:45 smithi059 ceph-mon[31664]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:45 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T17:05:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:45 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:45 smithi164 ceph-mon[36499]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:45 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T17:05:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:45 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:45 smithi154 ceph-mon[36479]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:45.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:45 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T17:05:45.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:45 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:46.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:46 smithi059 ceph-mon[31664]: Deploying daemon osd.3 on smithi059 2024-04-02T17:05:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:46 smithi164 ceph-mon[36499]: Deploying daemon osd.3 on smithi059 2024-04-02T17:05:46.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:46 smithi154 ceph-mon[36479]: Deploying daemon osd.3 on smithi059 2024-04-02T17:05:47.532 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:47 smithi059 ceph-mon[31664]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:47 smithi164 ceph-mon[36499]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:47 smithi154 ceph-mon[36479]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:49.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:49 smithi154 ceph-mon[36479]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:49.535 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:49 smithi059 ceph-mon[31664]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:49 smithi164 ceph-mon[36499]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:51 smithi059 ceph-mon[31664]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:51 smithi164 ceph-mon[36499]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:51 smithi154 ceph-mon[36479]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:53 smithi059 ceph-mon[31664]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:53 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:53 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:53 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:53 smithi164 ceph-mon[36499]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:53 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:53 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:53 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:53 smithi154 ceph-mon[36479]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:53 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:53 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:53 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:55.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:55 smithi059 ceph-mon[31664]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:55 smithi164 ceph-mon[36499]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:55 smithi154 ceph-mon[36479]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:57.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:57.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:57.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:57.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:57.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:57 smithi059 ceph-mon[31664]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:57.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.495+0000 7fcccaa00700 1 -- 172.21.15.59:0/2742141177 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccd0062a30 con 0x7fccb80a3c30 2024-04-02T17:05:57.496 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 3 on host 'smithi059' 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 msgr2=0x7fccb80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 0x7fccb80a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009680 tx=0x7fccc4009260 comp rx=0 tx=0).stop 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 msgr2=0x7fccd019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccc00098e0 tx=0x7fccc000bbb0 comp rx=0 tx=0).stop 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 shutdown_connections 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd0108310 0x7fccd01a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fccb80a3c30 0x7fccb80a60f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd01030b0 0x7fccd019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 --2- 172.21.15.59:0/2742141177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd01059e0 0x7fccd019aef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:05:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.498+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 >> 172.21.15.59:0/2742141177 conn(0x7fccd00fcac0 msgr2=0x7fccd0102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:05:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.499+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 shutdown_connections 2024-04-02T17:05:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:05:57.499+0000 7fccd8800700 1 -- 172.21.15.59:0/2742141177 wait complete. 2024-04-02T17:05:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:57.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:57 smithi164 ceph-mon[36499]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:05:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:57.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:57 smithi154 ceph-mon[36479]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T17:05:58.237 DEBUG:teuthology.orchestra.run.smithi059:osd.3> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T17:05:58.242 INFO:tasks.cephadm:Deploying osd.4 on smithi154 with /dev/vg_nvme/lv_3... 2024-04-02T17:05:58.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T17:05:58.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:58.440 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:05:58.426+0000 7fd792000700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T17:05:58.440 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:05:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:05:58.438+0000 7fd78ce00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:05:58.491 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.612 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:05:58.645 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:05:59.545 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] boot 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T17:05:59.546 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:05:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:05:59.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T17:05:59.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] boot 2024-04-02T17:05:59.656 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T17:05:59.656 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:05:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:05:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T17:05:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:05:59.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: osd.3 [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] boot 2024-04-02T17:05:59.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T17:05:59.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:05:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T17:06:00.195 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T17:06:00.195 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T17:06:00.196 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-02T17:06:00.196 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-02T17:06:00.196 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113576 s, 923 MB/s 2024-04-02T17:06:00.196 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-02T17:06:00.725 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:00 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:06:00.725 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:00 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:06:00.725 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:00 smithi154 ceph-mon[36479]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:06:00.725 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:00 smithi154 ceph-mon[36479]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T17:06:00.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-04-02T17:06:00.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:00 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:06:00.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:00 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:06:00.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:00 smithi059 ceph-mon[31664]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:06:00.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:00 smithi059 ceph-mon[31664]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T17:06:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:00 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:06:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:00 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:06:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:00 smithi164 ceph-mon[36499]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-02T17:06:00.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:00 smithi164 ceph-mon[36499]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T17:06:00.993 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:06:02.352 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:02 smithi154 ceph-mon[36479]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T17:06:02.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:02 smithi059 ceph-mon[31664]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T17:06:02.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.570+0000 7f10c8439700 1 -- 172.21.15.154:0/3523996559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c0105aa0 msgr2=0x7f10c0107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:02.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.570+0000 7f10c8439700 1 --2- 172.21.15.154:0/3523996559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c0105aa0 0x7f10c0107e90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009990 tx=0x7f10b0009ca0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- 172.21.15.154:0/3523996559 shutdown_connections 2024-04-02T17:06:02.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- 172.21.15.154:0/3523996559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c01083d0 0x7f10c010a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- 172.21.15.154:0/3523996559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 0x7f10c0105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- 172.21.15.154:0/3523996559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c0105aa0 0x7f10c0107e90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- 172.21.15.154:0/3523996559 >> 172.21.15.154:0/3523996559 conn(0x7f10c00fcac0 msgr2=0x7f10c00fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- 172.21.15.154:0/3523996559 shutdown_connections 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- 172.21.15.154:0/3523996559 wait complete. 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 Processor -- start 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- start start 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 0x7f10c0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c0105aa0 0x7f10c0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e240 con 0x7f10c0105aa0 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e0c0 con 0x7f10c01083d0 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.571+0000 7f10c8439700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e3c0 con 0x7f10c0103170 2024-04-02T17:06:02.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57840/0 (socket says 172.21.15.154:57840) 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 -- 172.21.15.154:0/2813460327 learned_addr learned my addr 172.21.15.154:0/2813460327 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c61d5700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 0x7f10c0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 -- 172.21.15.154:0/2813460327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 msgr2=0x7f10c0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 0x7f10c0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 -- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c0105aa0 msgr2=0x7f10c0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c0105aa0 0x7f10c0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 -- 172.21.15.154:0/2813460327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01a0490 con 0x7f10c01083d0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c69d6700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b970 tx=0x7f10b800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8019070 con 0x7f10c01083d0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a0780 con 0x7f10c01083d0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a0d40 con 0x7f10c01083d0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b800ce10 con 0x7f10c01083d0 2024-04-02T17:06:02.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.572+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80073d0 con 0x7f10c01083d0 2024-04-02T17:06:02.576 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:02 smithi164 ceph-mon[36499]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.573+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a4005360 con 0x7f10c01083d0 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.574+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10b801c070 con 0x7f10c01083d0 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.575+0000 7f10b77fe700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 0x7f10ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.575+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f10b80ce370 con 0x7f10c01083d0 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.575+0000 7f10c61d5700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 0x7f10ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:02.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.576+0000 7f10c61d5700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 0x7f10ac0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00ac50 tx=0x7f10bc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:02.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.580+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f10b809bf70 con 0x7f10c01083d0 2024-04-02T17:06:02.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:02.794+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f10a4000bf0 con 0x7f10ac0a3cc0 2024-04-02T17:06:03.525 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi059 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 8547M 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:03.526 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.527 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:03.527 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:03.527 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:03.527 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T17:06:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi059 2024-04-02T17:06:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 8547M 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:03.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:03.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:03.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi059 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 8547M 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:03.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:04 smithi059 ceph-mon[31664]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T17:06:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:04 smithi059 ceph-mon[31664]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:04 smithi164 ceph-mon[36499]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T17:06:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:04 smithi164 ceph-mon[36499]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:04.584 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:04 smithi154 ceph-mon[36479]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T17:06:04.584 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:04 smithi154 ceph-mon[36479]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:05.758 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:05 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/152647090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.758 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:05 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.758 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:05 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]': finished 2024-04-02T17:06:05.758 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:05 smithi154 ceph-mon[36479]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T17:06:05.759 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:05.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:05 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/152647090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:05 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:05 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]': finished 2024-04-02T17:06:05.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:05 smithi059 ceph-mon[31664]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T17:06:05.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:05 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/152647090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:05 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]: dispatch 2024-04-02T17:06:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:05 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e81e729a-401f-4f5c-9749-6f88a4c93261"}]': finished 2024-04-02T17:06:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:05 smithi164 ceph-mon[36499]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T17:06:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:06.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:06 smithi059 ceph-mon[31664]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:06 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/2663693979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:06 smithi164 ceph-mon[36499]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:06 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/2663693979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:06.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:06 smithi154 ceph-mon[36479]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:06.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:06 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/2663693979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:08.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:08 smithi059 ceph-mon[31664]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:08.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:08 smithi164 ceph-mon[36499]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:08.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:08 smithi154 ceph-mon[36479]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:10.698 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:10 smithi154 ceph-mon[36479]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:06:10.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:10 smithi059 ceph-mon[31664]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:06:10.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:10 smithi164 ceph-mon[36499]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:06:12.796 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:12 smithi154 ceph-mon[36479]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:06:12.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:12 smithi059 ceph-mon[31664]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:06:12.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:12 smithi164 ceph-mon[36499]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:06:13.749 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T17:06:13.750 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:13.750 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:13 smithi154 ceph-mon[36479]: Deploying daemon osd.4 on smithi154 2024-04-02T17:06:13.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T17:06:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:13 smithi059 ceph-mon[31664]: Deploying daemon osd.4 on smithi154 2024-04-02T17:06:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T17:06:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:13 smithi164 ceph-mon[36499]: Deploying daemon osd.4 on smithi154 2024-04-02T17:06:14.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:14 smithi154 ceph-mon[36479]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:14.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:14 smithi059 ceph-mon[31664]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:14 smithi164 ceph-mon[36499]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:16 smithi154 ceph-mon[36479]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:16.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:16 smithi059 ceph-mon[31664]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:16.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:16 smithi164 ceph-mon[36499]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:06:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:19 smithi059 ceph-mon[31664]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:06:19.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:19 smithi164 ceph-mon[36499]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:06:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:19.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:19 smithi154 ceph-mon[36479]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:06:19.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:19.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:21.229 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 4 on host 'smithi154' 2024-04-02T17:06:21.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.228+0000 7f10b77fe700 1 -- 172.21.15.154:0/2813460327 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f10a4000bf0 con 0x7f10ac0a3cc0 2024-04-02T17:06:21.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.230+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 msgr2=0x7f10ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:21.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.230+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 0x7f10ac0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00ac50 tx=0x7f10bc00a380 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.230+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 msgr2=0x7f10c019fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.230+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b970 tx=0x7f10b800bc80 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 shutdown_connections 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10c01083d0 0x7f10c019fdf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c0103170 0x7f10c0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f10ac0a3cc0 0x7f10ac0a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 --2- 172.21.15.154:0/2813460327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c0105aa0 0x7f10c0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 >> 172.21.15.154:0/2813460327 conn(0x7f10c00fcac0 msgr2=0x7f10c0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:06:21.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 shutdown_connections 2024-04-02T17:06:21.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:06:21.231+0000 7f10c8439700 1 -- 172.21.15.154:0/2813460327 wait complete. 2024-04-02T17:06:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:21 smithi154 ceph-mon[36479]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:21.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:21 smithi059 ceph-mon[31664]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:21.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:21 smithi164 ceph-mon[36499]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:22.070 DEBUG:teuthology.orchestra.run.smithi154:osd.4> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T17:06:22.072 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2024-04-02T17:06:22.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T17:06:22.321 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:06:22.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:22.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:22 smithi154 ceph-mon[36479]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:22.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:22 smithi154 ceph-mon[36479]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:22.401 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:06:22 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:06:22.114+0000 7f20809d07c0 -1 osd.4 0 log_to_monitors true 2024-04-02T17:06:22.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:22.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:22 smithi164 ceph-mon[36499]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:22.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:22 smithi164 ceph-mon[36499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:22 smithi059 ceph-mon[31664]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:22 smithi059 ceph-mon[31664]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:23.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:23.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:23.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T17:06:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T17:06:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:06:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:24.220 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-mon[36479]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:06:24.220 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-mon[36479]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T17:06:24.220 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:06:24.116+0000 7f2076035700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114577 s, 915 MB/s 2024-04-02T17:06:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-02T17:06:24.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:24 smithi164 ceph-mon[36499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:06:24.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:24 smithi164 ceph-mon[36499]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T17:06:24.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:24 smithi164 ceph-mon[36499]: from='osd.4 ' entity='osd.4' 2024-04-02T17:06:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:24 smithi059 ceph-mon[31664]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:06:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:24 smithi059 ceph-mon[31664]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T17:06:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:24 smithi059 ceph-mon[31664]: from='osd.4 ' entity='osd.4' 2024-04-02T17:06:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-mon[36479]: from='osd.4 ' entity='osd.4' 2024-04-02T17:06:24.650 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:06:24 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:06:24.259+0000 7f206de20700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:06:24.957 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-04-02T17:06:25.200 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:06:25.382 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:25 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:06:25.382 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:25 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:06:25.382 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:25 smithi154 ceph-mon[36479]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:25.382 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:25.382 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:25 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:06:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:25 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:06:25.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:25 smithi059 ceph-mon[31664]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:25.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:25.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:25.549 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:25 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:06:25.550 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:25 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:06:25.550 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:25 smithi164 ceph-mon[36499]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:25.550 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:25.550 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] boot 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:26.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.280 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] boot 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: osd.4 [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] boot 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:26.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:26.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 -- 172.21.15.164:0/3115268286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c103820 msgr2=0x7f2f1c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/3115268286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c103820 0x7f2f1c103c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009990 tx=0x7f2f10009ca0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 -- 172.21.15.164:0/3115268286 shutdown_connections 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/3115268286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1c1041c0 0x7f2f1c10aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/3115268286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c102620 0x7f2f1c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/3115268286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c103820 0x7f2f1c103c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 -- 172.21.15.164:0/3115268286 >> 172.21.15.164:0/3115268286 conn(0x7f2f1c0fdc00 msgr2=0x7f2f1c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:06:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 -- 172.21.15.164:0/3115268286 shutdown_connections 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.775+0000 7f2f21e5d700 1 -- 172.21.15.164:0/3115268286 wait complete. 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 Processor -- start 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 -- start start 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1c103820 0x7f2f1c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c1041c0 0x7f2f1c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c10e280 con 0x7f2f1c103820 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c10e100 con 0x7f2f1c1041c0 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f21e5d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c10e400 con 0x7f2f1c102620 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42266/0 (socket says 172.21.15.164:42266) 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 -- 172.21.15.164:0/1503631456 learned_addr learned my addr 172.21.15.164:0/1503631456 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:06:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1bfff700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c1041c0 0x7f2f1c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 -- 172.21.15.164:0/1503631456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c1041c0 msgr2=0x7f2f1c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c1041c0 0x7f2f1c19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 -- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1c103820 msgr2=0x7f2f1c198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1c103820 0x7f2f1c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 -- 172.21.15.164:0/1503631456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c1a04f0 con 0x7f2f1c102620 2024-04-02T17:06:26.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.776+0000 7f2f1b7fe700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00e980 tx=0x7f2f0c00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.777+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00cbd0 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.777+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c1a07e0 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.777+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c004510 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.777+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c010430 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.777+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c1a0da0 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.778+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c0692b0 con 0x7f2f1c102620 2024-04-02T17:06:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.779+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2f0c003e90 con 0x7f2f1c102620 2024-04-02T17:06:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.779+0000 7f2f18ff9700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 0x7f2f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.780+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f2f0c0cf6a0 con 0x7f2f1c102620 2024-04-02T17:06:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.780+0000 7f2f1affd700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 0x7f2f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.781+0000 7f2f1affd700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 0x7f2f040a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098e0 tx=0x7f2f10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:26.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.784+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2f0c09d0f0 con 0x7f2f1c102620 2024-04-02T17:06:26.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:26.994+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2f1c062a30 con 0x7f2f040a3c40 2024-04-02T17:06:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi154 2024-04-02T17:06:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 8540M 2024-04-02T17:06:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T17:06:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:27.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi154 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 8540M 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:27.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-02T17:06:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi154 2024-04-02T17:06:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 8540M 2024-04-02T17:06:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T17:06:27.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:27.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:27.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:28 smithi059 ceph-mon[31664]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:28 smithi164 ceph-mon[36499]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:28 smithi154 ceph-mon[36479]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:29.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:29 smithi059 ceph-mon[31664]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:29 smithi164 ceph-mon[36499]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:29 smithi154 ceph-mon[36479]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:30 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/2576595413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:30 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:30 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]': finished 2024-04-02T17:06:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:30 smithi164 ceph-mon[36499]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T17:06:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:30 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/2576595413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:30 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:30 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]': finished 2024-04-02T17:06:30.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:30 smithi059 ceph-mon[31664]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T17:06:30.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:30.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:30 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/2576595413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:30 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]: dispatch 2024-04-02T17:06:30.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:30 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "287aef0f-1e4b-4121-8e66-fad24be440d5"}]': finished 2024-04-02T17:06:30.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:30 smithi154 ceph-mon[36479]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T17:06:30.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:31 smithi059 ceph-mon[31664]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:31.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:31 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/4062043112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:31 smithi164 ceph-mon[36499]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:31 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/4062043112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:31 smithi154 ceph-mon[36479]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:31 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/4062043112' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:32.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:32 smithi154 ceph-mon[36479]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:32.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:32 smithi059 ceph-mon[31664]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:32.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:32 smithi164 ceph-mon[36499]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:34.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:34 smithi059 ceph-mon[31664]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:34.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:34 smithi164 ceph-mon[36499]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:34.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:34 smithi154 ceph-mon[36479]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:36.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:36 smithi059 ceph-mon[31664]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:36.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:36 smithi164 ceph-mon[36499]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:36 smithi154 ceph-mon[36479]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T17:06:37.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T17:06:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T17:06:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:38.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:38 smithi059 ceph-mon[31664]: Deploying daemon osd.5 on smithi164 2024-04-02T17:06:38.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:38 smithi059 ceph-mon[31664]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:38 smithi164 ceph-mon[36499]: Deploying daemon osd.5 on smithi164 2024-04-02T17:06:38.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:38 smithi164 ceph-mon[36499]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:38.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:38 smithi154 ceph-mon[36479]: Deploying daemon osd.5 on smithi164 2024-04-02T17:06:38.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:38 smithi154 ceph-mon[36479]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:40.721 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:40 smithi164 ceph-mon[36499]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:40.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:40 smithi059 ceph-mon[31664]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:40 smithi154 ceph-mon[36479]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:42.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:42 smithi059 ceph-mon[31664]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:42 smithi164 ceph-mon[36499]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:42.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:42 smithi154 ceph-mon[36479]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:43.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:43.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:43.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:44.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:44.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:44.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:44.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:44 smithi164 ceph-mon[36499]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:44.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:44 smithi154 ceph-mon[36479]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:45.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:44 smithi059 ceph-mon[31664]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:46.382 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2024-04-02T17:06:46.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.381+0000 7f2f18ff9700 1 -- 172.21.15.164:0/1503631456 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f1c062a30 con 0x7f2f040a3c40 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 msgr2=0x7f2f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 0x7f2f040a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f100098e0 tx=0x7f2f10005fb0 comp rx=0 tx=0).stop 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 msgr2=0x7f2f1c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00e980 tx=0x7f2f0c00ec90 comp rx=0 tx=0).stop 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 shutdown_connections 2024-04-02T17:06:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f1c1041c0 0x7f2f1c19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f1c102620 0x7f2f1c198890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f040a3c40 0x7f2f040a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 --2- 172.21.15.164:0/1503631456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1c103820 0x7f2f1c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 >> 172.21.15.164:0/1503631456 conn(0x7f2f1c0fdc00 msgr2=0x7f2f1c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 shutdown_connections 2024-04-02T17:06:46.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:06:46.383+0000 7f2f21e5d700 1 -- 172.21.15.164:0/1503631456 wait complete. 2024-04-02T17:06:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:46 smithi164 ceph-mon[36499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:46.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:46 smithi059 ceph-mon[31664]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:46.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:46 smithi154 ceph-mon[36479]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T17:06:47.047 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T17:06:47.049 INFO:tasks.cephadm:Deploying osd.6 on smithi059 with /dev/vg_nvme/lv_2... 2024-04-02T17:06:47.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T17:06:47.526 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:48 smithi154 ceph-mon[36479]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:48.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T17:06:48.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T17:06:48.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:48.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:48 smithi059 ceph-mon[31664]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: from='osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:06:48.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:48 smithi164 ceph-mon[36499]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: from='osd.5 ' entity='osd.5' 2024-04-02T17:06:49.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.573 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:06:49.291+0000 7fec4c23a700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T17:06:49.573 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:06:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:06:49.460+0000 7fec43824700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:06:49.600 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:06:49.600 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T17:06:49.600 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.600 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T17:06:49.601 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.601 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: from='osd.5 ' entity='osd.5' 2024-04-02T17:06:49.601 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: from='osd.5 ' entity='osd.5' 2024-04-02T17:06:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T17:06:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T17:06:50.192 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-04-02T17:06:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-04-02T17:06:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368093 s, 285 MB/s 2024-04-02T17:06:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi164 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 8540M 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:50.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:50.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] boot 2024-04-02T17:06:50.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T17:06:50.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi164 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 8540M 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] boot 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T17:06:50.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:50.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi164 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 8540M 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: osd.5 [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] boot 2024-04-02T17:06:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T17:06:50.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T17:06:51.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-04-02T17:06:51.520 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:06:52.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:52 smithi059 ceph-mon[31664]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T17:06:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:52 smithi164 ceph-mon[36499]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T17:06:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:52 smithi154 ceph-mon[36479]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T17:06:53.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.429+0000 7f78d2a00700 1 -- 172.21.15.59:0/1980456593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc076620 msgr2=0x7f78cc074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.429+0000 7f78d2a00700 1 --2- 172.21.15.59:0/1980456593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc076620 0x7f78cc074a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc009ca0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 -- 172.21.15.59:0/1980456593 shutdown_connections 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 --2- 172.21.15.59:0/1980456593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc10a500 0x7f78cc10c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 --2- 172.21.15.59:0/1980456593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc076620 0x7f78cc074a80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 --2- 172.21.15.59:0/1980456593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc0754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 -- 172.21.15.59:0/1980456593 >> 172.21.15.59:0/1980456593 conn(0x7f78cc0fdb90 msgr2=0x7f78cc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 -- 172.21.15.59:0/1980456593 shutdown_connections 2024-04-02T17:06:53.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.430+0000 7f78d2a00700 1 -- 172.21.15.59:0/1980456593 wait complete. 2024-04-02T17:06:53.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.431+0000 7f78d2a00700 1 Processor -- start 2024-04-02T17:06:53.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 -- start start 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 0x7f78cc19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc10a500 0x7f78cc1a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc110440 con 0x7f78cc075050 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1102c0 con 0x7f78cc10a500 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78d2a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1105c0 con 0x7f78cc076620 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78cbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78cbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35806/0 (socket says 172.21.15.59:35806) 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78cbe00700 1 -- 172.21.15.59:0/3250025074 learned_addr learned my addr 172.21.15.59:0/3250025074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:06:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.432+0000 7f78c3400700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 0x7f78cc19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78d0a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc10a500 0x7f78cc1a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 -- 172.21.15.59:0/3250025074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc10a500 msgr2=0x7f78cc1a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc10a500 0x7f78cc1a2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 -- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 msgr2=0x7f78cc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 -- 172.21.15.59:0/3250025074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a26f0 con 0x7f78cc076620 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78cbe00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c3400700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 0x7f78cc19b020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e9f0 tx=0x7f78b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.433+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b400cbd0 con 0x7f78cc076620 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.434+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4004500 con 0x7f78cc076620 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.434+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a29e0 con 0x7f78cc076620 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.434+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4010430 con 0x7f78cc076620 2024-04-02T17:06:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.434+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a2f70 con 0x7f78cc076620 2024-04-02T17:06:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.435+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78cc0692b0 con 0x7f78cc076620 2024-04-02T17:06:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.437+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78b4003e90 con 0x7f78cc076620 2024-04-02T17:06:53.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.438+0000 7f78c9600700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 0x7f78b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:06:53.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.438+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f78b40cf9a0 con 0x7f78cc076620 2024-04-02T17:06:53.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.438+0000 7f78cbe00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 0x7f78b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:06:53.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.439+0000 7f78cbe00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 0x7f78b00a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00b2e0 tx=0x7f78bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:06:53.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.441+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f78b409d1a0 con 0x7f78cc076620 2024-04-02T17:06:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:53 smithi059 ceph-mon[31664]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:53 smithi059 ceph-mon[31664]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T17:06:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:53 smithi164 ceph-mon[36499]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:53.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:53 smithi164 ceph-mon[36499]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T17:06:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:53 smithi154 ceph-mon[36479]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:53 smithi154 ceph-mon[36479]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T17:06:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:06:53.686+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f78cc062a30 con 0x7f78b00a3cc0 2024-04-02T17:06:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T17:06:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:54.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:54.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:54.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:54.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:54.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:54 smithi164 ceph-mon[36499]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:54 smithi154 ceph-mon[36479]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T17:06:54.660 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T17:06:54.660 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:54.660 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:06:54.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:06:54.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:06:54.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:06:54.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:54 smithi059 ceph-mon[31664]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T17:06:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:56 smithi059 ceph-mon[31664]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:56.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:56 smithi164 ceph-mon[36499]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:56.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:56 smithi154 ceph-mon[36479]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]: dispatch 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]': finished 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:06:58.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:06:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3182138856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]: dispatch 2024-04-02T17:06:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]': finished 2024-04-02T17:06:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T17:06:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:06:58.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:06:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3182138856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:06:58.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:06:58.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]: dispatch 2024-04-02T17:06:58.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/633217100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005113f3-2b16-467c-be8c-a79457eaab2c"}]': finished 2024-04-02T17:06:58.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T17:06:58.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:06:58.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:06:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3182138856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:00.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:00 smithi154 ceph-mon[36479]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-02T17:07:00.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:00 smithi059 ceph-mon[31664]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-02T17:07:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:00 smithi164 ceph-mon[36499]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-02T17:07:02.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:02 smithi059 ceph-mon[31664]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T17:07:02.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:02 smithi164 ceph-mon[36499]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T17:07:02.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:02 smithi154 ceph-mon[36479]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T17:07:04.578 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:04 smithi059 ceph-mon[31664]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-02T17:07:04.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:04 smithi164 ceph-mon[36499]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-02T17:07:04.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:04 smithi154 ceph-mon[36479]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-02T17:07:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:06 smithi059 ceph-mon[31664]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:07:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:06 smithi164 ceph-mon[36499]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:07:06.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:06 smithi154 ceph-mon[36479]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:07:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T17:07:07.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:07.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:07 smithi059 ceph-mon[31664]: Deploying daemon osd.6 on smithi059 2024-04-02T17:07:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T17:07:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:07 smithi164 ceph-mon[36499]: Deploying daemon osd.6 on smithi059 2024-04-02T17:07:07.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T17:07:07.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:07.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:07 smithi154 ceph-mon[36479]: Deploying daemon osd.6 on smithi059 2024-04-02T17:07:08.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:08 smithi059 ceph-mon[31664]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:08.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:08 smithi164 ceph-mon[36499]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:08.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:08 smithi154 ceph-mon[36479]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:10.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:10 smithi059 ceph-mon[31664]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:10.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:10 smithi164 ceph-mon[36499]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:10.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:10 smithi154 ceph-mon[36479]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:12.784 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:12 smithi059 ceph-mon[31664]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:12.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:12 smithi164 ceph-mon[36499]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:12.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:12 smithi154 ceph-mon[36479]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:14.701 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:14 smithi059 ceph-mon[31664]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:14 smithi164 ceph-mon[36499]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:14.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:14 smithi154 ceph-mon[36479]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:16.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:16 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:16 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:16 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:16 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:16 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:16 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:17 smithi059 ceph-mon[31664]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:17 smithi164 ceph-mon[36499]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:17 smithi154 ceph-mon[36479]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:18.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:18 smithi059 ceph-mon[31664]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:18 smithi154 ceph-mon[36479]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:18.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:18 smithi164 ceph-mon[36499]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:19 smithi154 ceph-mon[36479]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T17:07:20.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:20.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:20.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:19 smithi059 ceph-mon[31664]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T17:07:20.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:20.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:20.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:20.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:19 smithi164 ceph-mon[36499]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T17:07:20.283 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 6 on host 'smithi059' 2024-04-02T17:07:20.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.282+0000 7f78c9600700 1 -- 172.21.15.59:0/3250025074 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78cc062a30 con 0x7f78b00a3cc0 2024-04-02T17:07:20.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.286+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 msgr2=0x7f78b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:20.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.286+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 0x7f78b00a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00b2e0 tx=0x7f78bc005fb0 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 msgr2=0x7f78cc19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 0x7f78cc19b020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e9f0 tx=0x7f78b400ed00 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 shutdown_connections 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78cc10a500 0x7f78cc1a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f78b00a3cc0 0x7f78b00a6180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78cc075050 0x7f78cc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 --2- 172.21.15.59:0/3250025074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78cc076620 0x7f78cc19b020 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:20.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 >> 172.21.15.59:0/3250025074 conn(0x7f78cc0fdb90 msgr2=0x7f78cc106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:07:20.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 shutdown_connections 2024-04-02T17:07:20.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:07:20.287+0000 7f78d2a00700 1 -- 172.21.15.59:0/3250025074 wait complete. 2024-04-02T17:07:21.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:21.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T17:07:21.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T17:07:21.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:07:21.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:21.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:21.047 DEBUG:teuthology.orchestra.run.smithi059:osd.6> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.6.service 2024-04-02T17:07:21.053 INFO:tasks.cephadm:Deploying osd.7 on smithi154 with /dev/vg_nvme/lv_2... 2024-04-02T17:07:21.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T17:07:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T17:07:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T17:07:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:07:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:21.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:21.081 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:21.291 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:07:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-mon[31664]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:07:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-mon[31664]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T17:07:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:21.799 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:07:21.747+0000 7fde75200700 -1 osd.6 0 waiting for initial osdmap 2024-04-02T17:07:21.799 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:07:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:07:21.785+0000 7fde6b000700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:07:22.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:21 smithi164 ceph-mon[36499]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:07:22.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:21 smithi164 ceph-mon[36499]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T17:07:22.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:21 smithi154 ceph-mon[36479]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:07:22.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:21 smithi154 ceph-mon[36479]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T17:07:22.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:22 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:07:22.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:22 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:07:22.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:22 smithi059 ceph-mon[31664]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:22.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:22 smithi059 ceph-mon[31664]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' 2024-04-02T17:07:22.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:22 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:07:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:22 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:07:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:22 smithi154 ceph-mon[36479]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:22 smithi154 ceph-mon[36479]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' 2024-04-02T17:07:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:22 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:07:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:22 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:07:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:22 smithi164 ceph-mon[36499]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:23.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:22 smithi164 ceph-mon[36499]: from='osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867]' entity='osd.6' 2024-04-02T17:07:23.235 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T17:07:23.235 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T17:07:23.235 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-02T17:07:23.235 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-02T17:07:23.236 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120331 s, 871 MB/s 2024-04-02T17:07:23.236 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-02T17:07:23.775 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:23.775 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:23 smithi154 ceph-mon[36479]: osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] boot 2024-04-02T17:07:23.775 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:23 smithi154 ceph-mon[36479]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T17:07:23.775 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:23.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:23 smithi059 ceph-mon[31664]: osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] boot 2024-04-02T17:07:23.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:23 smithi059 ceph-mon[31664]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T17:07:23.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:23.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-04-02T17:07:24.076 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:24.076 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:23 smithi164 ceph-mon[36499]: osd.6 [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] boot 2024-04-02T17:07:24.076 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:23 smithi164 ceph-mon[36499]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T17:07:24.077 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T17:07:24.083 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:07:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:24 smithi059 ceph-mon[31664]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:25.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:24 smithi059 ceph-mon[31664]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T17:07:25.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:24 smithi164 ceph-mon[36499]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:25.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:24 smithi164 ceph-mon[36499]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T17:07:25.139 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:24 smithi154 ceph-mon[36479]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-02T17:07:25.140 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:24 smithi154 ceph-mon[36479]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 -- 172.21.15.154:0/1604524650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc102710 msgr2=0x7f09bc102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 --2- 172.21.15.154:0/1604524650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc102710 0x7f09bc102b70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f09b00099c0 tx=0x7f09b0009cd0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 -- 172.21.15.154:0/1604524650 shutdown_connections 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 --2- 172.21.15.154:0/1604524650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc1030b0 0x7f09bc109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 --2- 172.21.15.154:0/1604524650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc101510 0x7f09bc101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 --2- 172.21.15.154:0/1604524650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc102710 0x7f09bc102b70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 -- 172.21.15.154:0/1604524650 >> 172.21.15.154:0/1604524650 conn(0x7f09bc0fcaf0 msgr2=0x7f09bc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:07:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 -- 172.21.15.154:0/1604524650 shutdown_connections 2024-04-02T17:07:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.446+0000 7f09c19a4700 1 -- 172.21.15.154:0/1604524650 wait complete. 2024-04-02T17:07:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 Processor -- start 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 -- start start 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc101510 0x7f09bc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc102710 0x7f09bc198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10d1f0 con 0x7f09bc102710 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10d070 con 0x7f09bc1030b0 2024-04-02T17:07:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09c19a4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc10d370 con 0x7f09bc101510 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51260/0 (socket says 172.21.15.154:51260) 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 -- 172.21.15.154:0/3029792473 learned_addr learned my addr 172.21.15.154:0/3029792473 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09ba7fc700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc102710 0x7f09bc198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09baffd700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc101510 0x7f09bc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 -- 172.21.15.154:0/3029792473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc101510 msgr2=0x7f09bc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc101510 0x7f09bc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 -- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc102710 msgr2=0x7f09bc198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc102710 0x7f09bc198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.447+0000 7f09bb7fe700 1 -- 172.21.15.154:0/3029792473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc1a0600 con 0x7f09bc1030b0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09bb7fe700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b970 tx=0x7f09b400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4019070 con 0x7f09bc1030b0 2024-04-02T17:07:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4014440 con 0x7f09bc1030b0 2024-04-02T17:07:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc1a08f0 con 0x7f09bc1030b0 2024-04-02T17:07:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b40075d0 con 0x7f09bc1030b0 2024-04-02T17:07:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.448+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc1a0f00 con 0x7f09bc1030b0 2024-04-02T17:07:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.449+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09bc0692b0 con 0x7f09bc1030b0 2024-04-02T17:07:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.450+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09b401c070 con 0x7f09bc1030b0 2024-04-02T17:07:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.450+0000 7f09c09a2700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 0x7f09a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.451+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f09b40cea20 con 0x7f09bc1030b0 2024-04-02T17:07:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.451+0000 7f09baffd700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 0x7f09a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.451+0000 7f09baffd700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 0x7f09a40a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09ac006fd0 tx=0x7f09ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:07:25.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.455+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f09b409c110 con 0x7f09bc1030b0 2024-04-02T17:07:25.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:25.663+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f09bc06b440 con 0x7f09a40a3cc0 2024-04-02T17:07:26.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:26.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:26.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:26.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:26.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:26.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:26.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:26.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:26.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:26 smithi154 ceph-mon[36479]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:26.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:26 smithi154 ceph-mon[36479]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:27.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:26 smithi059 ceph-mon[31664]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:27.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:26 smithi059 ceph-mon[31664]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:27.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:26 smithi164 ceph-mon[36499]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:27.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:26 smithi164 ceph-mon[36499]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi059 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 5698M 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:28.151 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:28.151 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi059 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 5698M 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:28.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi059 2024-04-02T17:07:28.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 5698M 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:28.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:29.223 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:29.224 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/3677490495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.224 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.224 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]': finished 2024-04-02T17:07:29.224 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T17:07:29.224 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:29.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/3677490495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]': finished 2024-04-02T17:07:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T17:07:29.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/3677490495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]: dispatch 2024-04-02T17:07:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5a328eb-01df-4384-ba8a-ec41925acd17"}]': finished 2024-04-02T17:07:29.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T17:07:29.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:29 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/3093578443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:29 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/3093578443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:29 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/3093578443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:30 smithi059 ceph-mon[31664]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:31.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:30 smithi164 ceph-mon[36499]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:30 smithi154 ceph-mon[36479]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:33.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:32 smithi154 ceph-mon[36479]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:32 smithi059 ceph-mon[31664]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:32 smithi164 ceph-mon[36499]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:35.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:34 smithi059 ceph-mon[31664]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:34 smithi164 ceph-mon[36499]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:34 smithi154 ceph-mon[36479]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:36.256 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T17:07:36.257 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T17:07:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T17:07:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:37.193 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:36 smithi154 ceph-mon[36479]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:37.194 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:36 smithi154 ceph-mon[36479]: Deploying daemon osd.7 on smithi154 2024-04-02T17:07:37.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:36 smithi059 ceph-mon[31664]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:37.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:36 smithi059 ceph-mon[31664]: Deploying daemon osd.7 on smithi154 2024-04-02T17:07:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:36 smithi164 ceph-mon[36499]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:36 smithi164 ceph-mon[36499]: Deploying daemon osd.7 on smithi154 2024-04-02T17:07:39.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:38 smithi154 ceph-mon[36479]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:39.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:38 smithi059 ceph-mon[31664]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:38 smithi164 ceph-mon[36499]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:41.037 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:40 smithi154 ceph-mon[36479]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:40 smithi059 ceph-mon[31664]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:40 smithi164 ceph-mon[36499]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:42.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:42.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:42 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:42.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:42.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:42.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:42 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:42 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:43 smithi059 ceph-mon[31664]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:43.381 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:43 smithi154 ceph-mon[36479]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:43 smithi164 ceph-mon[36499]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:44.692 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 7 on host 'smithi154' 2024-04-02T17:07:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.692+0000 7f09c09a2700 1 -- 172.21.15.154:0/3029792473 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09bc06b440 con 0x7f09a40a3cc0 2024-04-02T17:07:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 msgr2=0x7f09a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 0x7f09a40a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09ac006fd0 tx=0x7f09ac008040 comp rx=0 tx=0).stop 2024-04-02T17:07:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 msgr2=0x7f09bc19ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b970 tx=0x7f09b400bd30 comp rx=0 tx=0).stop 2024-04-02T17:07:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 shutdown_connections 2024-04-02T17:07:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09bc1030b0 0x7f09bc19ff60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09bc101510 0x7f09bc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09a40a3cc0 0x7f09a40a6180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 --2- 172.21.15.154:0/3029792473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09bc102710 0x7f09bc198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 >> 172.21.15.154:0/3029792473 conn(0x7f09bc0fcaf0 msgr2=0x7f09bc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:07:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 shutdown_connections 2024-04-02T17:07:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:07:44.694+0000 7f09c19a4700 1 -- 172.21.15.154:0/3029792473 wait complete. 2024-04-02T17:07:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:45.272 DEBUG:teuthology.orchestra.run.smithi154:osd.7> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.7.service 2024-04-02T17:07:45.274 INFO:tasks.cephadm:Deploying osd.8 on smithi164 with /dev/vg_nvme/lv_2... 2024-04-02T17:07:45.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:45.304 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:45.304 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:45.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:07:45.522 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:07:46.188 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:45 smithi154 ceph-mon[36479]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:46.189 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:45 smithi154 ceph-mon[36479]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:45 smithi059 ceph-mon[31664]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:45 smithi059 ceph-mon[31664]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:46.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:45 smithi164 ceph-mon[36499]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:46.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:45 smithi164 ceph-mon[36499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T17:07:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T17:07:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T17:07:47.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:47.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:46 smithi164 ceph-mon[36499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.268 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:47.268 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T17:07:47.268 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.268 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T17:07:47.269 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:47.269 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-mon[36479]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.269 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:07:46.921+0000 7fdeae0a4700 -1 osd.7 0 waiting for initial osdmap 2024-04-02T17:07:47.269 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:07:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:07:46.935+0000 7fdea9e97700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:07:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T17:07:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: from='osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T17:07:47.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:47.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:46 smithi059 ceph-mon[31664]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:07:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114583 s, 915 MB/s 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 2024-04-02T17:07:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-02T17:07:47.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-04-02T17:07:48.217 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] boot 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T17:07:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:07:48.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] boot 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T17:07:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:48.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:07:48.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:07:48.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: osd.7 [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] boot 2024-04-02T17:07:48.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T17:07:48.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi154 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 5693M 2024-04-02T17:07:49.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:48 smithi164 ceph-mon[36499]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T17:07:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:07:49.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:07:49.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:49.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi154 2024-04-02T17:07:49.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 5693M 2024-04-02T17:07:49.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:48 smithi059 ceph-mon[31664]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T17:07:49.302 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:07:49.302 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:07:49.302 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-02T17:07:49.302 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi154 2024-04-02T17:07:49.302 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 5693M 2024-04-02T17:07:49.303 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:48 smithi154 ceph-mon[36479]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T17:07:49.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1648189983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 msgr2=0x7f3cd0101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1648189983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0101e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009990 tx=0x7f3cc0009ca0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1648189983 shutdown_connections 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1648189983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0103120 0x7f3cd01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1648189983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0101e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1648189983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1648189983 >> 172.21.15.164:0/1648189983 conn(0x7f3cd00fc260 msgr2=0x7f3cd00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1648189983 shutdown_connections 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.773+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1648189983 wait complete. 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 Processor -- start 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 -- start start 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0103120 0x7f3cd019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:49.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd010d130 con 0x7f3cd0103120 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd010cfb0 con 0x7f3cd0102760 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cd4dde700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd010d2b0 con 0x7f3cd0101a40 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cce59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cce59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46700/0 (socket says 172.21.15.164:46700) 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cce59c700 1 -- 172.21.15.164:0/1882323915 learned_addr learned my addr 172.21.15.164:0/1882323915 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 -- 172.21.15.164:0/1882323915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 msgr2=0x7f3cd0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 -- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0103120 msgr2=0x7f3cd019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0103120 0x7f3cd019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cc5d9b700 1 -- 172.21.15.164:0/1882323915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd01a04c0 con 0x7f3cd0102760 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.774+0000 7f3cce59c700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cc5d9b700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0198da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c940 tx=0x7f3cb800cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:07:49.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8004220 con 0x7f3cd0102760 2024-04-02T17:07:49.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd01a07b0 con 0x7f3cd0102760 2024-04-02T17:07:49.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb800ce30 con 0x7f3cd0102760 2024-04-02T17:07:49.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8003ce0 con 0x7f3cd0102760 2024-04-02T17:07:49.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.775+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01a0d70 con 0x7f3cd0102760 2024-04-02T17:07:49.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.776+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd00692b0 con 0x7f3cd0102760 2024-04-02T17:07:49.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.777+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3cb80043c0 con 0x7f3cd0102760 2024-04-02T17:07:49.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.777+0000 7f3cc7fff700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 0x7f3cbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:07:49.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.778+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f3cb80cf550 con 0x7f3cd0102760 2024-04-02T17:07:49.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.778+0000 7f3cce59c700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 0x7f3cbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:07:49.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.779+0000 7f3cce59c700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 0x7f3cbc0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009750 tx=0x7f3cc0003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:07:49.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.781+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3cb8014030 con 0x7f3cd0102760 2024-04-02T17:07:49.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:07:49.994+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3cd00fe410 con 0x7f3cbc0a3c40 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:51.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:51.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:51.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T17:07:51.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:51.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:51.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:51.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:51.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T17:07:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:07:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:07:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:07:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:07:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:51 smithi059 ceph-mon[31664]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T17:07:52.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:51 smithi164 ceph-mon[36499]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T17:07:52.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:51 smithi154 ceph-mon[36479]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T17:07:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: pgmap v196: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/3706617008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]': finished 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T17:07:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: pgmap v196: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/3706617008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]': finished 2024-04-02T17:07:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T17:07:53.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: pgmap v196: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/3706617008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]: dispatch 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77628eb3-b084-412c-9e27-2c9d9bdc3aeb"}]': finished 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T17:07:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:07:54.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:53 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/2972799402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:54.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:53 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/2972799402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:53 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/2972799402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:07:55.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:54 smithi059 ceph-mon[31664]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:54 smithi164 ceph-mon[36499]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:55.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:54 smithi154 ceph-mon[36479]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:57.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:56 smithi164 ceph-mon[36499]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:56 smithi059 ceph-mon[31664]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:56 smithi154 ceph-mon[36479]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:59.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:07:58 smithi059 ceph-mon[31664]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:59.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:07:58 smithi164 ceph-mon[36499]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:07:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:07:58 smithi154 ceph-mon[36479]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:01.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:00 smithi164 ceph-mon[36499]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:01.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T17:08:01.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:01.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:00 smithi164 ceph-mon[36499]: Deploying daemon osd.8 on smithi164 2024-04-02T17:08:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:00 smithi059 ceph-mon[31664]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:01.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T17:08:01.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:01.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:00 smithi059 ceph-mon[31664]: Deploying daemon osd.8 on smithi164 2024-04-02T17:08:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:00 smithi154 ceph-mon[36479]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T17:08:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:00 smithi154 ceph-mon[36479]: Deploying daemon osd.8 on smithi164 2024-04-02T17:08:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:02 smithi059 ceph-mon[31664]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:02 smithi164 ceph-mon[36499]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:02 smithi154 ceph-mon[36479]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:05.051 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:04 smithi164 ceph-mon[36499]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:04 smithi059 ceph-mon[31664]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:05.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:04 smithi154 ceph-mon[36479]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:07.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:06 smithi059 ceph-mon[31664]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:07.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:06 smithi164 ceph-mon[36499]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:07.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:06 smithi154 ceph-mon[36479]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:07.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:07.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:08.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:08 smithi164 ceph-mon[36499]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:09.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:08 smithi059 ceph-mon[31664]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:08 smithi154 ceph-mon[36479]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:09.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.180+0000 7f3cc7fff700 1 -- 172.21.15.164:0/1882323915 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3cd00fe410 con 0x7f3cbc0a3c40 2024-04-02T17:08:09.180 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 8 on host 'smithi164' 2024-04-02T17:08:09.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 msgr2=0x7f3cbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 0x7f3cbc0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009750 tx=0x7f3cc0003960 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 msgr2=0x7f3cd0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0198da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c940 tx=0x7f3cb800cc50 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 shutdown_connections 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cbc0a3c40 0x7f3cbc0a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0103120 0x7f3cd019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd0101a40 0x7f3cd0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 --2- 172.21.15.164:0/1882323915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0102760 0x7f3cd0198da0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 >> 172.21.15.164:0/1882323915 conn(0x7f3cd00fc260 msgr2=0x7f3cd00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 shutdown_connections 2024-04-02T17:08:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:08:09.183+0000 7f3cd4dde700 1 -- 172.21.15.164:0/1882323915 wait complete. 2024-04-02T17:08:09.774 DEBUG:teuthology.orchestra.run.smithi164:osd.8> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.8.service 2024-04-02T17:08:09.776 INFO:tasks.cephadm:Deploying osd.9 on smithi059 with /dev/vg_nvme/lv_1... 2024-04-02T17:08:09.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T17:08:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:10.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:10 smithi164 ceph-mon[36499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:10.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:10.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:10 smithi059 ceph-mon[31664]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:10.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:10.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:10.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:10 smithi154 ceph-mon[36479]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T17:08:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T17:08:11.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T17:08:11.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:11.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-mon[36499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:11.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:11 smithi059 ceph-mon[31664]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T17:08:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: from='osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T17:08:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:11 smithi154 ceph-mon[36479]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:08:11.823 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:08:11.472+0000 7fb8db986700 -1 osd.8 0 waiting for initial osdmap 2024-04-02T17:08:11.823 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:08:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:08:11.483+0000 7fb8d7779700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:08:12.702 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:12 smithi059 ceph-mon[31664]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:12.702 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:12 smithi059 ceph-mon[31664]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:08:12.702 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:12 smithi059 ceph-mon[31664]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T17:08:12.702 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:12.754 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:12 smithi164 ceph-mon[36499]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:12.754 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:12 smithi164 ceph-mon[36499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:08:12.755 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:12 smithi164 ceph-mon[36499]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T17:08:12.755 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:12.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:12 smithi154 ceph-mon[36479]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-02T17:08:12.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:12 smithi154 ceph-mon[36479]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:08:12.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:12 smithi154 ceph-mon[36479]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T17:08:12.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362214 s, 289 MB/s 2024-04-02T17:08:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-04-02T17:08:13.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:08:13.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:08:13.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] boot 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi164 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 5693M 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:13.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:13 smithi059 ceph-mon[31664]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T17:08:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] boot 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi164 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 5693M 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:13 smithi164 ceph-mon[36499]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: osd.8 [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] boot 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi164 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 5693M 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:13.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:13 smithi154 ceph-mon[36479]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T17:08:14.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-04-02T17:08:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:08:14.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:14 smithi059 ceph-mon[31664]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:14 smithi164 ceph-mon[36499]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:14.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:14 smithi154 ceph-mon[36479]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:15 smithi059 ceph-mon[31664]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T17:08:15.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:15 smithi164 ceph-mon[36499]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T17:08:15.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:15 smithi154 ceph-mon[36479]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T17:08:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 -- 172.21.15.59:0/2774544621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 msgr2=0x7f463810ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 --2- 172.21.15.59:0/2774544621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f463810ccc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f463000b0d0 tx=0x7f463000b3e0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 -- 172.21.15.59:0/2774544621 shutdown_connections 2024-04-02T17:08:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 --2- 172.21.15.59:0/2774544621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f463810ccc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 --2- 172.21.15.59:0/2774544621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638104870 0x7f4638104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 --2- 172.21.15.59:0/2774544621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638105a70 0x7f4638105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.643+0000 7f463f200700 1 -- 172.21.15.59:0/2774544621 >> 172.21.15.59:0/2774544621 conn(0x7f46380ffe10 msgr2=0x7f4638102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:08:16.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.644+0000 7f463f200700 1 -- 172.21.15.59:0/2774544621 shutdown_connections 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.644+0000 7f463f200700 1 -- 172.21.15.59:0/2774544621 wait complete. 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.645+0000 7f463f200700 1 Processor -- start 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 -- start start 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 0x7f463819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f46381a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110490 con 0x7f4638105a70 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110310 con 0x7f4638106410 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463f200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4638110610 con 0x7f4638104870 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f4637400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 0x7f463819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f4637e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f4637e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41798/0 (socket says 172.21.15.59:41798) 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f4637e00700 1 -- 172.21.15.59:0/1683607237 learned_addr learned my addr 172.21.15.59:0/1683607237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f4637400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 0x7f463819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55530/0 (socket says 172.21.15.59:55530) 2024-04-02T17:08:16.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.646+0000 7f463d200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f46381a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 -- 172.21.15.59:0/1683607237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 msgr2=0x7f46381a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f46381a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 -- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 msgr2=0x7f463819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 0x7f463819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 -- 172.21.15.59:0/1683607237 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46381a2640 con 0x7f4638104870 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.647+0000 7f4637e00700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f462800e9a0 tx=0x7f462800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:08:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.648+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628004220 con 0x7f4638104870 2024-04-02T17:08:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.648+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46280043c0 con 0x7f4638104870 2024-04-02T17:08:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.648+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46381a2930 con 0x7f4638104870 2024-04-02T17:08:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.648+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628003df0 con 0x7f4638104870 2024-04-02T17:08:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.648+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a2ec0 con 0x7f4638104870 2024-04-02T17:08:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.649+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46380692b0 con 0x7f4638104870 2024-04-02T17:08:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.651+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f462800c810 con 0x7f4638104870 2024-04-02T17:08:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.652+0000 7f4634c00700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 0x7f461c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.653+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f46280cfe00 con 0x7f4638104870 2024-04-02T17:08:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.653+0000 7f4637400700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 0x7f461c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.653+0000 7f4637400700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 0x7f461c0a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4624009c60 tx=0x7f4624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:08:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.655+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f462809d0f0 con 0x7f4638104870 2024-04-02T17:08:16.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:16 smithi059 ceph-mon[31664]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:16.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:16 smithi059 ceph-mon[31664]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T17:08:16.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:16 smithi164 ceph-mon[36499]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:16.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:16 smithi164 ceph-mon[36499]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T17:08:16.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:16.897+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4638062a30 con 0x7f461c0a3cc0 2024-04-02T17:08:16.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:16 smithi154 ceph-mon[36479]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:16.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:16 smithi154 ceph-mon[36479]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T17:08:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:17 smithi059 ceph-mon[31664]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T17:08:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:17 smithi059 ceph-mon[31664]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:17 smithi164 ceph-mon[36499]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T17:08:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:17 smithi164 ceph-mon[36499]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:17.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:17.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:17.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:17 smithi154 ceph-mon[36479]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T17:08:17.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:17 smithi154 ceph-mon[36479]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:17.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:17.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:17.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:18 smithi059 ceph-mon[31664]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:18.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:18 smithi164 ceph-mon[36499]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:18.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:18 smithi154 ceph-mon[36479]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:20.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:20 smithi164 ceph-mon[36499]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-02T17:08:20.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:20 smithi154 ceph-mon[36479]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-02T17:08:21.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:20 smithi059 ceph-mon[31664]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-02T17:08:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:21 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]: dispatch 2024-04-02T17:08:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:21 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]': finished 2024-04-02T17:08:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:21 smithi059 ceph-mon[31664]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T17:08:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:21 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]: dispatch 2024-04-02T17:08:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:21 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]': finished 2024-04-02T17:08:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:21 smithi164 ceph-mon[36499]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T17:08:21.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:21.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:21 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]: dispatch 2024-04-02T17:08:21.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:21 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1751615843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeabc6e-28cb-4639-9ef1-5982c75d2e25"}]': finished 2024-04-02T17:08:21.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:21 smithi154 ceph-mon[36479]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T17:08:21.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:22.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:22 smithi164 ceph-mon[36499]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:08:22.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:22 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2174212235' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:22.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:22 smithi154 ceph-mon[36479]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:08:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:22 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2174212235' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:23.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:22 smithi059 ceph-mon[31664]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T17:08:23.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:22 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2174212235' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:24 smithi164 ceph-mon[36499]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-02T17:08:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:24 smithi154 ceph-mon[36479]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-02T17:08:25.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:24 smithi059 ceph-mon[31664]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-02T17:08:26.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:26 smithi059 ceph-mon[31664]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-02T17:08:26.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:26 smithi164 ceph-mon[36499]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-02T17:08:26.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:26 smithi154 ceph-mon[36479]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-02T17:08:28.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:28 smithi059 ceph-mon[31664]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:08:29.413 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:28 smithi154 ceph-mon[36479]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:08:29.414 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:28 smithi164 ceph-mon[36499]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:08:30.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:30 smithi154 ceph-mon[36479]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:30 smithi059 ceph-mon[31664]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:31.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:30 smithi164 ceph-mon[36499]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:31.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T17:08:31.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:31.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:31 smithi154 ceph-mon[36479]: Deploying daemon osd.9 on smithi059 2024-04-02T17:08:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T17:08:31.902 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:31.902 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:31 smithi164 ceph-mon[36499]: Deploying daemon osd.9 on smithi059 2024-04-02T17:08:31.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T17:08:31.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:31.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:31 smithi059 ceph-mon[31664]: Deploying daemon osd.9 on smithi059 2024-04-02T17:08:32.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:32 smithi059 ceph-mon[31664]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:32.843 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:32 smithi164 ceph-mon[36499]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:32.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:32 smithi154 ceph-mon[36479]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:34.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:34 smithi154 ceph-mon[36479]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:34.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:34 smithi059 ceph-mon[31664]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:35.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:34 smithi164 ceph-mon[36499]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:36.785 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:36 smithi059 ceph-mon[31664]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:36.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:36 smithi154 ceph-mon[36479]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:37.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:36 smithi164 ceph-mon[36499]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:38.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:38 smithi154 ceph-mon[36479]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:39.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:38 smithi059 ceph-mon[31664]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:39.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:38 smithi164 ceph-mon[36499]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:40.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:40.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:40.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:40.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:41.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:40 smithi059 ceph-mon[31664]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:41.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:40 smithi154 ceph-mon[36479]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:40 smithi164 ceph-mon[36499]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:43.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.198+0000 7f4634c00700 1 -- 172.21.15.59:0/1683607237 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4638062a30 con 0x7f461c0a3cc0 2024-04-02T17:08:43.200 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 9 on host 'smithi059' 2024-04-02T17:08:43.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 msgr2=0x7f461c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:43.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 0x7f461c0a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4624009c60 tx=0x7f4624009260 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 msgr2=0x7f463819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f462800e9a0 tx=0x7f462800ecb0 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 shutdown_connections 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4638106410 0x7f46381a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4638104870 0x7f463819a9e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f461c0a3cc0 0x7f461c0a6180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 --2- 172.21.15.59:0/1683607237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4638105a70 0x7f463819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 >> 172.21.15.59:0/1683607237 conn(0x7f46380ffe10 msgr2=0x7f4638108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 shutdown_connections 2024-04-02T17:08:43.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:08:43.202+0000 7f463f200700 1 -- 172.21.15.59:0/1683607237 wait complete. 2024-04-02T17:08:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T17:08:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T17:08:43.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:44.008 DEBUG:teuthology.orchestra.run.smithi059:osd.9> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.9.service 2024-04-02T17:08:44.014 INFO:tasks.cephadm:Deploying osd.10 on smithi154 with /dev/vg_nvme/lv_1... 2024-04-02T17:08:44.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T17:08:44.265 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:08:44.389 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:44 smithi154 ceph-mon[36479]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T17:08:44.389 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:44 smithi154 ceph-mon[36479]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T17:08:44.389 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:44 smithi154 ceph-mon[36479]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:08:44.389 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:44 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:44.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:44 smithi059 ceph-mon[31664]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T17:08:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:44 smithi059 ceph-mon[31664]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T17:08:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:44 smithi059 ceph-mon[31664]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:08:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:44 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:44 smithi164 ceph-mon[36499]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T17:08:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:44 smithi164 ceph-mon[36499]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T17:08:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:44 smithi164 ceph-mon[36499]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-04-02T17:08:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:44 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-mon[31664]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-mon[31664]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:08:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-mon[31664]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T17:08:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.550 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:45 smithi154 ceph-mon[36479]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:45.550 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:45 smithi154 ceph-mon[36479]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:08:45.550 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:45 smithi154 ceph-mon[36479]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T17:08:45.550 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:45 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.550 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:45 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:45 smithi164 ceph-mon[36499]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:45 smithi164 ceph-mon[36499]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-04-02T17:08:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:45 smithi164 ceph-mon[36499]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T17:08:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:45 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:45 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:45.707 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:08:45.325+0000 7f163bc00700 -1 osd.9 0 waiting for initial osdmap 2024-04-02T17:08:45.707 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:08:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:08:45.382+0000 7f1631a00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011421 s, 918 MB/s 2024-04-02T17:08:46.233 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-02T17:08:46.346 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:46 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:08:46.346 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:46 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:08:46.346 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:46 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:46.347 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:46 smithi059 ceph-mon[31664]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' 2024-04-02T17:08:46.544 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:46 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:08:46.544 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:46 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:08:46.544 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:46 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:46.544 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:46 smithi154 ceph-mon[36479]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' 2024-04-02T17:08:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:46 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:08:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:46 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:08:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:46 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:46 smithi164 ceph-mon[36499]: from='osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556]' entity='osd.9' 2024-04-02T17:08:46.850 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-04-02T17:08:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:08:47.470 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:47 smithi154 ceph-mon[36479]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:47.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:47.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:47 smithi154 ceph-mon[36479]: osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] boot 2024-04-02T17:08:47.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:47 smithi154 ceph-mon[36479]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T17:08:47.471 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:47 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:47.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:47 smithi059 ceph-mon[31664]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:47.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:47.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:47 smithi059 ceph-mon[31664]: osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] boot 2024-04-02T17:08:47.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:47 smithi059 ceph-mon[31664]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T17:08:47.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:47 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:47 smithi164 ceph-mon[36499]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-02T17:08:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:47 smithi164 ceph-mon[36499]: osd.9 [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] boot 2024-04-02T17:08:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:47 smithi164 ceph-mon[36499]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T17:08:47.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:47 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T17:08:48.461 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:48 smithi154 ceph-mon[36479]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T17:08:48.461 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:48 smithi154 ceph-mon[36479]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:48.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:48 smithi059 ceph-mon[31664]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T17:08:48.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:48 smithi059 ceph-mon[31664]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 -- 172.21.15.154:0/1757656948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 msgr2=0x7fbe2c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 --2- 172.21.15.154:0/1757656948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c100d50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009990 tx=0x7fbe1c009ca0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 -- 172.21.15.154:0/1757656948 shutdown_connections 2024-04-02T17:08:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 --2- 172.21.15.154:0/1757656948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c1042b0 0x7fbe2c10ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 --2- 172.21.15.154:0/1757656948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c100d50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 --2- 172.21.15.154:0/1757656948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.668+0000 7fbe324a4700 1 -- 172.21.15.154:0/1757656948 >> 172.21.15.154:0/1757656948 conn(0x7fbe2c0fc290 msgr2=0x7fbe2c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- 172.21.15.154:0/1757656948 shutdown_connections 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- 172.21.15.154:0/1757656948 wait complete. 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 Processor -- start 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- start start 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c1042b0 0x7fbe2c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c10e2c0 con 0x7fbe2c1042b0 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c10e140 con 0x7fbe2c100930 2024-04-02T17:08:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.669+0000 7fbe324a4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c10e440 con 0x7fbe2c103910 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47144/0 (socket says 172.21.15.154:47144) 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 -- 172.21.15.154:0/3670821867 learned_addr learned my addr 172.21.15.154:0/3670821867 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2b7fe700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 -- 172.21.15.154:0/3670821867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 msgr2=0x7fbe2c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 -- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c1042b0 msgr2=0x7fbe2c19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c1042b0 0x7fbe2c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 -- 172.21.15.154:0/3670821867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1a0540 con 0x7fbe2c100930 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2b7fe700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe2bfff700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c027070 con 0x7fbe2c100930 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c1a07d0 con 0x7fbe2c100930 2024-04-02T17:08:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1c022470 con 0x7fbe2c100930 2024-04-02T17:08:48.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00f6c0 con 0x7fbe2c100930 2024-04-02T17:08:48.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.670+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c1a0ca0 con 0x7fbe2c100930 2024-04-02T17:08:48.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.671+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c0692b0 con 0x7fbe2c100930 2024-04-02T17:08:48.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.672+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbe1c02a070 con 0x7fbe2c100930 2024-04-02T17:08:48.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.673+0000 7fbe297fa700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 0x7fbe140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:08:48.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.673+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fbe1c0d0b10 con 0x7fbe2c100930 2024-04-02T17:08:48.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.673+0000 7fbe2b7fe700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 0x7fbe140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:08:48.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.674+0000 7fbe2b7fe700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 0x7fbe140a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe20005fd0 tx=0x7fbe20005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:08:48.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.677+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbe1c013070 con 0x7fbe2c100930 2024-04-02T17:08:48.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:48 smithi164 ceph-mon[36499]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T17:08:48.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:48 smithi164 ceph-mon[36499]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:48.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:08:48.886+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbe2c062a30 con 0x7fbe140a3c40 2024-04-02T17:08:49.627 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi059 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 4273M 2024-04-02T17:08:49.628 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.629 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.629 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:49.629 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:49 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi059 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 4273M 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:49.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:49 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:08:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:08:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi059 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 4273M 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:49.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:49 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:50.631 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:50 smithi154 ceph-mon[36479]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:50.632 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:50.632 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:50.632 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:50.632 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:50.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:50 smithi059 ceph-mon[31664]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:50.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:50.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:50.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:50.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:50.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:50 smithi164 ceph-mon[36499]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:50.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:08:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:08:50.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/3268665860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]': finished 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:08:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:52 smithi059 ceph-mon[31664]: from='client.? 172.21.15.154:0/3012479089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:52.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:52.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/3268665860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]': finished 2024-04-02T17:08:52.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T17:08:52.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:08:52.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:52 smithi164 ceph-mon[36499]: from='client.? 172.21.15.154:0/3012479089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/3268665860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]: dispatch 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7079db2d-8f1f-4aea-b0dd-b7171210f38d"}]': finished 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:08:52.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:52 smithi154 ceph-mon[36479]: from='client.? 172.21.15.154:0/3012479089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:08:54.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:54 smithi059 ceph-mon[31664]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:54.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:54 smithi164 ceph-mon[36499]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:54.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:54 smithi154 ceph-mon[36479]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:56.791 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:56 smithi154 ceph-mon[36479]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:56.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:56 smithi059 ceph-mon[31664]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:56.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:56 smithi164 ceph-mon[36499]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:58.794 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:58 smithi154 ceph-mon[36479]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:58.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:58 smithi059 ceph-mon[31664]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:58 smithi164 ceph-mon[36499]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:08:59.806 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T17:08:59.806 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:59.806 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:08:59 smithi154 ceph-mon[36479]: Deploying daemon osd.10 on smithi154 2024-04-02T17:08:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T17:08:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:08:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:08:59 smithi164 ceph-mon[36499]: Deploying daemon osd.10 on smithi154 2024-04-02T17:09:00.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T17:09:00.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:00.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:08:59 smithi059 ceph-mon[31664]: Deploying daemon osd.10 on smithi154 2024-04-02T17:09:00.821 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:00 smithi154 ceph-mon[36479]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:00 smithi164 ceph-mon[36499]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:01.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:00 smithi059 ceph-mon[31664]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:02.737 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:02 smithi154 ceph-mon[36479]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:02.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:02 smithi164 ceph-mon[36499]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:03.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:02 smithi059 ceph-mon[31664]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:04.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:04 smithi164 ceph-mon[36499]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:04.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:04.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:04 smithi059 ceph-mon[31664]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:04.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:04.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:04 smithi154 ceph-mon[36479]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:04.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:04.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:06 smithi059 ceph-mon[31664]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:06 smithi164 ceph-mon[36499]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:06.829 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:06 smithi154 ceph-mon[36479]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:07.969 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 10 on host 'smithi154' 2024-04-02T17:09:07.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.969+0000 7fbe297fa700 1 -- 172.21.15.154:0/3670821867 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbe2c062a30 con 0x7fbe140a3c40 2024-04-02T17:09:07.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.971+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 msgr2=0x7fbe140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:07.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.971+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 0x7fbe140a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe20005fd0 tx=0x7fbe20005dc0 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.971+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 msgr2=0x7fbe2c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.971+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c009f90 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 shutdown_connections 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbe140a3c40 0x7fbe140a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c1042b0 0x7fbe2c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe2c100930 0x7fbe2c1988e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 --2- 172.21.15.154:0/3670821867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c103910 0x7fbe2c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 >> 172.21.15.154:0/3670821867 conn(0x7fbe2c0fc290 msgr2=0x7fbe2c106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 shutdown_connections 2024-04-02T17:09:07.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-02T17:09:07.972+0000 7fbe324a4700 1 -- 172.21.15.154:0/3670821867 wait complete. 2024-04-02T17:09:08.526 DEBUG:teuthology.orchestra.run.smithi154:osd.10> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.10.service 2024-04-02T17:09:08.529 INFO:tasks.cephadm:Deploying osd.11 on smithi164 with /dev/vg_nvme/lv_1... 2024-04-02T17:09:08.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T17:09:08.781 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.294 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:10.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:09 smithi154 ceph-mon[36479]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T17:09:10.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:09 smithi154 ceph-mon[36479]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:09 smithi154 ceph-mon[36479]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T17:09:10.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:09 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:10.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:09 smithi154 ceph-mon[36479]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.199 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:09 smithi164 ceph-mon[36499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T17:09:10.199 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:09 smithi164 ceph-mon[36499]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.199 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:09 smithi164 ceph-mon[36499]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T17:09:10.200 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:09 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:10.200 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:09 smithi164 ceph-mon[36499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:09 smithi059 ceph-mon[31664]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T17:09:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:09 smithi059 ceph-mon[31664]: from='osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:09 smithi059 ceph-mon[31664]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T17:09:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:09 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:10.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:09 smithi059 ceph-mon[31664]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-02T17:09:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T17:09:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T17:09:10.757 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-02T17:09:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-02T17:09:10.757 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112562 s, 932 MB/s 2024-04-02T17:09:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-02T17:09:11.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-mon[36479]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:11.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-mon[36479]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:09:11.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-mon[36479]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T17:09:11.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:11.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-mon[36479]: from='osd.10 ' entity='osd.10' 2024-04-02T17:09:11.150 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:09:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:09:10.866+0000 7f6ab13fd700 -1 osd.10 0 waiting for initial osdmap 2024-04-02T17:09:11.151 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:09:11.048+0000 7f6aa91e8700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:09:11.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:10 smithi164 ceph-mon[36499]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:11.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:10 smithi164 ceph-mon[36499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:09:11.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:10 smithi164 ceph-mon[36499]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T17:09:11.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:11.264 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:10 smithi164 ceph-mon[36499]: from='osd.10 ' entity='osd.10' 2024-04-02T17:09:11.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-04-02T17:09:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:10 smithi059 ceph-mon[31664]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:10 smithi059 ceph-mon[31664]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-02T17:09:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:10 smithi059 ceph-mon[31664]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T17:09:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:10 smithi059 ceph-mon[31664]: from='osd.10 ' entity='osd.10' 2024-04-02T17:09:11.530 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:09:12.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:09:12.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:09:12.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:12.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-mon[36479]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T17:09:12.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:12.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:11 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:09:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:11 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:09:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:11 smithi059 ceph-mon[31664]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T17:09:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:11 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:09:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:11 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:09:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:11 smithi164 ceph-mon[36499]: osdmap e73: 11 total, 10 up, 11 in 2024-04-02T17:09:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 -- 172.21.15.164:0/2517593430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e41017f0 msgr2=0x7f06e4101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 --2- 172.21.15.164:0/2517593430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e41017f0 0x7f06e4101c70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009990 tx=0x7f06d4009ca0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 -- 172.21.15.164:0/2517593430 shutdown_connections 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 --2- 172.21.15.164:0/2517593430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41021b0 0x7f06e4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 --2- 172.21.15.164:0/2517593430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4100e90 0x7f06e41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 --2- 172.21.15.164:0/2517593430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e41017f0 0x7f06e4101c70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 -- 172.21.15.164:0/2517593430 >> 172.21.15.164:0/2517593430 conn(0x7f06e40fcaf0 msgr2=0x7f06e40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 -- 172.21.15.164:0/2517593430 shutdown_connections 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 -- 172.21.15.164:0/2517593430 wait complete. 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.136+0000 7f06eb48b700 1 Processor -- start 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 -- start start 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41017f0 0x7f06e419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e41021b0 0x7f06e41a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4110450 con 0x7f06e41021b0 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41102d0 con 0x7f06e41017f0 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41105d0 con 0x7f06e4100e90 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52866/0 (socket says 172.21.15.164:52866) 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 -- 172.21.15.164:0/1115811659 learned_addr learned my addr 172.21.15.164:0/1115811659 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:09:13.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e8a26700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41017f0 0x7f06e419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 -- 172.21.15.164:0/1115811659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41017f0 msgr2=0x7f06e419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41017f0 0x7f06e419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 -- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e41021b0 msgr2=0x7f06e41a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e41021b0 0x7f06e41a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 -- 172.21.15.164:0/1115811659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e41a2700 con 0x7f06e4100e90 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06e9227700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b560 tx=0x7f06e000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0015020 con 0x7f06e4100e90 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e41a29f0 con 0x7f06e4100e90 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e0011d10 con 0x7f06e4100e90 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.137+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000f2e0 con 0x7f06e4100e90 2024-04-02T17:09:13.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.138+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41a2f30 con 0x7f06e4100e90 2024-04-02T17:09:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.139+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e40692b0 con 0x7f06e4100e90 2024-04-02T17:09:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.140+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06e0018020 con 0x7f06e4100e90 2024-04-02T17:09:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.140+0000 7f06da7fc700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 0x7f06d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.141+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f06e00cfa50 con 0x7f06e4100e90 2024-04-02T17:09:13.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.141+0000 7f06e8a26700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 0x7f06d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:13.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.142+0000 7f06e8a26700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 0x7f06d00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b2e0 tx=0x7f06d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:13.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.144+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f06e001f020 con 0x7f06e4100e90 2024-04-02T17:09:13.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:13.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] boot 2024-04-02T17:09:13.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi154 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 4270M 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:13.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.301 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] boot 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi154 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 4270M 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:13.351+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f06e4062a30 con 0x7f06d00a3c40 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: osd.10 [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] boot 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi154 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 4270M 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:13.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:14.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:13 smithi164 ceph-mon[36499]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T17:09:14.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:09:14.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:09:14.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:14.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:13 smithi059 ceph-mon[31664]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T17:09:14.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:09:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:09:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:13 smithi154 ceph-mon[36479]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T17:09:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T17:09:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T17:09:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:15.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:14 smithi164 ceph-mon[36499]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:15.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:14 smithi164 ceph-mon[36499]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:15.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:14 smithi164 ceph-mon[36499]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T17:09:15.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:14 smithi164 ceph-mon[36499]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T17:09:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:14 smithi059 ceph-mon[31664]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:15.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:14 smithi059 ceph-mon[31664]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:15.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:14 smithi059 ceph-mon[31664]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T17:09:15.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:14 smithi059 ceph-mon[31664]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T17:09:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:14 smithi154 ceph-mon[36479]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:14 smithi154 ceph-mon[36479]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:14 smithi154 ceph-mon[36479]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T17:09:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:14 smithi154 ceph-mon[36479]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T17:09:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T17:09:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/2785925406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]': finished 2024-04-02T17:09:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T17:09:16.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:16 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/2785925406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]': finished 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T17:09:16.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:16 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:16.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: osdmap e78: 11 total, 11 up, 11 in 2024-04-02T17:09:16.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:16.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/2785925406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]: dispatch 2024-04-02T17:09:16.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4"}]': finished 2024-04-02T17:09:16.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T17:09:16.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:16 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:17.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:17 smithi154 ceph-mon[36479]: from='client.? 172.21.15.164:0/4291422045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:09:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:17 smithi059 ceph-mon[31664]: from='client.? 172.21.15.164:0/4291422045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:09:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:17 smithi164 ceph-mon[36499]: from='client.? 172.21.15.164:0/4291422045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T17:09:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:18 smithi154 ceph-mon[36479]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:18 smithi059 ceph-mon[31664]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:18.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:18 smithi164 ceph-mon[36499]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:20.656 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:20 smithi164 ceph-mon[36499]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-02T17:09:20.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:20 smithi059 ceph-mon[31664]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-02T17:09:20.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:20 smithi154 ceph-mon[36479]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-02T17:09:22.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:22 smithi059 ceph-mon[31664]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:09:22.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:22 smithi164 ceph-mon[36499]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:09:22.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:22 smithi154 ceph-mon[36479]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T17:09:24.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:24 smithi059 ceph-mon[31664]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:09:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T17:09:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:24 smithi059 ceph-mon[31664]: Deploying daemon osd.11 on smithi164 2024-04-02T17:09:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:24 smithi164 ceph-mon[36499]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:09:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T17:09:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:24 smithi164 ceph-mon[36499]: Deploying daemon osd.11 on smithi164 2024-04-02T17:09:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:24 smithi154 ceph-mon[36479]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T17:09:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T17:09:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:24 smithi154 ceph-mon[36479]: Deploying daemon osd.11 on smithi164 2024-04-02T17:09:26.763 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:26 smithi164 ceph-mon[36499]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:09:26.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:26 smithi059 ceph-mon[31664]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:09:26.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:26 smithi154 ceph-mon[36479]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T17:09:28.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:28 smithi059 ceph-mon[31664]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-02T17:09:28.807 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:28 smithi164 ceph-mon[36499]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-02T17:09:28.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:28 smithi154 ceph-mon[36479]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-02T17:09:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:30.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:31.304 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:31 smithi164 ceph-mon[36499]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:09:31.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:31 smithi154 ceph-mon[36479]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:09:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:31 smithi059 ceph-mon[31664]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T17:09:32.535 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 11 on host 'smithi164' 2024-04-02T17:09:32.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.535+0000 7f06da7fc700 1 -- 172.21.15.164:0/1115811659 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f06e4062a30 con 0x7f06d00a3c40 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 msgr2=0x7f06d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 0x7f06d00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b2e0 tx=0x7f06d401a040 comp rx=0 tx=0).stop 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 msgr2=0x7f06e419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b560 tx=0x7f06e000b870 comp rx=0 tx=0).stop 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 shutdown_connections 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06d00a3c40 0x7f06d00a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e41021b0 0x7f06e41a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e4100e90 0x7f06e419aaa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:32.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 --2- 172.21.15.164:0/1115811659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e41017f0 0x7f06e419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:32.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 >> 172.21.15.164:0/1115811659 conn(0x7f06e40fcaf0 msgr2=0x7f06e40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:32.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 shutdown_connections 2024-04-02T17:09:32.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:32.537+0000 7f06eb48b700 1 -- 172.21.15.164:0/1115811659 wait complete. 2024-04-02T17:09:33.149 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:33 smithi164 ceph-mon[36499]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:33.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:33 smithi164 ceph-mon[36499]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:33 smithi164 ceph-mon[36499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.174 DEBUG:teuthology.orchestra.run.smithi164:osd.11> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.11.service 2024-04-02T17:09:33.177 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-02T17:09:33.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T17:09:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:33 smithi154 ceph-mon[36479]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:33 smithi154 ceph-mon[36479]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:33 smithi154 ceph-mon[36479]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:33 smithi059 ceph-mon[31664]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:33.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:33 smithi059 ceph-mon[31664]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:33 smithi059 ceph-mon[31664]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T17:09:33.658 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T17:09:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:34 smithi164 ceph-mon[36499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:34.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:34 smithi154 ceph-mon[36479]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:34.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:34 smithi059 ceph-mon[31664]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-02T17:09:35.505 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:09:35.399+0000 7f3caaed5700 -1 osd.11 0 waiting for initial osdmap 2024-04-02T17:09:35.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:35 smithi059 ceph-mon[31664]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:09:35.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:35 smithi059 ceph-mon[31664]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T17:09:35.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:35.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:35 smithi059 ceph-mon[31664]: from='osd.11 ' entity='osd.11' 2024-04-02T17:09:35.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-mon[36499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:09:35.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-mon[36499]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T17:09:35.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:35.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-mon[36499]: from='osd.11 ' entity='osd.11' 2024-04-02T17:09:35.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:09:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:09:35.532+0000 7f3ca2cc0700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T17:09:35.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 -- 172.21.15.59:0/565163798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130105a70 msgr2=0x7f8130105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 --2- 172.21.15.59:0/565163798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130105a70 0x7f8130105ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f811c009990 tx=0x7f811c009ca0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 -- 172.21.15.59:0/565163798 shutdown_connections 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 --2- 172.21.15.59:0/565163798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130106410 0x7f813010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 --2- 172.21.15.59:0/565163798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130104870 0x7f8130104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 --2- 172.21.15.59:0/565163798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130105a70 0x7f8130105ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 -- 172.21.15.59:0/565163798 >> 172.21.15.59:0/565163798 conn(0x7f81300ffe10 msgr2=0x7f8130102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 -- 172.21.15.59:0/565163798 shutdown_connections 2024-04-02T17:09:35.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.872+0000 7f8136400700 1 -- 172.21.15.59:0/565163798 wait complete. 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.873+0000 7f8136400700 1 Processor -- start 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.873+0000 7f8136400700 1 -- start start 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.873+0000 7f8136400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.873+0000 7f8136400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 0x7f813019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f8136400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130106410 0x7f81301a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f8136400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8130110480 con 0x7f8130104870 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f8136400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8130110300 con 0x7f8130105a70 2024-04-02T17:09:35.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f8136400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8130110600 con 0x7f8130106410 2024-04-02T17:09:35.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130106410 0x7f81301a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46760/0 (socket says 172.21.15.59:46760) 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 -- 172.21.15.59:0/792636506 learned_addr learned my addr 172.21.15.59:0/792636506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812ea00700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 0x7f813019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 -- 172.21.15.59:0/792636506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130106410 msgr2=0x7f81301a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130106410 0x7f81301a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 -- 172.21.15.59:0/792636506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 msgr2=0x7f813019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 0x7f813019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812f400700 1 -- 172.21.15.59:0/792636506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a2710 con 0x7f8130104870 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.874+0000 7f812ea00700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 0x7f813019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f812f400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f812000e9a0 tx=0x7f812000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812000cbd0 con 0x7f8130104870 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a2a00 con 0x7f8130104870 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8120004500 con 0x7f8130104870 2024-04-02T17:09:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120003de0 con 0x7f8130104870 2024-04-02T17:09:35.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.875+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a2f90 con 0x7f8130104870 2024-04-02T17:09:35.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.876+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81300692b0 con 0x7f8130104870 2024-04-02T17:09:35.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.880+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8120003fa0 con 0x7f8130104870 2024-04-02T17:09:35.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.881+0000 7f8127e00700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 0x7f81140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:35.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.881+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f81200d03b0 con 0x7f8130104870 2024-04-02T17:09:35.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.882+0000 7f812ea00700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 0x7f81140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:35.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.882+0000 7f812ea00700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 0x7f81140a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b2e0 tx=0x7f811c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:35.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:35.884+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8120053f20 con 0x7f8130104870 2024-04-02T17:09:35.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:35 smithi154 ceph-mon[36479]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-02T17:09:35.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:35 smithi154 ceph-mon[36479]: osdmap e81: 12 total, 11 up, 12 in 2024-04-02T17:09:35.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:35.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:35 smithi154 ceph-mon[36479]: from='osd.11 ' entity='osd.11' 2024-04-02T17:09:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.126+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f813002dd30 con 0x7f8130104870 2024-04-02T17:09:36.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.127+0000 7f8127e00700 1 -- 172.21.15.59:0/792636506 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f812009d2b0 con 0x7f8130104870 2024-04-02T17:09:36.126 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 msgr2=0x7f81140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 0x7f81140a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b2e0 tx=0x7f811c01a040 comp rx=0 tx=0).stop 2024-04-02T17:09:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 msgr2=0x7f813019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f812000e9a0 tx=0x7f812000ed60 comp rx=0 tx=0).stop 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 shutdown_connections 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130106410 0x7f81301a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c40 0x7f81140a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130104870 0x7f813019aab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 --2- 172.21.15.59:0/792636506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130105a70 0x7f813019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 >> 172.21.15.59:0/792636506 conn(0x7f81300ffe10 msgr2=0x7f8130108d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:36.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 shutdown_connections 2024-04-02T17:09:36.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:36.130+0000 7f8136400700 1 -- 172.21.15.59:0/792636506 wait complete. 2024-04-02T17:09:36.719 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:36 smithi164 ceph-mon[36499]: purged_snaps scrub starts 2024-04-02T17:09:36.719 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:36 smithi164 ceph-mon[36499]: purged_snaps scrub ok 2024-04-02T17:09:36.719 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:36 smithi164 ceph-mon[36499]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:36.719 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:36.719 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:36 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/792636506' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:36 smithi059 ceph-mon[31664]: purged_snaps scrub starts 2024-04-02T17:09:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:36 smithi059 ceph-mon[31664]: purged_snaps scrub ok 2024-04-02T17:09:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:36 smithi059 ceph-mon[31664]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:36.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:36 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/792636506' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:36.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:36 smithi154 ceph-mon[36479]: purged_snaps scrub starts 2024-04-02T17:09:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:36 smithi154 ceph-mon[36479]: purged_snaps scrub ok 2024-04-02T17:09:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:36 smithi154 ceph-mon[36479]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:36.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:36 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/792636506' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:36.987 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1712077751,"num_in_osds":12,"osd_in_since":1712077756,"num_remapped_pgs":0} 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] boot 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: Detected new or changed devices on smithi164 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 4270M 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] boot 2024-04-02T17:09:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: Detected new or changed devices on smithi164 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 4270M 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:37.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.824 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: osd.11 [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] boot 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: Detected new or changed devices on smithi164 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 4270M 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:37.901 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:37.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T17:09:38.445 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:09:38.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:38 smithi059 ceph-mon[31664]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:38.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:38 smithi059 ceph-mon[31664]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T17:09:39.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:38 smithi154 ceph-mon[36479]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:39.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:38 smithi154 ceph-mon[36479]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T17:09:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:38 smithi164 ceph-mon[36499]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-02T17:09:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:38 smithi164 ceph-mon[36499]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T17:09:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.417+0000 7f4902e00700 1 -- 172.21.15.59:0/3971636562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc105970 msgr2=0x7f48fc105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.417+0000 7f4902e00700 1 --2- 172.21.15.59:0/3971636562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc105970 0x7f48fc105dd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f48e80099c0 tx=0x7f48e8009cd0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 -- 172.21.15.59:0/3971636562 shutdown_connections 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 --2- 172.21.15.59:0/3971636562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc106310 0x7f48fc10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 --2- 172.21.15.59:0/3971636562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 --2- 172.21.15.59:0/3971636562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc105970 0x7f48fc105dd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 -- 172.21.15.59:0/3971636562 >> 172.21.15.59:0/3971636562 conn(0x7f48fc0ffd10 msgr2=0x7f48fc102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 -- 172.21.15.59:0/3971636562 shutdown_connections 2024-04-02T17:09:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.418+0000 7f4902e00700 1 -- 172.21.15.59:0/3971636562 wait complete. 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.419+0000 7f4902e00700 1 Processor -- start 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.419+0000 7f4902e00700 1 -- start start 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc106310 0x7f48fc1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc110350 con 0x7f48fc106310 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1101d0 con 0x7f48fc105970 2024-04-02T17:09:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1104d0 con 0x7f48fc104770 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4901000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4902400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc106310 0x7f48fc1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4901a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4901000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48996/0 (socket says 172.21.15.59:48996) 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4901a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50942/0 (socket says 172.21.15.59:50942) 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.420+0000 7f4901000700 1 -- 172.21.15.59:0/3473833429 learned_addr learned my addr 172.21.15.59:0/3473833429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 -- 172.21.15.59:0/3473833429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 msgr2=0x7f48fc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 -- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc106310 msgr2=0x7f48fc1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc106310 0x7f48fc1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 -- 172.21.15.59:0/3473833429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc1a25e0 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4901000700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8027070 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e800fe00 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8017650 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.421+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48fc1a2870 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.422+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48fc1a2c70 con 0x7f48fc105970 2024-04-02T17:09:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.423+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48fc0692b0 con 0x7f48fc105970 2024-04-02T17:09:40.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.425+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48e802a070 con 0x7f48fc105970 2024-04-02T17:09:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.426+0000 7f48fa000700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 0x7f48e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.426+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f48e80d0fd0 con 0x7f48fc105970 2024-04-02T17:09:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.427+0000 7f4901a00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 0x7f48e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:40.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.427+0000 7f4901a00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 0x7f48e40a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1057d0 tx=0x7f48f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:40.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.429+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f48e8031080 con 0x7f48fc105970 2024-04-02T17:09:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.662+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f48fc02de00 con 0x7f48fc105970 2024-04-02T17:09:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.663+0000 7f48fa000700 1 -- 172.21.15.59:0/3473833429 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f48e8013070 con 0x7f48fc105970 2024-04-02T17:09:40.662 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 msgr2=0x7f48e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 0x7f48e40a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1057d0 tx=0x7f48f000b410 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 msgr2=0x7f48fc19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e800fa00 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 shutdown_connections 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f48e40a3cc0 0x7f48e40a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48fc106310 0x7f48fc1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc104770 0x7f48fc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 --2- 172.21.15.59:0/3473833429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48fc105970 0x7f48fc19aec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 >> 172.21.15.59:0/3473833429 conn(0x7f48fc0ffd10 msgr2=0x7f48fc108c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 shutdown_connections 2024-04-02T17:09:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:40.666+0000 7f4902e00700 1 -- 172.21.15.59:0/3473833429 wait complete. 2024-04-02T17:09:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:40 smithi059 ceph-mon[31664]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3473833429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:41.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:40 smithi154 ceph-mon[36479]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:41.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:40 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3473833429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:40 smithi164 ceph-mon[36499]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:40 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3473833429' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T17:09:41.431 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1712077776,"num_in_osds":12,"osd_in_since":1712077756,"num_remapped_pgs":0} 2024-04-02T17:09:41.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T17:09:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:09:43.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:42 smithi154 ceph-mon[36479]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:43.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:42 smithi059 ceph-mon[31664]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:42 smithi164 ceph-mon[36499]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 -- 172.21.15.59:0/646750029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad41051a0 msgr2=0x7f2ad41116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 --2- 172.21.15.59:0/646750029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad41051a0 0x7f2ad41116b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b0a0 tx=0x7f2acc00b3b0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 -- 172.21.15.59:0/646750029 shutdown_connections 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 --2- 172.21.15.59:0/646750029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad41051a0 0x7f2ad41116b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 --2- 172.21.15.59:0/646750029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad410a7e0 0x7f2ad410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 --2- 172.21.15.59:0/646750029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ad41047e0 0x7f2ad4104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 -- 172.21.15.59:0/646750029 >> 172.21.15.59:0/646750029 conn(0x7f2ad41002d0 msgr2=0x7f2ad41026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 -- 172.21.15.59:0/646750029 shutdown_connections 2024-04-02T17:09:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.972+0000 7f2ada800700 1 -- 172.21.15.59:0/646750029 wait complete. 2024-04-02T17:09:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.973+0000 7f2ada800700 1 Processor -- start 2024-04-02T17:09:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.973+0000 7f2ada800700 1 -- start start 2024-04-02T17:09:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ad41047e0 0x7f2ad419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 0x7f2ad419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4114c40 con 0x7f2ad410a7e0 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4114ac0 con 0x7f2ad41047e0 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ada800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4114dc0 con 0x7f2ad41051a0 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 0x7f2ad419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43046/0 (socket says 172.21.15.59:43046) 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 -- 172.21.15.59:0/3574003387 learned_addr learned my addr 172.21.15.59:0/3574003387 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad2a00700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 0x7f2ad419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36896/0 (socket says 172.21.15.59:36896) 2024-04-02T17:09:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 -- 172.21.15.59:0/3574003387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 msgr2=0x7f2ad419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 0x7f2ad419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 -- 172.21.15.59:0/3574003387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ad41047e0 msgr2=0x7f2ad419b110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T17:09:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ad41047e0 0x7f2ad419b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.974+0000 7f2ad3e00700 1 -- 172.21.15.59:0/3574003387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad41a0120 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2ad3e00700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b830 tx=0x7f2acc00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc021020 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2acc007dc0 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad41a03b0 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc005560 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.975+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad41a7c90 con 0x7f2ad410a7e0 2024-04-02T17:09:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.976+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad40692b0 con 0x7f2ad410a7e0 2024-04-02T17:09:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.979+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2acc024020 con 0x7f2ad410a7e0 2024-04-02T17:09:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.980+0000 7f2acbe00700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 0x7f2ab80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.980+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2acc0d1bc0 con 0x7f2ad410a7e0 2024-04-02T17:09:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.980+0000 7f2ad3400700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 0x7f2ab80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.981+0000 7f2ad3400700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 0x7f2ab80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009730 tx=0x7f2ac4006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:43.983+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2acc02b020 con 0x7f2ad410a7e0 2024-04-02T17:09:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.222+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ad402dd30 con 0x7f2ad410a7e0 2024-04-02T17:09:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.224+0000 7f2acbe00700 1 -- 172.21.15.59:0/3574003387 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18458 (secure 0 0 0) 0x7f2acc01a3c0 con 0x7f2ad410a7e0 2024-04-02T17:09:44.224 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:44.224 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:09:37.821176+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 msgr2=0x7f2ab80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 0x7f2ab80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009730 tx=0x7f2ac4006cb0 comp rx=0 tx=0).stop 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 msgr2=0x7f2ad419f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b830 tx=0x7f2acc00ce90 comp rx=0 tx=0).stop 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 shutdown_connections 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ab80a3c40 0x7f2ab80a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad410a7e0 0x7f2ad419f9e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ad41047e0 0x7f2ad419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:44.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 --2- 172.21.15.59:0/3574003387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad41051a0 0x7f2ad419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:44.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.227+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 >> 172.21.15.59:0/3574003387 conn(0x7f2ad41002d0 msgr2=0x7f2ad41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:44.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.228+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 shutdown_connections 2024-04-02T17:09:44.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:44.228+0000 7f2ada800700 1 -- 172.21.15.59:0/3574003387 wait complete. 2024-04-02T17:09:44.879 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:09:44.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T17:09:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:44 smithi154 ceph-mon[36479]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:45.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:44 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3574003387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:44 smithi059 ceph-mon[31664]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:44 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3574003387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:44 smithi164 ceph-mon[36499]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:44 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3574003387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:45.337 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:09:47.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:46 smithi059 ceph-mon[31664]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:47.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:46 smithi154 ceph-mon[36479]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:47.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:46 smithi164 ceph-mon[36499]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 -- 172.21.15.59:0/172650182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892406a2a0 msgr2=0x7f892406a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 --2- 172.21.15.59:0/172650182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892406a2a0 0x7f892406a720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f890c009920 tx=0x7f890c009c30 comp rx=0 tx=0).stop 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 -- 172.21.15.59:0/172650182 shutdown_connections 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 --2- 172.21.15.59:0/172650182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406ac60 0x7f892410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 --2- 172.21.15.59:0/172650182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8924108650 0x7f8924108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 --2- 172.21.15.59:0/172650182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892406a2a0 0x7f892406a720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 -- 172.21.15.59:0/172650182 >> 172.21.15.59:0/172650182 conn(0x7f8924078420 msgr2=0x7f8924078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 -- 172.21.15.59:0/172650182 shutdown_connections 2024-04-02T17:09:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.325+0000 7f892ae00700 1 -- 172.21.15.59:0/172650182 wait complete. 2024-04-02T17:09:47.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.326+0000 7f892ae00700 1 Processor -- start 2024-04-02T17:09:47.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 -- start start 2024-04-02T17:09:47.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406a2a0 0x7f8924198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 0x7f8924199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 0x7f892419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8924112930 con 0x7f892406ac60 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89241127b0 con 0x7f8924108650 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.327+0000 7f892ae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8924112ab0 con 0x7f892406a2a0 2024-04-02T17:09:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f891b400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 0x7f8924199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f891b400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 0x7f8924199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43052/0 (socket says 172.21.15.59:43052) 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f891b400700 1 -- 172.21.15.59:0/1281388947 learned_addr learned my addr 172.21.15.59:0/1281388947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 0x7f892419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8923e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406a2a0 0x7f8924198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 -- 172.21.15.59:0/1281388947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406a2a0 msgr2=0x7f8924198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406a2a0 0x7f8924198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 -- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 msgr2=0x7f8924199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 0x7f8924199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 -- 172.21.15.59:0/1281388947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892419de50 con 0x7f8924108650 2024-04-02T17:09:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.328+0000 7f8928e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 0x7f892419d710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e970 tx=0x7f891c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.329+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c004220 con 0x7f8924108650 2024-04-02T17:09:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.329+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c004d10 con 0x7f8924108650 2024-04-02T17:09:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.329+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892419e140 con 0x7f8924108650 2024-04-02T17:09:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.329+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c010700 con 0x7f8924108650 2024-04-02T17:09:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.329+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241014f0 con 0x7f8924108650 2024-04-02T17:09:47.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.330+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89240692b0 con 0x7f8924108650 2024-04-02T17:09:47.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.332+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f891c004750 con 0x7f8924108650 2024-04-02T17:09:47.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.333+0000 7f8921600700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 0x7f89080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:47.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.334+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f891c0cffb0 con 0x7f8924108650 2024-04-02T17:09:47.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.334+0000 7f8923e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 0x7f89080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:47.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.334+0000 7f8923e00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 0x7f89080a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f891400f540 tx=0x7f891400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:47.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.336+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f891c09cd90 con 0x7f8924108650 2024-04-02T17:09:47.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.568+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f892419a1a0 con 0x7f8924108650 2024-04-02T17:09:47.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.569+0000 7f8921600700 1 -- 172.21.15.59:0/1281388947 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f891c09c690 con 0x7f8924108650 2024-04-02T17:09:47.568 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 msgr2=0x7f89080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 0x7f89080a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f891400f540 tx=0x7f891400b040 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 msgr2=0x7f892419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 0x7f892419d710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e970 tx=0x7f891c00ec80 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 shutdown_connections 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924108650 0x7f892419d710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892406a2a0 0x7f8924198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89080a3c40 0x7f89080a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 --2- 172.21.15.59:0/1281388947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892406ac60 0x7f8924199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 >> 172.21.15.59:0/1281388947 conn(0x7f8924078420 msgr2=0x7f89241001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 shutdown_connections 2024-04-02T17:09:47.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:47.572+0000 7f892ae00700 1 -- 172.21.15.59:0/1281388947 wait complete. 2024-04-02T17:09:48.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:47 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1281388947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:48.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:47 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1281388947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:48.247 INFO:tasks.cephadm:Adding mds.a on smithi059 2024-04-02T17:09:48.247 INFO:tasks.cephadm:Adding mds.d on smithi059 2024-04-02T17:09:48.247 INFO:tasks.cephadm:Adding mds.g on smithi059 2024-04-02T17:09:48.247 INFO:tasks.cephadm:Adding mds.j on smithi059 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.b on smithi154 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.e on smithi154 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.h on smithi154 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.k on smithi154 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.c on smithi164 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.f on smithi164 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.i on smithi164 2024-04-02T17:09:48.248 INFO:tasks.cephadm:Adding mds.l on smithi164 2024-04-02T17:09:48.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l' 2024-04-02T17:09:48.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:47 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1281388947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:48.502 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:09:48.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:48 smithi164 ceph-mon[36499]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:49.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:48 smithi154 ceph-mon[36479]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:48 smithi059 ceph-mon[31664]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 -- 172.21.15.164:0/2734549292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10104280 msgr2=0x7f9b1010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 --2- 172.21.15.164:0/2734549292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10104280 0x7f9b1010ab30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b0a0 tx=0x7f9b0800b3b0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 -- 172.21.15.164:0/2734549292 shutdown_connections 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 --2- 172.21.15.164:0/2734549292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10104280 0x7f9b1010ab30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 --2- 172.21.15.164:0/2734549292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b10100900 0x7f9b10100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 --2- 172.21.15.164:0/2734549292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b101038e0 0x7f9b10103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 -- 172.21.15.164:0/2734549292 >> 172.21.15.164:0/2734549292 conn(0x7f9b100fc280 msgr2=0x7f9b100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 -- 172.21.15.164:0/2734549292 shutdown_connections 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.949+0000 7f9b1507d700 1 -- 172.21.15.164:0/2734549292 wait complete. 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.950+0000 7f9b1507d700 1 Processor -- start 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.950+0000 7f9b1507d700 1 -- start start 2024-04-02T17:09:49.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.950+0000 7f9b1507d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 0x7f9b10198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10104280 0x7f9b1019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1010e290 con 0x7f9b10104280 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1010e110 con 0x7f9b101038e0 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1010e410 con 0x7f9b10100900 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39330/0 (socket says 172.21.15.164:39330) 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 -- 172.21.15.164:0/1182453344 learned_addr learned my addr 172.21.15.164:0/1182453344 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0e59c700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 0x7f9b10198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 -- 172.21.15.164:0/1182453344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 msgr2=0x7f9b10198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 0x7f9b10198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 -- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10104280 msgr2=0x7f9b1019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:49.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10104280 0x7f9b1019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 -- 172.21.15.164:0/1182453344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101a0450 con 0x7f9b10100900 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0e59c700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 0x7f9b10198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b0ed9d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d9b0 tx=0x7f9b0000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00017070 con 0x7f9b10100900 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b101a0740 con 0x7f9b10100900 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b00009b40 con 0x7f9b10100900 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000f620 con 0x7f9b10100900 2024-04-02T17:09:49.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.951+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a0c30 con 0x7f9b10100900 2024-04-02T17:09:49.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.953+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b100692b0 con 0x7f9b10100900 2024-04-02T17:09:49.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.954+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b00009cf0 con 0x7f9b10100900 2024-04-02T17:09:49.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.954+0000 7f9af7fff700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 0x7f9af80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:49.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.954+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f9b000cfbb0 con 0x7f9b10100900 2024-04-02T17:09:49.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.955+0000 7f9b0e59c700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 0x7f9af80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:49.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.956+0000 7f9b0e59c700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 0x7f9af80a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b040099d0 tx=0x7f9b04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:49.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:49.959+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b0009ca30 con 0x7f9b10100900 2024-04-02T17:09:50.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.171+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}) -- 0x7f9b10062cf0 con 0x7f9af80a3c40 2024-04-02T17:09:50.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.188+0000 7f9af7fff700 1 -- 172.21.15.164:0/1182453344 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9b10062cf0 con 0x7f9af80a3c40 2024-04-02T17:09:50.189 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mds.all update... 2024-04-02T17:09:50.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 msgr2=0x7f9af80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 0x7f9af80a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b040099d0 tx=0x7f9b04008040 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 msgr2=0x7f9b101987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d9b0 tx=0x7f9b0000dd70 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 shutdown_connections 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9af80a3c40 0x7f9af80a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10104280 0x7f9b1019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b10100900 0x7f9b101987f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 --2- 172.21.15.164:0/1182453344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b101038e0 0x7f9b10198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 >> 172.21.15.164:0/1182453344 conn(0x7f9b100fc280 msgr2=0x7f9b10106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 shutdown_connections 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-02T17:09:50.191+0000 7f9b1507d700 1 -- 172.21.15.164:0/1182453344 wait complete. 2024-04-02T17:09:50.846 DEBUG:teuthology.orchestra.run.smithi059:mds.a> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.a.service 2024-04-02T17:09:50.854 DEBUG:teuthology.orchestra.run.smithi059:mds.d> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.d.service 2024-04-02T17:09:50.900 DEBUG:teuthology.orchestra.run.smithi059:mds.g> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.g.service 2024-04-02T17:09:50.947 DEBUG:teuthology.orchestra.run.smithi059:mds.j> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.j.service 2024-04-02T17:09:50.991 DEBUG:teuthology.orchestra.run.smithi154:mds.b> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.b.service 2024-04-02T17:09:50.994 DEBUG:teuthology.orchestra.run.smithi154:mds.e> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.e.service 2024-04-02T17:09:51.039 DEBUG:teuthology.orchestra.run.smithi154:mds.h> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.h.service 2024-04-02T17:09:51.081 DEBUG:teuthology.orchestra.run.smithi154:mds.k> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.k.service 2024-04-02T17:09:51.124 DEBUG:teuthology.orchestra.run.smithi164:mds.c> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.c.service 2024-04-02T17:09:51.126 DEBUG:teuthology.orchestra.run.smithi164:mds.f> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.f.service 2024-04-02T17:09:51.169 DEBUG:teuthology.orchestra.run.smithi164:mds.i> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.i.service 2024-04-02T17:09:51.212 DEBUG:teuthology.orchestra.run.smithi164:mds.l> sudo journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.l.service 2024-04-02T17:09:51.255 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-02T17:09:51.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:09:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:51 smithi154 ceph-mon[36479]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:51 smithi154 ceph-mon[36479]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:51 smithi154 ceph-mon[36479]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-02T17:09:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:51.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 -- 172.21.15.59:0/1185098753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448104d00 msgr2=0x7ff4481070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 --2- 172.21.15.59:0/1185098753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448104d00 0x7ff4481070f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff430009900 tx=0x7ff43002bc30 comp rx=0 tx=0).stop 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 -- 172.21.15.59:0/1185098753 shutdown_connections 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 --2- 172.21.15.59:0/1185098753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff448107630 0x7ff448109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 --2- 172.21.15.59:0/1185098753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448104d00 0x7ff4481070f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 --2- 172.21.15.59:0/1185098753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4481023d0 0x7ff4481047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 -- 172.21.15.59:0/1185098753 >> 172.21.15.59:0/1185098753 conn(0x7ff4480fbde0 msgr2=0x7ff4480fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 -- 172.21.15.59:0/1185098753 shutdown_connections 2024-04-02T17:09:51.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.476+0000 7ff447e00640 1 -- 172.21.15.59:0/1185098753 wait complete. 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.477+0000 7ff447e00640 1 Processor -- start 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.477+0000 7ff447e00640 1 -- start start 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 0x7ff4481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448107630 0x7ff44819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d610 con 0x7ff448104d00 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d490 con 0x7ff448107630 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d790 con 0x7ff4481023d0 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 0x7ff4481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54140/0 (socket says 172.21.15.59:54140) 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff447400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448107630 0x7ff44819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 0x7ff4481981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54354/0 (socket says 172.21.15.59:54354) 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 -- 172.21.15.59:0/1250450652 learned_addr learned my addr 172.21.15.59:0/1250450652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 -- 172.21.15.59:0/1250450652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448107630 msgr2=0x7ff44819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448107630 0x7ff44819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 -- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 msgr2=0x7ff4481981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 0x7ff4481981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.478+0000 7ff446a00640 1 -- 172.21.15.59:0/1250450652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44819f890 con 0x7ff4481023d0 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff446a00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d950 tx=0x7ff43800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x7ff4481023d0 2024-04-02T17:09:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff438009c30 con 0x7ff4481023d0 2024-04-02T17:09:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44819fb80 con 0x7ff4481023d0 2024-04-02T17:09:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43800f6b0 con 0x7ff4481023d0 2024-04-02T17:09:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.479+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a0120 con 0x7ff4481023d0 2024-04-02T17:09:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.480+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4480fd8d0 con 0x7ff4481023d0 2024-04-02T17:09:51.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.483+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4380109d0 con 0x7ff4481023d0 2024-04-02T17:09:51.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.484+0000 7ff43f400640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 0x7ff42c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.485+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff4380cfb70 con 0x7ff4481023d0 2024-04-02T17:09:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.485+0000 7ff446000640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 0x7ff42c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.486+0000 7ff446000640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 0x7ff42c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff43002c170 tx=0x7ff43002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.488+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff438009de0 con 0x7ff4481023d0 2024-04-02T17:09:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:51 smithi059 ceph-mon[31664]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:51 smithi059 ceph-mon[31664]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:51 smithi059 ceph-mon[31664]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-02T17:09:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:51 smithi164 ceph-mon[36499]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:51 smithi164 ceph-mon[36499]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:09:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:51 smithi164 ceph-mon[36499]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-02T17:09:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:09:51.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.711+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff448062700 con 0x7ff4481023d0 2024-04-02T17:09:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.712+0000 7ff43f400640 1 -- 172.21.15.59:0/1250450652 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18458 (secure 0 0 0) 0x7ff43809c300 con 0x7ff4481023d0 2024-04-02T17:09:51.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:51.713 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:09:37.821176+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:09:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 msgr2=0x7ff42c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 0x7ff42c0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff43002c170 tx=0x7ff43002df90 comp rx=0 tx=0).stop 2024-04-02T17:09:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 msgr2=0x7ff448197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d950 tx=0x7ff43800de10 comp rx=0 tx=0).stop 2024-04-02T17:09:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 shutdown_connections 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff42c0a3c30 0x7ff42c0a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff448104d00 0x7ff4481981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff448107630 0x7ff44819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 --2- 172.21.15.59:0/1250450652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4481023d0 0x7ff448197c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 >> 172.21.15.59:0/1250450652 conn(0x7ff4480fbde0 msgr2=0x7ff448102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 shutdown_connections 2024-04-02T17:09:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.715+0000 7ff447e00640 1 -- 172.21.15.59:0/1250450652 wait complete. 2024-04-02T17:09:51.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:09:51.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:09:51.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 -- 172.21.15.59:0/222608735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 msgr2=0x7f3cc8110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 --2- 172.21.15.59:0/222608735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc8110580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009be0 tx=0x7f3cc002bef0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 -- 172.21.15.59:0/222608735 shutdown_connections 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 --2- 172.21.15.59:0/222608735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8109330 0x7f3cc81097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 --2- 172.21.15.59:0/222608735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc8110580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 --2- 172.21.15.59:0/222608735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8108140 0x7f3cc8108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 -- 172.21.15.59:0/222608735 >> 172.21.15.59:0/222608735 conn(0x7f3cc80770f0 msgr2=0x7f3cc8079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 -- 172.21.15.59:0/222608735 shutdown_connections 2024-04-02T17:09:51.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.970+0000 7f3ccd600640 1 -- 172.21.15.59:0/222608735 wait complete. 2024-04-02T17:09:51.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 Processor -- start 2024-04-02T17:09:51.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 -- start start 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8109330 0x7f3cc819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc81a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8113df0 con 0x7f3cc8108140 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8113c70 con 0x7f3cc8109cf0 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.971+0000 7f3ccd600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8113f70 con 0x7f3cc8109330 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54374/0 (socket says 172.21.15.59:54374) 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 -- 172.21.15.59:0/2957193548 learned_addr learned my addr 172.21.15.59:0/2957193548 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6000640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8109330 0x7f3cc819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc7400640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc81a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 -- 172.21.15.59:0/2957193548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8109330 msgr2=0x7f3cc819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8109330 0x7f3cc819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 -- 172.21.15.59:0/2957193548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 msgr2=0x7f3cc81a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc81a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 -- 172.21.15.59:0/2957193548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc81a5fc0 con 0x7f3cc8108140 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.972+0000 7f3cc6a00640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.973+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800cd80 con 0x7f3cc8108140 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.973+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc81a62b0 con 0x7f3cc8108140 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.973+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8010910 con 0x7f3cc8108140 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.973+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8018a40 con 0x7f3cc8108140 2024-04-02T17:09:51.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.973+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc81a6850 con 0x7f3cc8108140 2024-04-02T17:09:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.974+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc810f400 con 0x7f3cc8108140 2024-04-02T17:09:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.976+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3cb8010ac0 con 0x7f3cc8108140 2024-04-02T17:09:51.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.978+0000 7f3cbf400640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:51.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.978+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f3cb80d09f0 con 0x7f3cc8108140 2024-04-02T17:09:51.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.978+0000 7f3cc6000640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:51.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.979+0000 7f3cc6000640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3cc819f8b0 tx=0x7f3cb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:51.980+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3cb8026070 con 0x7f3cc8108140 2024-04-02T17:09:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.188+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cc8062700 con 0x7f3cc8108140 2024-04-02T17:09:52.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.195+0000 7f3cbf400640 1 -- 172.21.15.59:0/2957193548 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f3cb809d7d0 con 0x7f3cc8108140 2024-04-02T17:09:52.195 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.197+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 msgr2=0x7f3cac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.197+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3cc819f8b0 tx=0x7f3cb4005f90 comp rx=0 tx=0).stop 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.197+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 msgr2=0x7f3cc819e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.197+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ee60 comp rx=0 tx=0).stop 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 shutdown_connections 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc8109cf0 0x7f3cc81a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cc8109330 0x7f3cc819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 --2- 172.21.15.59:0/2957193548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc8108140 0x7f3cc819e3a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 >> 172.21.15.59:0/2957193548 conn(0x7f3cc80770f0 msgr2=0x7f3cc810c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:52.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 shutdown_connections 2024-04-02T17:09:52.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.198+0000 7f3ccd600640 1 -- 172.21.15.59:0/2957193548 wait complete. 2024-04-02T17:09:52.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:52.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.394 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:52 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1250450652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:52.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 -- 172.21.15.59:0/3237904595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c107630 msgr2=0x7f354c109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 --2- 172.21.15.59:0/3237904595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c107630 0x7f354c109a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3544009900 tx=0x7f354402bc30 comp rx=0 tx=0).stop 2024-04-02T17:09:52.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 -- 172.21.15.59:0/3237904595 shutdown_connections 2024-04-02T17:09:52.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 --2- 172.21.15.59:0/3237904595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c107630 0x7f354c109a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 --2- 172.21.15.59:0/3237904595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c104d00 0x7f354c1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 --2- 172.21.15.59:0/3237904595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 -- 172.21.15.59:0/3237904595 >> 172.21.15.59:0/3237904595 conn(0x7f354c0fbde0 msgr2=0x7f354c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:52.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.439+0000 7f3551200640 1 -- 172.21.15.59:0/3237904595 shutdown_connections 2024-04-02T17:09:52.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.440+0000 7f3551200640 1 -- 172.21.15.59:0/3237904595 wait complete. 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.440+0000 7f3551200640 1 Processor -- start 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 -- start start 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 0x7f354c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 0x7f354c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f354c10d620 con 0x7f354c104d00 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f354c10d4a0 con 0x7f354c1023d0 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f3551200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f354c10d7a0 con 0x7f354c107630 2024-04-02T17:09:52.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f354b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f354b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42860/0 (socket says 172.21.15.59:42860) 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f354b400640 1 -- 172.21.15.59:0/1496604980 learned_addr learned my addr 172.21.15.59:0/1496604980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.441+0000 7f354be00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 0x7f354c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354aa00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 0x7f354c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 -- 172.21.15.59:0/1496604980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 msgr2=0x7f354c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 0x7f354c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 -- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 msgr2=0x7f354c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 0x7f354c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 -- 172.21.15.59:0/1496604980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c19f830 con 0x7f354c1023d0 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354be00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 0x7f354c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:09:52.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354aa00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 0x7f354c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f354b400640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9f0 tx=0x7f353c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c00cd80 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c010910 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c19fb20 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.442+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c018990 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.443+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1a0070 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.444+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c0fd8d0 con 0x7f354c1023d0 2024-04-02T17:09:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.446+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f353c010ac0 con 0x7f354c1023d0 2024-04-02T17:09:52.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.447+0000 7f3543e00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:52.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.447+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f353c0d05a0 con 0x7f354c1023d0 2024-04-02T17:09:52.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.447+0000 7f354aa00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:52.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.448+0000 7f354aa00640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 0x7f35300a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f354c0697c0 tx=0x7f3538009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:52.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.449+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f353c09d420 con 0x7f354c1023d0 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:52 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1250450652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:52.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:52 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1250450652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:09:52.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.701+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354c062700 con 0x7f354c1023d0 2024-04-02T17:09:52.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.702+0000 7f3543e00640 1 -- 172.21.15.59:0/1496604980 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f353c09cd20 con 0x7f354c1023d0 2024-04-02T17:09:52.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:52.703 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-02T17:09:52.703 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 1 2024-04-02T17:09:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 msgr2=0x7f35300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 0x7f35300a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f354c0697c0 tx=0x7f3538009500 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 msgr2=0x7f354c197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9f0 tx=0x7f353c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 shutdown_connections 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f354c104d00 0x7f354c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354c107630 0x7f354c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 --2- 172.21.15.59:0/1496604980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c1023d0 0x7f354c197c20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 >> 172.21.15.59:0/1496604980 conn(0x7f354c0fbde0 msgr2=0x7f354c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 shutdown_connections 2024-04-02T17:09:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:52.707+0000 7f3551200640 1 -- 172.21.15.59:0/1496604980 wait complete. 2024-04-02T17:09:53.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:53 smithi164 ceph-mon[36499]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:53.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:53 smithi164 ceph-mon[36499]: Deploying daemon mds.l on smithi164 2024-04-02T17:09:53.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:53 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2957193548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:53.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:53 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1496604980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:53 smithi059 ceph-mon[31664]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:53 smithi059 ceph-mon[31664]: Deploying daemon mds.l on smithi164 2024-04-02T17:09:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:53 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2957193548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:53 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1496604980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:53.572 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:53 smithi164 systemd[1]: Starting Ceph mds.l for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:09:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:53 smithi154 ceph-mon[36479]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:53 smithi154 ceph-mon[36479]: Deploying daemon mds.l on smithi164 2024-04-02T17:09:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:53 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2957193548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:09:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:53 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1496604980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:54.017 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:53 smithi164 podman[66594]: 2024-04-02 17:09:53.614415937 +0000 UTC m=+0.018070363 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:09:54.017 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:53 smithi164 podman[66594]: 2024-04-02 17:09:53.807492458 +0000 UTC m=+0.211146919 container create 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:09:54.017 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 podman[66594]: 2024-04-02 17:09:54.018164465 +0000 UTC m=+0.421818873 container init 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T17:09:54.322 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 podman[66594]: 2024-04-02 17:09:54.020923803 +0000 UTC m=+0.424578225 container start 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:09:54.322 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 ceph-mds[66613]: -- 172.21.15.164:0/2745203651 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d579fda40 con 0x559d579d1000 2024-04-02T17:09:54.322 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: starting mds.l at 2024-04-02T17:09:54.322 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 bash[66594]: 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f 2024-04-02T17:09:54.323 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:09:54 smithi164 systemd[1]: Started Ceph mds.l for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:09:54.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 -- 172.21.15.59:0/2754660211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e24107800 msgr2=0x7f6e24109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 --2- 172.21.15.59:0/2754660211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e24107800 0x7f6e24109c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009ca0 tx=0x7f6e1c02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 -- 172.21.15.59:0/2754660211 shutdown_connections 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 --2- 172.21.15.59:0/2754660211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e24107800 0x7f6e24109c50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 --2- 172.21.15.59:0/2754660211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24104ed0 0x7f6e241072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 --2- 172.21.15.59:0/2754660211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e241025a0 0x7f6e24104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 -- 172.21.15.59:0/2754660211 >> 172.21.15.59:0/2754660211 conn(0x7f6e240fbfb0 msgr2=0x7f6e240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:54.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 -- 172.21.15.59:0/2754660211 shutdown_connections 2024-04-02T17:09:54.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.969+0000 7f6e2b800640 1 -- 172.21.15.59:0/2754660211 wait complete. 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 Processor -- start 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 -- start start 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 0x7f6e2419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24107800 0x7f6e241a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410d7c0 con 0x7f6e241025a0 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410d640 con 0x7f6e24104ed0 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e2b800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410d940 con 0x7f6e24107800 2024-04-02T17:09:54.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54424/0 (socket says 172.21.15.59:54424) 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e28e00640 1 -- 172.21.15.59:0/3778030554 learned_addr learned my addr 172.21.15.59:0/3778030554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.971+0000 7f6e1be00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 0x7f6e2419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e29800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24107800 0x7f6e241a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 -- 172.21.15.59:0/3778030554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24107800 msgr2=0x7f6e241a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24107800 0x7f6e241a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 -- 172.21.15.59:0/3778030554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 msgr2=0x7f6e2419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 0x7f6e2419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 -- 172.21.15.59:0/3778030554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241a1c30 con 0x7f6e241025a0 2024-04-02T17:09:54.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e1be00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 0x7f6e2419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:09:54.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e28e00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400e9f0 tx=0x7f6e1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:54.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400cd80 con 0x7f6e241025a0 2024-04-02T17:09:54.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e14010910 con 0x7f6e241025a0 2024-04-02T17:09:54.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.972+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241a1f20 con 0x7f6e241025a0 2024-04-02T17:09:54.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.973+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14018990 con 0x7f6e241025a0 2024-04-02T17:09:54.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.973+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e241a23e0 con 0x7f6e241025a0 2024-04-02T17:09:54.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.974+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e241092f0 con 0x7f6e241025a0 2024-04-02T17:09:54.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.976+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e14010ac0 con 0x7f6e241025a0 2024-04-02T17:09:54.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.978+0000 7f6e22000640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 0x7f6e080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:54.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.978+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6e140d0670 con 0x7f6e241025a0 2024-04-02T17:09:54.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.978+0000 7f6e1be00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 0x7f6e080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:54.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.979+0000 7f6e1be00640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 0x7f6e080a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e2419b520 tx=0x7f6e0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:54.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:54.980+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6e1409ce00 con 0x7f6e241025a0 2024-04-02T17:09:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6832/3588884595,v1:172.21.15.164:6833/3588884595] up:boot 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: fsmap 1 up:standby 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:55.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:55.151 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:55 smithi154 ceph-mon[36479]: Deploying daemon mds.k on smithi154 2024-04-02T17:09:55.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.230+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2402d370 con 0x7f6e241025a0 2024-04-02T17:09:55.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.231+0000 7f6e22000640 1 -- 172.21.15.59:0/3778030554 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f6e140a1100 con 0x7f6e241025a0 2024-04-02T17:09:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:55.232 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T17:09:55.232 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 2 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 msgr2=0x7f6e080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 0x7f6e080a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e2419b520 tx=0x7f6e0c005f90 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 msgr2=0x7f6e2419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400e9f0 tx=0x7f6e1400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 shutdown_connections 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e24107800 0x7f6e241a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e24104ed0 0x7f6e2419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6e080a3c30 0x7f6e080a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 --2- 172.21.15.59:0/3778030554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e241025a0 0x7f6e2419a010 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:55.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.234+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 >> 172.21.15.59:0/3778030554 conn(0x7f6e240fbfb0 msgr2=0x7f6e24101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:55.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.235+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 shutdown_connections 2024-04-02T17:09:55.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:55.235+0000 7f6e2b800640 1 -- 172.21.15.59:0/3778030554 wait complete. 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6832/3588884595,v1:172.21.15.164:6833/3588884595] up:boot 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: fsmap 1 up:standby 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:55 smithi059 ceph-mon[31664]: Deploying daemon mds.k on smithi154 2024-04-02T17:09:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6832/3588884595,v1:172.21.15.164:6833/3588884595] up:boot 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: fsmap 1 up:standby 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:55.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:55 smithi164 ceph-mon[36499]: Deploying daemon mds.k on smithi154 2024-04-02T17:09:56.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:56 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3778030554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:56.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:56 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3778030554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3778030554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:57.077 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:09:57 smithi154 systemd[1]: Started Ceph mds.k for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:09:57.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:09:57.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:57 smithi059 ceph-mon[31664]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:57.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:57 smithi154 ceph-mon[36479]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:57.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 -- 172.21.15.59:0/1541630358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4103d40 msgr2=0x7f00e4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 --2- 172.21.15.59:0/1541630358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4103d40 0x7f00e4104160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009900 tx=0x7f00d402bc30 comp rx=0 tx=0).stop 2024-04-02T17:09:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 -- 172.21.15.59:0/1541630358 shutdown_connections 2024-04-02T17:09:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 --2- 172.21.15.59:0/1541630358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e41058f0 0x7f00e410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 --2- 172.21.15.59:0/1541630358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e4104f30 0x7f00e41053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 --2- 172.21.15.59:0/1541630358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4103d40 0x7f00e4104160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 -- 172.21.15.59:0/1541630358 >> 172.21.15.59:0/1541630358 conn(0x7f00e40ff2b0 msgr2=0x7f00e41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 -- 172.21.15.59:0/1541630358 shutdown_connections 2024-04-02T17:09:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.453+0000 7f00ea400640 1 -- 172.21.15.59:0/1541630358 wait complete. 2024-04-02T17:09:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.455+0000 7f00ea400640 1 Processor -- start 2024-04-02T17:09:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.455+0000 7f00ea400640 1 -- start start 2024-04-02T17:09:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.455+0000 7f00ea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4104f30 0x7f00e419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00ea400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 0x7f00e41a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00ea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00e410f9e0 con 0x7f00e4104f30 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00ea400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e410f860 con 0x7f00e41058f0 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00ea400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e410fb60 con 0x7f00e4103d40 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54230/0 (socket says 172.21.15.59:54230) 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4104f30 0x7f00e419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 -- 172.21.15.59:0/60461520 learned_addr learned my addr 172.21.15.59:0/60461520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3e00640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 0x7f00e41a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 -- 172.21.15.59:0/60461520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 msgr2=0x7f00e41a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 0x7f00e41a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 -- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4104f30 msgr2=0x7f00e419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4104f30 0x7f00e419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3400640 1 -- 172.21.15.59:0/60461520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e41a1c50 con 0x7f00e4103d40 2024-04-02T17:09:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.456+0000 7f00e3e00640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 0x7f00e41a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00e3400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c00 tx=0x7f00d402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4043020 con 0x7f00e4103d40 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d40054e0 con 0x7f00e4103d40 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e41a1ee0 con 0x7f00e4103d40 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4003c40 con 0x7f00e4103d40 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.457+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e41a23a0 con 0x7f00e4103d40 2024-04-02T17:09:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.459+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e410b060 con 0x7f00e4103d40 2024-04-02T17:09:57.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.460+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00d4046020 con 0x7f00e4103d40 2024-04-02T17:09:57.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.462+0000 7f00dbe00640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:09:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.463+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f00d40f1f70 con 0x7f00e4103d40 2024-04-02T17:09:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.463+0000 7f00e2a00640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:09:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.463+0000 7f00e2a00640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00e419b540 tx=0x7f00d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:09:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.466+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f00d404d020 con 0x7f00e4103d40 2024-04-02T17:09:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:57 smithi164 ceph-mon[36499]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.771+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00e402d370 con 0x7f00e4103d40 2024-04-02T17:09:57.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.772+0000 7f00dbe00640 1 -- 172.21.15.59:0/60461520 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f00d40bed50 con 0x7f00e4103d40 2024-04-02T17:09:57.772 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:09:57.772 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T17:09:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-04-02T17:09:57.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.774+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 msgr2=0x7f00c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.774+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00e419b540 tx=0x7f00d0005f90 comp rx=0 tx=0).stop 2024-04-02T17:09:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.774+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 msgr2=0x7f00e419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:09:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.774+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c00 tx=0x7f00d402c650 comp rx=0 tx=0).stop 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 shutdown_connections 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e4104f30 0x7f00e419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e41058f0 0x7f00e41a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 --2- 172.21.15.59:0/60461520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4103d40 0x7f00e419a030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 >> 172.21.15.59:0/60461520 conn(0x7f00e40ff2b0 msgr2=0x7f00e4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 shutdown_connections 2024-04-02T17:09:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:09:57.775+0000 7f00ea400640 1 -- 172.21.15.59:0/60461520 wait complete. 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6832/1241853080,v1:172.21.15.154:6833/1241853080] up:boot 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: fsmap 2 up:standby 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: Deploying daemon mds.j on smithi059 2024-04-02T17:09:58.320 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/60461520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6832/1241853080,v1:172.21.15.154:6833/1241853080] up:boot 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: fsmap 2 up:standby 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:58.331 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:58.331 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:58.331 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: Deploying daemon mds.j on smithi059 2024-04-02T17:09:58.331 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/60461520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:58.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6832/1241853080,v1:172.21.15.154:6833/1241853080] up:boot 2024-04-02T17:09:58.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: fsmap 2 up:standby 2024-04-02T17:09:58.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: Deploying daemon mds.j on smithi059 2024-04-02T17:09:58.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/60461520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:09:59.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:09:59 smithi154 ceph-mon[36479]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:59.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:09:59 smithi059 ceph-mon[31664]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:09:59 smithi164 ceph-mon[36499]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:09:59.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.147+0000 7fef17a00640 1 -- 172.21.15.59:0/460821319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef101058f0 msgr2=0x7fef1010c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.147+0000 7fef17a00640 1 --2- 172.21.15.59:0/460821319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef101058f0 0x7fef1010c180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef08009ca0 tx=0x7fef0802c0f0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 -- 172.21.15.59:0/460821319 shutdown_connections 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 --2- 172.21.15.59:0/460821319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef101058f0 0x7fef1010c180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 --2- 172.21.15.59:0/460821319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 0x7fef101053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 --2- 172.21.15.59:0/460821319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef10103d40 0x7fef10104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 -- 172.21.15.59:0/460821319 >> 172.21.15.59:0/460821319 conn(0x7fef100ff2b0 msgr2=0x7fef101016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 -- 172.21.15.59:0/460821319 shutdown_connections 2024-04-02T17:10:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.148+0000 7fef17a00640 1 -- 172.21.15.59:0/460821319 wait complete. 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 Processor -- start 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 -- start start 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 0x7fef10199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 0x7fef1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 0x7fef101a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010f9c0 con 0x7fef10103d40 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010f840 con 0x7fef10104f30 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef17a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010fb40 con 0x7fef101058f0 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.149+0000 7fef15000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 0x7fef10199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef0fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 0x7fef1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 0x7fef10199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54460/0 (socket says 172.21.15.59:54460) 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15000640 1 -- 172.21.15.59:0/4140558793 learned_addr learned my addr 172.21.15.59:0/4140558793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 0x7fef101a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 -- 172.21.15.59:0/4140558793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 msgr2=0x7fef1019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 0x7fef1019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 -- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 msgr2=0x7fef10199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 0x7fef10199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 -- 172.21.15.59:0/4140558793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101a1bf0 con 0x7fef101058f0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.150+0000 7fef15a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 0x7fef101a1580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef08037040 tx=0x7fef08032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.151+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08041020 con 0x7fef101058f0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.151+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef08034cd0 con 0x7fef101058f0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.151+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101a1e80 con 0x7fef101058f0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.151+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08039040 con 0x7fef101058f0 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.151+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101a22e0 con 0x7fef101058f0 2024-04-02T17:10:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.152+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1010b060 con 0x7fef101058f0 2024-04-02T17:10:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.154+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef08044020 con 0x7fef101058f0 2024-04-02T17:10:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.155+0000 7fef0d600640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 0x7feef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.155+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fef080f1d90 con 0x7fef101058f0 2024-04-02T17:10:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.156+0000 7fef15000640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 0x7feef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.157+0000 7fef15000640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 0x7feef80a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef040058c0 tx=0x7fef04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.160+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fef0804b020 con 0x7fef101058f0 2024-04-02T17:10:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:00 smithi059 ceph-mon[31664]: overall HEALTH_OK 2024-04-02T17:10:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:00 smithi154 ceph-mon[36479]: overall HEALTH_OK 2024-04-02T17:10:00.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.436+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef1002d370 con 0x7fef101058f0 2024-04-02T17:10:00.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.438+0000 7fef0d600640 1 -- 172.21.15.59:0/4140558793 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fef080bec00 con 0x7fef101058f0 2024-04-02T17:10:00.438 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:00.438 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T17:10:00.439 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-04-02T17:10:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 msgr2=0x7feef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 0x7feef80a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef040058c0 tx=0x7fef04005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 msgr2=0x7fef101a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 0x7fef101a1580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef08037040 tx=0x7fef08032ea0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 shutdown_connections 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef101058f0 0x7fef101a1580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef10104f30 0x7fef1019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feef80a3bb0 0x7feef80a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 --2- 172.21.15.59:0/4140558793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef10103d40 0x7fef10199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 >> 172.21.15.59:0/4140558793 conn(0x7fef100ff2b0 msgr2=0x7fef10108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 shutdown_connections 2024-04-02T17:10:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:00.441+0000 7fef17a00640 1 -- 172.21.15.59:0/4140558793 wait complete. 2024-04-02T17:10:00.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:00 smithi164 ceph-mon[36499]: overall HEALTH_OK 2024-04-02T17:10:00.659 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:10:00 smithi059 podman[70128]: 2024-04-02 17:10:00.609879238 +0000 UTC m=+0.337400212 container start d0735e66ce624e88849361b2e50c9e5fff7d887b533e7f31429c897b6e3daa11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True) 2024-04-02T17:10:01.048 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:10:00 smithi059 ceph-mds[70148]: -- 172.21.15.59:0/2037808320 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560537bb3a40 con 0x560537b86800 2024-04-02T17:10:01.049 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:10:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j[70144]: starting mds.j at 2024-04-02T17:10:01.049 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:10:00 smithi059 bash[70128]: d0735e66ce624e88849361b2e50c9e5fff7d887b533e7f31429c897b6e3daa11 2024-04-02T17:10:01.049 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:10:00 smithi059 systemd[1]: Started Ceph mds.j for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4140558793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4140558793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:01.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4140558793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:01.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:02 smithi154 ceph-mon[36479]: Deploying daemon mds.d on smithi059 2024-04-02T17:10:02.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:02 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6834/346805939,v1:172.21.15.59:6835/346805939] up:boot 2024-04-02T17:10:02.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:02 smithi154 ceph-mon[36479]: fsmap 3 up:standby 2024-04-02T17:10:02.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T17:10:02.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:02.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:02 smithi059 ceph-mon[31664]: Deploying daemon mds.d on smithi059 2024-04-02T17:10:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:02 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6834/346805939,v1:172.21.15.59:6835/346805939] up:boot 2024-04-02T17:10:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:02 smithi059 ceph-mon[31664]: fsmap 3 up:standby 2024-04-02T17:10:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T17:10:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:02 smithi164 ceph-mon[36499]: Deploying daemon mds.d on smithi059 2024-04-02T17:10:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:02 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6834/346805939,v1:172.21.15.59:6835/346805939] up:boot 2024-04-02T17:10:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:02 smithi164 ceph-mon[36499]: fsmap 3 up:standby 2024-04-02T17:10:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 -- 172.21.15.59:0/1137734325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 msgr2=0x7fe7241103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 --2- 172.21.15.59:0/1137734325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241103c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ca0 tx=0x7fe71c02c100 comp rx=0 tx=0).stop 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 -- 172.21.15.59:0/1137734325 shutdown_connections 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 --2- 172.21.15.59:0/1137734325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241103c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 --2- 172.21.15.59:0/1137734325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7241090e0 0x7fe724109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 --2- 172.21.15.59:0/1137734325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe724107f10 0x7fe724108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 -- 172.21.15.59:0/1137734325 >> 172.21.15.59:0/1137734325 conn(0x7fe724076f10 msgr2=0x7fe724079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.706+0000 7fe729a00640 1 -- 172.21.15.59:0/1137734325 shutdown_connections 2024-04-02T17:10:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.707+0000 7fe729a00640 1 -- 172.21.15.59:0/1137734325 wait complete. 2024-04-02T17:10:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.708+0000 7fe729a00640 1 Processor -- start 2024-04-02T17:10:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.708+0000 7fe729a00640 1 -- start start 2024-04-02T17:10:02.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241090e0 0x7fe72419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113bb0 con 0x7fe7241090e0 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113a30 con 0x7fe724109aa0 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113d30 con 0x7fe724107f10 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57686/0 (socket says 172.21.15.59:57686) 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 -- 172.21.15.59:0/1364724006 learned_addr learned my addr 172.21.15.59:0/1364724006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723400640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241090e0 0x7fe72419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe729000640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 -- 172.21.15.59:0/1364724006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 msgr2=0x7fe7241a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 -- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241090e0 msgr2=0x7fe72419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241090e0 0x7fe72419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.709+0000 7fe723e00640 1 -- 172.21.15.59:0/1364724006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a5db0 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe723e00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9c0 tx=0x7fe71400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cd80 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7140035f0 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a60a0 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010690 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.710+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a65f0 con 0x7fe724107f10 2024-04-02T17:10:02.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.711+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72410f320 con 0x7fe724107f10 2024-04-02T17:10:02.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.713+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe7140037a0 con 0x7fe724107f10 2024-04-02T17:10:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.715+0000 7fe720c00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 0x7fe7080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.715+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe7140d04b0 con 0x7fe724107f10 2024-04-02T17:10:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.715+0000 7fe723400640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 0x7fe7080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.716+0000 7fe723400640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 0x7fe7080a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7100095f0 tx=0x7fe710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.719+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe71409d290 con 0x7fe724107f10 2024-04-02T17:10:02.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.975+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe72402d370 con 0x7fe724107f10 2024-04-02T17:10:02.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.976+0000 7fe720c00640 1 -- 172.21.15.59:0/1364724006 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fe71409cb90 con 0x7fe724107f10 2024-04-02T17:10:02.977 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:02.977 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T17:10:02.977 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 4 2024-04-02T17:10:02.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 msgr2=0x7fe7080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 0x7fe7080a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7100095f0 tx=0x7fe710005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 msgr2=0x7fe72419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9c0 tx=0x7fe71400ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 shutdown_connections 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7080a3bb0 0x7fe7080a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241090e0 0x7fe72419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724109aa0 0x7fe7241a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 --2- 172.21.15.59:0/1364724006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724107f10 0x7fe72419e190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 >> 172.21.15.59:0/1364724006 conn(0x7fe724076f10 msgr2=0x7fe72410c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 shutdown_connections 2024-04-02T17:10:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:02.979+0000 7fe729a00640 1 -- 172.21.15.59:0/1364724006 wait complete. 2024-04-02T17:10:03.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:03 smithi059 ceph-mon[31664]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:03.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:03 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1364724006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:03 smithi164 ceph-mon[36499]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:03 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1364724006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:03 smithi154 ceph-mon[36479]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:03 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1364724006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:04.800 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 podman[70476]: 2024-04-02 17:10:04.557400419 +0000 UTC m=+0.156935395 container create 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T17:10:05.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:05.081 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 podman[70476]: 2024-04-02 17:10:04.819640289 +0000 UTC m=+0.419175258 container init 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:10:05.082 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 podman[70476]: 2024-04-02 17:10:04.829951881 +0000 UTC m=+0.429486847 container start 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:10:05.082 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 ceph-mds[70495]: -- 172.21.15.59:0/3362606174 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaadfe3a40 con 0x55aaadfb6c00 2024-04-02T17:10:05.082 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: starting mds.d at 2024-04-02T17:10:05.082 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 bash[70476]: 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 2024-04-02T17:10:05.082 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:10:04 smithi059 systemd[1]: Started Ceph mds.d for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.224+0000 7fd2f0a00640 1 -- 172.21.15.59:0/3572955985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 msgr2=0x7fd2ec1097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.224+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/3572955985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec1097b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009ca0 tx=0x7fd2d402c100 comp rx=0 tx=0).stop 2024-04-02T17:10:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 -- 172.21.15.59:0/3572955985 shutdown_connections 2024-04-02T17:10:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/3572955985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec1097b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/3572955985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/3572955985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 0x7fd2ec108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 -- 172.21.15.59:0/3572955985 >> 172.21.15.59:0/3572955985 conn(0x7fd2ec077110 msgr2=0x7fd2ec079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 -- 172.21.15.59:0/3572955985 shutdown_connections 2024-04-02T17:10:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.225+0000 7fd2f0a00640 1 -- 172.21.15.59:0/3572955985 wait complete. 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 Processor -- start 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 -- start start 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 0x7fd2ec19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec113dd0 con 0x7fd2ec109330 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec113c50 con 0x7fd2ec108140 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2f0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec113f50 con 0x7fd2ec109cf0 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 0x7fd2ec19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39836/0 (socket says 172.21.15.59:39836) 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 -- 172.21.15.59:0/455644534 learned_addr learned my addr 172.21.15.59:0/455644534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.226+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec1a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57702/0 (socket says 172.21.15.59:57702) 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 -- 172.21.15.59:0/455644534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 msgr2=0x7fd2ec1a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec1a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 -- 172.21.15.59:0/455644534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 msgr2=0x7fd2ec19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 0x7fd2ec19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 -- 172.21.15.59:0/455644534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec1a6010 con 0x7fd2ec109330 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e9400640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009dc0 tx=0x7fd2d4035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.227+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4039040 con 0x7fd2ec109330 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.228+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d4044070 con 0x7fd2ec109330 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.228+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4004600 con 0x7fd2ec109330 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.228+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec1a62a0 con 0x7fd2ec109330 2024-04-02T17:10:05.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.228+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec1a67e0 con 0x7fd2ec109330 2024-04-02T17:10:05.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.229+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec10f460 con 0x7fd2ec109330 2024-04-02T17:10:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.231+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd2d4032890 con 0x7fd2ec109330 2024-04-02T17:10:05.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.232+0000 7fd2e2a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 0x7fd2cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:05.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.233+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd2d40f2620 con 0x7fd2ec109330 2024-04-02T17:10:05.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.233+0000 7fd2e9e00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 0x7fd2cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:05.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.234+0000 7fd2e9e00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 0x7fd2cc0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec109190 tx=0x7fd2dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:05.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.237+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd2d40bf400 con 0x7fd2ec109330 2024-04-02T17:10:05.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.490+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2ec02d370 con 0x7fd2ec109330 2024-04-02T17:10:05.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.491+0000 7fd2e2a00640 1 -- 172.21.15.59:0/455644534 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fd2d40bed00 con 0x7fd2ec109330 2024-04-02T17:10:05.492 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:05.492 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-02T17:10:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 5 2024-04-02T17:10:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 msgr2=0x7fd2cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 0x7fd2cc0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec109190 tx=0x7fd2dc005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 msgr2=0x7fd2ec19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009dc0 tx=0x7fd2d4035d90 comp rx=0 tx=0).stop 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 shutdown_connections 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd2cc0a3c30 0x7fd2cc0a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec109330 0x7fd2ec19e930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2ec109cf0 0x7fd2ec1a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 --2- 172.21.15.59:0/455644534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2ec108140 0x7fd2ec19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 >> 172.21.15.59:0/455644534 conn(0x7fd2ec077110 msgr2=0x7fd2ec10c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.494+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 shutdown_connections 2024-04-02T17:10:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:05.495+0000 7fd2f0a00640 1 -- 172.21.15.59:0/455644534 wait complete. 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:05.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:05.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:05.593 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:06.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:06 smithi154 ceph-mon[36479]: Deploying daemon mds.e on smithi154 2024-04-02T17:10:06.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:06 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6836/448006801,v1:172.21.15.59:6837/448006801] up:boot 2024-04-02T17:10:06.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:06 smithi154 ceph-mon[36479]: fsmap 4 up:standby 2024-04-02T17:10:06.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:10:06.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:06 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/455644534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:06.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:06 smithi059 ceph-mon[31664]: Deploying daemon mds.e on smithi154 2024-04-02T17:10:06.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:06 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6836/448006801,v1:172.21.15.59:6837/448006801] up:boot 2024-04-02T17:10:06.542 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:06 smithi059 ceph-mon[31664]: fsmap 4 up:standby 2024-04-02T17:10:06.542 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:10:06.542 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:06 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/455644534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:06.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:06 smithi164 ceph-mon[36499]: Deploying daemon mds.e on smithi154 2024-04-02T17:10:06.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:06 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6836/448006801,v1:172.21.15.59:6837/448006801] up:boot 2024-04-02T17:10:06.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:06 smithi164 ceph-mon[36499]: fsmap 4 up:standby 2024-04-02T17:10:06.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:10:06.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:06 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/455644534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:07 smithi154 ceph-mon[36479]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:07.400 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 systemd[1]: Starting Ceph mds.e for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:10:07.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:07.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:07 smithi059 ceph-mon[31664]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:07 smithi164 ceph-mon[36499]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:07.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 -- 172.21.15.59:0/1729197222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 msgr2=0x7fcf201031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 --2- 172.21.15.59:0/1729197222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf201031b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009900 tx=0x7fcf0802bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 -- 172.21.15.59:0/1729197222 shutdown_connections 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 --2- 172.21.15.59:0/1729197222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf201031b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 --2- 172.21.15.59:0/1729197222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 0x7fcf20109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 --2- 172.21.15.59:0/1729197222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf20101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 -- 172.21.15.59:0/1729197222 >> 172.21.15.59:0/1729197222 conn(0x7fcf200fd0b0 msgr2=0x7fcf200ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 -- 172.21.15.59:0/1729197222 shutdown_connections 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.727+0000 7fcf25a00640 1 -- 172.21.15.59:0/1729197222 wait complete. 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.728+0000 7fcf25a00640 1 Processor -- start 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.728+0000 7fcf25a00640 1 -- start start 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf2019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 0x7fcf201a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2010d810 con 0x7fcf20102d30 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2010d690 con 0x7fcf20101b40 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf25a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2010d990 con 0x7fcf201036f0 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf2019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59840/0 (socket says 172.21.15.59:59840) 2024-04-02T17:10:07.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 -- 172.21.15.59:0/2674040653 learned_addr learned my addr 172.21.15.59:0/2674040653 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1f400640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 0x7fcf201a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 -- 172.21.15.59:0/2674040653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 msgr2=0x7fcf201a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 0x7fcf201a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 -- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 msgr2=0x7fcf2019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf2019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.729+0000 7fcf1ea00640 1 -- 172.21.15.59:0/2674040653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf201a1c60 con 0x7fcf20101b40 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf1ea00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9c0 tx=0x7fcf1000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000cd80 con 0x7fcf20101b40 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf100035f0 con 0x7fcf20101b40 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10010690 con 0x7fcf20101b40 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf201a1f50 con 0x7fcf20101b40 2024-04-02T17:10:07.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.730+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf201a23b0 con 0x7fcf20101b40 2024-04-02T17:10:07.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.731+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf201a2640 con 0x7fcf20101b40 2024-04-02T17:10:07.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.733+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcf100037a0 con 0x7fcf20101b40 2024-04-02T17:10:07.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.735+0000 7fcf17400640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 0x7fcf040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:07.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.735+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fcf100d03e0 con 0x7fcf20101b40 2024-04-02T17:10:07.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.735+0000 7fcf17e00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 0x7fcf040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:07.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.736+0000 7fcf17e00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 0x7fcf040a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf0802c240 tx=0x7fcf0802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:07.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.739+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcf1009d1c0 con 0x7fcf20101b40 2024-04-02T17:10:07.794 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 podman[66165]: 2024-04-02 17:10:07.376953131 +0000 UTC m=+0.017957811 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:10:07.794 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 podman[66165]: 2024-04-02 17:10:07.547590855 +0000 UTC m=+0.188595523 container create cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T17:10:07.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.992+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf201a2ef0 con 0x7fcf20101b40 2024-04-02T17:10:07.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:07.997+0000 7fcf17400640 1 -- 172.21.15.59:0/2674040653 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fcf1009cac0 con 0x7fcf20101b40 2024-04-02T17:10:07.998 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:07.998 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-02T17:10:07.998 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 5 2024-04-02T17:10:08.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 msgr2=0x7fcf040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 0x7fcf040a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf0802c240 tx=0x7fcf0802df90 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 msgr2=0x7fcf2019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9c0 tx=0x7fcf1000ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 shutdown_connections 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcf040a3bb0 0x7fcf040a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf20102d30 0x7fcf2019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf201036f0 0x7fcf201a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 --2- 172.21.15.59:0/2674040653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf20101b40 0x7fcf2019a090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 >> 172.21.15.59:0/2674040653 conn(0x7fcf200fd0b0 msgr2=0x7fcf20073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 shutdown_connections 2024-04-02T17:10:08.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:08.001+0000 7fcf25a00640 1 -- 172.21.15.59:0/2674040653 wait complete. 2024-04-02T17:10:08.149 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 podman[66165]: 2024-04-02 17:10:07.795513747 +0000 UTC m=+0.436518407 container init cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:10:08.150 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 podman[66165]: 2024-04-02 17:10:07.798281332 +0000 UTC m=+0.439285999 container start cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:10:08.150 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 ceph-mds[66184]: -- 172.21.15.154:0/3877139214 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581771b1a40 con 0x558177184800 2024-04-02T17:10:08.150 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: starting mds.e at 2024-04-02T17:10:08.150 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 bash[66165]: cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 2024-04-02T17:10:08.150 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:10:07 smithi154 systemd[1]: Started Ceph mds.e for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:09.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2674040653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: Deploying daemon mds.f on smithi164 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6834/1503733829,v1:172.21.15.154:6835/1503733829] up:boot 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: fsmap 5 up:standby 2024-04-02T17:10:09.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:08 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2674040653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:09.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: Deploying daemon mds.f on smithi164 2024-04-02T17:10:09.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6834/1503733829,v1:172.21.15.154:6835/1503733829] up:boot 2024-04-02T17:10:09.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: fsmap 5 up:standby 2024-04-02T17:10:09.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:08 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:10:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2674040653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: Deploying daemon mds.f on smithi164 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6834/1503733829,v1:172.21.15.154:6835/1503733829] up:boot 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: fsmap 5 up:standby 2024-04-02T17:10:09.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:08 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:10:10.024 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 -- 172.21.15.59:0/3927130451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0109aa0 msgr2=0x7ff9f01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/3927130451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01103c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009be0 tx=0x7ff9e802bef0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 -- 172.21.15.59:0/3927130451 shutdown_connections 2024-04-02T17:10:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/3927130451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01103c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/3927130451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f01090e0 0x7ff9f0109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/3927130451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f0107f10 0x7ff9f0108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 -- 172.21.15.59:0/3927130451 >> 172.21.15.59:0/3927130451 conn(0x7ff9f0076f10 msgr2=0x7ff9f0079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 -- 172.21.15.59:0/3927130451 shutdown_connections 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.259+0000 7ff9f4c00640 1 -- 172.21.15.59:0/3927130451 wait complete. 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.260+0000 7ff9f4c00640 1 Processor -- start 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 -- start start 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f01090e0 0x7ff9f019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113c00 con 0x7ff9f01090e0 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113a80 con 0x7ff9f0109aa0 2024-04-02T17:10:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9f4c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113d80 con 0x7ff9f0107f10 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57734/0 (socket says 172.21.15.59:57734) 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9ef400640 1 -- 172.21.15.59:0/1374595685 learned_addr learned my addr 172.21.15.59:0/1374595685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9eea00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f01090e0 0x7ff9f019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.261+0000 7ff9efe00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 -- 172.21.15.59:0/1374595685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f0109aa0 msgr2=0x7ff9f01a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 -- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f01090e0 msgr2=0x7ff9f019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f01090e0 0x7ff9f019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 -- 172.21.15.59:0/1374595685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01a5da0 con 0x7ff9f0107f10 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9ef400640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000e9f0 tx=0x7ff9e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000ccc0 con 0x7ff9f0107f10 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e00035f0 con 0x7ff9f0107f10 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f01a6090 con 0x7ff9f0107f10 2024-04-02T17:10:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.262+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000ce60 con 0x7ff9f0107f10 2024-04-02T17:10:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.263+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01a65e0 con 0x7ff9f0107f10 2024-04-02T17:10:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.264+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f010f2a0 con 0x7ff9f0107f10 2024-04-02T17:10:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.266+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9e00037a0 con 0x7ff9f0107f10 2024-04-02T17:10:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.267+0000 7ff9e7e00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 0x7ff9d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.267+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff9e00d0230 con 0x7ff9f0107f10 2024-04-02T17:10:10.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.267+0000 7ff9eea00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 0x7ff9d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:10.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.268+0000 7ff9eea00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 0x7ff9d40a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9f019f690 tx=0x7ff9dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.273+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff9e009d010 con 0x7ff9f0107f10 2024-04-02T17:10:10.322 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 systemd[1]: Starting Ceph mds.f for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:10:10.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.522+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9f002d370 con 0x7ff9f0107f10 2024-04-02T17:10:10.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.523+0000 7ff9e7e00640 1 -- 172.21.15.59:0/1374595685 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7ff9e009c910 con 0x7ff9f0107f10 2024-04-02T17:10:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T17:10:10.525 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 6 2024-04-02T17:10:10.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 msgr2=0x7ff9d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:10.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 0x7ff9d40a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9f019f690 tx=0x7ff9dc00b390 comp rx=0 tx=0).stop 2024-04-02T17:10:10.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 msgr2=0x7ff9f019e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000e9f0 tx=0x7ff9e000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 shutdown_connections 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9d40a3bb0 0x7ff9d40a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9f01090e0 0x7ff9f019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9f0109aa0 0x7ff9f01a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 --2- 172.21.15.59:0/1374595685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9f0107f10 0x7ff9f019e180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 >> 172.21.15.59:0/1374595685 conn(0x7ff9f0076f10 msgr2=0x7ff9f010c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 shutdown_connections 2024-04-02T17:10:10.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:10.527+0000 7ff9f4c00640 1 -- 172.21.15.59:0/1374595685 wait complete. 2024-04-02T17:10:10.719 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 podman[66908]: 2024-04-02 17:10:10.368912902 +0000 UTC m=+0.017547155 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:10:10.719 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 podman[66908]: 2024-04-02 17:10:10.484129099 +0000 UTC m=+0.132763364 container create 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T17:10:10.719 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 podman[66908]: 2024-04-02 17:10:10.719983825 +0000 UTC m=+0.368618054 container init 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T17:10:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1374595685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 podman[66908]: 2024-04-02 17:10:10.722741261 +0000 UTC m=+0.371375474 container start 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-mds[66927]: -- 172.21.15.164:0/4159775736 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1b6e87a40 con 0x55a1b6e5ac00 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: starting mds.f at 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 bash[66908]: 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de 2024-04-02T17:10:11.073 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:10:10 smithi164 systemd[1]: Started Ceph mds.f for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1374595685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:11.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1374595685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:12.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:11 smithi164 ceph-mon[36499]: Deploying daemon mds.c on smithi164 2024-04-02T17:10:12.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:11 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6834/2083325907,v1:172.21.15.164:6835/2083325907] up:boot 2024-04-02T17:10:12.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:11 smithi164 ceph-mon[36499]: fsmap 6 up:standby 2024-04-02T17:10:12.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:10:12.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:11 smithi059 ceph-mon[31664]: Deploying daemon mds.c on smithi164 2024-04-02T17:10:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:11 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6834/2083325907,v1:172.21.15.164:6835/2083325907] up:boot 2024-04-02T17:10:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:11 smithi059 ceph-mon[31664]: fsmap 6 up:standby 2024-04-02T17:10:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:10:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:11 smithi154 ceph-mon[36479]: Deploying daemon mds.c on smithi164 2024-04-02T17:10:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:11 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6834/2083325907,v1:172.21.15.164:6835/2083325907] up:boot 2024-04-02T17:10:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:11 smithi154 ceph-mon[36479]: fsmap 6 up:standby 2024-04-02T17:10:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:10:12.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 -- 172.21.15.59:0/416017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 msgr2=0x7ff37010c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 --2- 172.21.15.59:0/416017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff37010c180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff368009ca0 tx=0x7ff36802c100 comp rx=0 tx=0).stop 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 -- 172.21.15.59:0/416017586 shutdown_connections 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 --2- 172.21.15.59:0/416017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff37010c180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 --2- 172.21.15.59:0/416017586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370104f30 0x7ff3701053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 --2- 172.21.15.59:0/416017586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370103d40 0x7ff370104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 -- 172.21.15.59:0/416017586 >> 172.21.15.59:0/416017586 conn(0x7ff3700ff2f0 msgr2=0x7ff370101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 -- 172.21.15.59:0/416017586 shutdown_connections 2024-04-02T17:10:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.781+0000 7ff377400640 1 -- 172.21.15.59:0/416017586 wait complete. 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.782+0000 7ff377400640 1 Processor -- start 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.782+0000 7ff377400640 1 -- start start 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370103d40 0x7ff37019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff3701a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010f9a0 con 0x7ff3701058f0 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010f820 con 0x7ff370103d40 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff377400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010fb20 con 0x7ff370104f30 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff367e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff375400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff3701a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff367e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58006/0 (socket says 172.21.15.59:58006) 2024-04-02T17:10:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff367e00640 1 -- 172.21.15.59:0/2760505963 learned_addr learned my addr 172.21.15.59:0/2760505963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.783+0000 7ff374a00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370103d40 0x7ff37019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 -- 172.21.15.59:0/2760505963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370103d40 msgr2=0x7ff37019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370103d40 0x7ff37019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 -- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 msgr2=0x7ff3701a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff3701a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 -- 172.21.15.59:0/2760505963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3701a1c70 con 0x7ff370104f30 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff375400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff3701a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff367e00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff35800d950 tx=0x7ff35800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358017070 con 0x7ff370104f30 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff358009c30 con 0x7ff370104f30 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3701a1f60 con 0x7ff370104f30 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35800f6b0 con 0x7ff370104f30 2024-04-02T17:10:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.784+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701a2470 con 0x7ff370104f30 2024-04-02T17:10:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.786+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37010b000 con 0x7ff370104f30 2024-04-02T17:10:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.788+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff3580109d0 con 0x7ff370104f30 2024-04-02T17:10:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.789+0000 7ff36e000640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 0x7ff3500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.789+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff3580cfae0 con 0x7ff370104f30 2024-04-02T17:10:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.789+0000 7ff374a00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 0x7ff3500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.790+0000 7ff374a00640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 0x7ff3500a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff370104d90 tx=0x7ff360005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:12.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:12.794+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff358009de0 con 0x7ff370104f30 2024-04-02T17:10:13.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.041+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff37002d790 con 0x7ff370104f30 2024-04-02T17:10:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.043+0000 7ff36e000640 1 -- 172.21.15.59:0/2760505963 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7ff35809c270 con 0x7ff370104f30 2024-04-02T17:10:13.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:13.043 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-02T17:10:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 msgr2=0x7ff3500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 0x7ff3500a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff370104d90 tx=0x7ff360005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 msgr2=0x7ff37019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff35800d950 tx=0x7ff35800de10 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 shutdown_connections 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3500a3c30 0x7ff3500a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3701058f0 0x7ff3701a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff370104f30 0x7ff37019a590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 --2- 172.21.15.59:0/2760505963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff370103d40 0x7ff37019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:13.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.045+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 >> 172.21.15.59:0/2760505963 conn(0x7ff3700ff2f0 msgr2=0x7ff3701081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.046+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 shutdown_connections 2024-04-02T17:10:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:13.046+0000 7ff377400640 1 -- 172.21.15.59:0/2760505963 wait complete. 2024-04-02T17:10:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:12 smithi059 ceph-mon[31664]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:12 smithi164 ceph-mon[36499]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:13.322 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 systemd[1]: Starting Ceph mds.c for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:10:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:12 smithi154 ceph-mon[36479]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:13.651 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 podman[67217]: 2024-04-02 17:10:13.23315584 +0000 UTC m=+0.018083912 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:10:13.651 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 podman[67217]: 2024-04-02 17:10:13.353785465 +0000 UTC m=+0.138713519 container create 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T17:10:13.651 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 podman[67217]: 2024-04-02 17:10:13.652057908 +0000 UTC m=+0.436985976 container init 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T17:10:13.994 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 podman[67217]: 2024-04-02 17:10:13.65484302 +0000 UTC m=+0.439771074 container start 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T17:10:13.994 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mds[67235]: -- 172.21.15.164:0/4279468572 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d24ce37a40 con 0x55d24ce0b000 2024-04-02T17:10:13.994 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: starting mds.c at 2024-04-02T17:10:13.994 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 bash[67217]: 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae 2024-04-02T17:10:13.994 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:10:13 smithi164 systemd[1]: Started Ceph mds.c for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2760505963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2760505963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:14.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2760505963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:14.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:15.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:15.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:15 smithi154 ceph-mon[36479]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:15.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:15 smithi154 ceph-mon[36479]: Deploying daemon mds.h on smithi154 2024-04-02T17:10:15.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:15 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:boot 2024-04-02T17:10:15.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:15 smithi154 ceph-mon[36479]: fsmap 7 up:standby 2024-04-02T17:10:15.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:10:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:15 smithi059 ceph-mon[31664]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:15 smithi059 ceph-mon[31664]: Deploying daemon mds.h on smithi154 2024-04-02T17:10:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:15 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:boot 2024-04-02T17:10:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:15 smithi059 ceph-mon[31664]: fsmap 7 up:standby 2024-04-02T17:10:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:10:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.298+0000 7f8136c00640 1 -- 172.21.15.59:0/1369688087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81301036c0 msgr2=0x7f8130109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.298+0000 7f8136c00640 1 --2- 172.21.15.59:0/1369688087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81301036c0 0x7f8130109f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8128009860 tx=0x7f812802bb70 comp rx=0 tx=0).stop 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 -- 172.21.15.59:0/1369688087 shutdown_connections 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 --2- 172.21.15.59:0/1369688087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81301036c0 0x7f8130109f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 --2- 172.21.15.59:0/1369688087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130102d00 0x7f8130103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 --2- 172.21.15.59:0/1369688087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8130101b10 0x7f8130101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 -- 172.21.15.59:0/1369688087 >> 172.21.15.59:0/1369688087 conn(0x7f81300fd0e0 msgr2=0x7f81300ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 -- 172.21.15.59:0/1369688087 shutdown_connections 2024-04-02T17:10:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.299+0000 7f8136c00640 1 -- 172.21.15.59:0/1369688087 wait complete. 2024-04-02T17:10:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 Processor -- start 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 -- start start 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 0x7f813019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81301036c0 0x7f81301a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d770 con 0x7f8130102d00 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d5f0 con 0x7f8130101b10 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.300+0000 7f8136c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d8f0 con 0x7f81301036c0 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 0x7f813019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39208/0 (socket says 172.21.15.59:39208) 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812f400640 1 -- 172.21.15.59:0/990987506 learned_addr learned my addr 172.21.15.59:0/990987506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f8134c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81301036c0 0x7f81301a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 -- 172.21.15.59:0/990987506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81301036c0 msgr2=0x7f81301a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81301036c0 0x7f81301a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 -- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 msgr2=0x7f813019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 -- 172.21.15.59:0/990987506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a1c20 con 0x7f8130101b10 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812f400640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.301+0000 7f812fe00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 0x7f813019a000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8120005fd0 tx=0x7f812000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.302+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120003970 con 0x7f8130101b10 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.302+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8120004030 con 0x7f8130101b10 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.302+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200107f0 con 0x7f8130101b10 2024-04-02T17:10:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.302+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a1f10 con 0x7f8130101b10 2024-04-02T17:10:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.302+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a23d0 con 0x7f8130101b10 2024-04-02T17:10:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.303+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81301a2690 con 0x7f8130101b10 2024-04-02T17:10:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.305+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81200041e0 con 0x7f8130101b10 2024-04-02T17:10:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.306+0000 7f812cc00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 0x7f81140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.306+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f81200d0790 con 0x7f8130101b10 2024-04-02T17:10:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.306+0000 7f812f400640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 0x7f81140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.307+0000 7f812f400640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 0x7f81140a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f81180096c0 tx=0x7f8118009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.310+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f812009d570 con 0x7f8130101b10 2024-04-02T17:10:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:15 smithi164 ceph-mon[36499]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:15 smithi164 ceph-mon[36499]: Deploying daemon mds.h on smithi154 2024-04-02T17:10:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:15 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:boot 2024-04-02T17:10:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:15 smithi164 ceph-mon[36499]: fsmap 7 up:standby 2024-04-02T17:10:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:10:15.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.560+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81301a2860 con 0x7f8130101b10 2024-04-02T17:10:15.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.561+0000 7f812cc00640 1 -- 172.21.15.59:0/990987506 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f812009ce70 con 0x7f8130101b10 2024-04-02T17:10:15.562 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:15.562 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T17:10:15.563 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.564+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 msgr2=0x7f81140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.564+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 0x7f81140a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f81180096c0 tx=0x7f8118009260 comp rx=0 tx=0).stop 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.564+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 msgr2=0x7f813019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.564+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 0x7f813019a000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8120005fd0 tx=0x7f812000ef90 comp rx=0 tx=0).stop 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 shutdown_connections 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81140a3c30 0x7f81140a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8130102d00 0x7f813019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81301036c0 0x7f81301a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 --2- 172.21.15.59:0/990987506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8130101b10 0x7f813019a000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:15.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 >> 172.21.15.59:0/990987506 conn(0x7f81300fd0e0 msgr2=0x7f8130073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:15.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 shutdown_connections 2024-04-02T17:10:15.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:15.565+0000 7f8136c00640 1 -- 172.21.15.59:0/990987506 wait complete. 2024-04-02T17:10:16.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:16 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/990987506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:16.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:16 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/990987506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:16 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/990987506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:16.447 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:10:16 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: starting mds.h at 2024-04-02T17:10:16.900 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:10:16 smithi154 bash[66475]: c5f867ee1cc4ce6cc9bedda3d07a9404099cac71086041ebbaae22349d406ee4 2024-04-02T17:10:16.900 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:10:16 smithi154 systemd[1]: Started Ceph mds.h for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:17.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:17.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:17.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:17.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:17.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 -- 172.21.15.59:0/1326110802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948104d40 msgr2=0x7f99481051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 --2- 172.21.15.59:0/1326110802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948104d40 0x7f99481051c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9934009900 tx=0x7f993402bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 -- 172.21.15.59:0/1326110802 shutdown_connections 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 --2- 172.21.15.59:0/1326110802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948105700 0x7f994810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 --2- 172.21.15.59:0/1326110802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948104d40 0x7f99481051c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 --2- 172.21.15.59:0/1326110802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948103b50 0x7f9948103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 -- 172.21.15.59:0/1326110802 >> 172.21.15.59:0/1326110802 conn(0x7f99480ff0e0 msgr2=0x7f9948101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 -- 172.21.15.59:0/1326110802 shutdown_connections 2024-04-02T17:10:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.808+0000 7f9947e00640 1 -- 172.21.15.59:0/1326110802 wait complete. 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.809+0000 7f9947e00640 1 Processor -- start 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 -- start start 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948104d40 0x7f994819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948105700 0x7f99481a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f800 con 0x7f9948103b50 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f680 con 0x7f9948104d40 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f994810f980 con 0x7f9948105700 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9946a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9946a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39236/0 (socket says 172.21.15.59:39236) 2024-04-02T17:10:17.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9946a00640 1 -- 172.21.15.59:0/4270603504 learned_addr learned my addr 172.21.15.59:0/4270603504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9946000640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948104d40 0x7f994819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.810+0000 7f9947400640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948105700 0x7f99481a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 -- 172.21.15.59:0/4270603504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948105700 msgr2=0x7f99481a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948105700 0x7f99481a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 -- 172.21.15.59:0/4270603504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948104d40 msgr2=0x7f994819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948104d40 0x7f994819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 -- 172.21.15.59:0/4270603504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99481a1a80 con 0x7f9948103b50 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9946a00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f993800e9c0 tx=0x7f993800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993800cde0 con 0x7f9948103b50 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9938010910 con 0x7f9948103b50 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99481a1d70 con 0x7f9948103b50 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938018a00 con 0x7f9948103b50 2024-04-02T17:10:17.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.811+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a21d0 con 0x7f9948103b50 2024-04-02T17:10:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.813+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994810ae10 con 0x7f9948103b50 2024-04-02T17:10:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.815+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f993801f020 con 0x7f9948103b50 2024-04-02T17:10:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.816+0000 7f993f400640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 0x7f992c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.816+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f99380d0260 con 0x7f9948103b50 2024-04-02T17:10:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.816+0000 7f9946000640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 0x7f992c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.817+0000 7f9946000640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 0x7f992c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f993402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:17.821+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f993809d040 con 0x7f9948103b50 2024-04-02T17:10:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.069+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99480627a0 con 0x7f9948103b50 2024-04-02T17:10:18.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.071+0000 7f993f400640 1 -- 172.21.15.59:0/4270603504 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f993809c940 con 0x7f9948103b50 2024-04-02T17:10:18.072 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:18.072 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T17:10:18.072 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 9 2024-04-02T17:10:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.073+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 msgr2=0x7f992c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.073+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 0x7f992c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f993402df90 comp rx=0 tx=0).stop 2024-04-02T17:10:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.073+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 msgr2=0x7f9948199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.073+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f993800e9c0 tx=0x7f993800ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 shutdown_connections 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9948105700 0x7f99481a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9948104d40 0x7f994819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f992c0a3bb0 0x7f992c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 --2- 172.21.15.59:0/4270603504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9948103b50 0x7f9948199e60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 >> 172.21.15.59:0/4270603504 conn(0x7f99480ff0e0 msgr2=0x7f9948107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 shutdown_connections 2024-04-02T17:10:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:18.074+0000 7f9947e00640 1 -- 172.21.15.59:0/4270603504 wait complete. 2024-04-02T17:10:18.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:18 smithi059 ceph-mon[31664]: Deploying daemon mds.i on smithi164 2024-04-02T17:10:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:18 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:boot 2024-04-02T17:10:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:18 smithi059 ceph-mon[31664]: fsmap 8 up:standby 2024-04-02T17:10:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:18 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:18 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:18.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:18 smithi164 ceph-mon[36499]: Deploying daemon mds.i on smithi164 2024-04-02T17:10:18.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:18 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:boot 2024-04-02T17:10:18.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:18 smithi164 ceph-mon[36499]: fsmap 8 up:standby 2024-04-02T17:10:18.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:18 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:18.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:18 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:18 smithi154 ceph-mon[36479]: Deploying daemon mds.i on smithi164 2024-04-02T17:10:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:18 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:boot 2024-04-02T17:10:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:18 smithi154 ceph-mon[36479]: fsmap 8 up:standby 2024-04-02T17:10:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:18 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:18 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:10:19.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:19 smithi059 ceph-mon[31664]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:19.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:19 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4270603504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:19 smithi164 ceph-mon[36499]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:19 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4270603504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:19.323 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:10:19 smithi164 systemd[1]: Started Ceph mds.i for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:19 smithi154 ceph-mon[36479]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:19 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4270603504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:20.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:20.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 -- 172.21.15.59:0/2268472762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c41036d0 msgr2=0x7fb5c4109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:20.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2268472762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c41036d0 0x7fb5c4109f60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009ca0 tx=0x7fb5bc02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 -- 172.21.15.59:0/2268472762 shutdown_connections 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2268472762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c41036d0 0x7fb5c4109f60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2268472762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 0x7fb5c4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2268472762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c4101b20 0x7fb5c4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 -- 172.21.15.59:0/2268472762 >> 172.21.15.59:0/2268472762 conn(0x7fb5c40fd0b0 msgr2=0x7fb5c40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 -- 172.21.15.59:0/2268472762 shutdown_connections 2024-04-02T17:10:20.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.303+0000 7fb5c9600640 1 -- 172.21.15.59:0/2268472762 wait complete. 2024-04-02T17:10:20.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 Processor -- start 2024-04-02T17:10:20.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 -- start start 2024-04-02T17:10:20.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 0x7fb5c4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c41036d0 0x7fb5c4074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d780 con 0x7fb5c4101b20 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d600 con 0x7fb5c41036d0 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.304+0000 7fb5c9600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d900 con 0x7fb5c4102d10 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 0x7fb5c4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39252/0 (socket says 172.21.15.59:39252) 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 -- 172.21.15.59:0/2002377479 learned_addr learned my addr 172.21.15.59:0/2002377479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c3400640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c41036d0 0x7fb5c4074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:20.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 -- 172.21.15.59:0/2002377479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 msgr2=0x7fb5c4073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 0x7fb5c4073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 -- 172.21.15.59:0/2002377479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c41036d0 msgr2=0x7fb5c4074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c41036d0 0x7fb5c4074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.305+0000 7fb5c2a00640 1 -- 172.21.15.59:0/2002377479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c4074cc0 con 0x7fb5c4101b20 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5c2a00640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400e9c0 tx=0x7fb5b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400cde0 con 0x7fb5c4101b20 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b4010910 con 0x7fb5c4101b20 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a1f10 con 0x7fb5c4101b20 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40189d0 con 0x7fb5c4101b20 2024-04-02T17:10:20.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.306+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a2320 con 0x7fb5c4101b20 2024-04-02T17:10:20.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.308+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c4108e40 con 0x7fb5c4101b20 2024-04-02T17:10:20.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.310+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5b4010ac0 con 0x7fb5c4101b20 2024-04-02T17:10:20.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.311+0000 7fb5bb400640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 0x7fb5a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:20.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.311+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb5b40d04f0 con 0x7fb5c4101b20 2024-04-02T17:10:20.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.311+0000 7fb5c2000640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 0x7fb5a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:20.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.312+0000 7fb5c2000640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 0x7fb5a80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4075620 tx=0x7fb5b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.316+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb5b4053f20 con 0x7fb5c4101b20 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:boot 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: fsmap 9 up:standby 2024-04-02T17:10:20.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:20 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:10:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.567+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5c40627d0 con 0x7fb5c4101b20 2024-04-02T17:10:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.569+0000 7fb5bb400640 1 -- 172.21.15.59:0/2002377479 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fb5b409d2d0 con 0x7fb5c4101b20 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 10 2024-04-02T17:10:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 msgr2=0x7fb5a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 0x7fb5a80a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4075620 tx=0x7fb5b000b390 comp rx=0 tx=0).stop 2024-04-02T17:10:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 msgr2=0x7fb5c40758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400e9c0 tx=0x7fb5b400ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 shutdown_connections 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5c41036d0 0x7fb5c4074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5c4102d10 0x7fb5c4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb5a80a3bb0 0x7fb5a80a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 --2- 172.21.15.59:0/2002377479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5c4101b20 0x7fb5c40758b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 >> 172.21.15.59:0/2002377479 conn(0x7fb5c40fd0b0 msgr2=0x7fb5c4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 shutdown_connections 2024-04-02T17:10:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:20.572+0000 7fb5c9600640 1 -- 172.21.15.59:0/2002377479 wait complete. 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:20.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:boot 2024-04-02T17:10:20.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: fsmap 9 up:standby 2024-04-02T17:10:20.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:20 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:boot 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: fsmap 9 up:standby 2024-04-02T17:10:20.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:20 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:10:21.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:21 smithi154 ceph-mon[36479]: Deploying daemon mds.g on smithi059 2024-04-02T17:10:21.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:21 smithi154 ceph-mon[36479]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:21.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:21 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2002377479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:21 smithi164 ceph-mon[36499]: Deploying daemon mds.g on smithi059 2024-04-02T17:10:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:21 smithi164 ceph-mon[36499]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:21 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2002377479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:21.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:21 smithi059 ceph-mon[31664]: Deploying daemon mds.g on smithi059 2024-04-02T17:10:21.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:21 smithi059 ceph-mon[31664]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:21.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:21 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2002377479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:22.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 -- 172.21.15.59:0/3286493710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 msgr2=0x7f86501053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 --2- 172.21.15.59:0/3286493710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f86501053b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f863c009900 tx=0x7f863c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 -- 172.21.15.59:0/3286493710 shutdown_connections 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 --2- 172.21.15.59:0/3286493710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f865010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 --2- 172.21.15.59:0/3286493710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f86501053b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 --2- 172.21.15.59:0/3286493710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 0x7f8650104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.799+0000 7f8656c00640 1 -- 172.21.15.59:0/3286493710 >> 172.21.15.59:0/3286493710 conn(0x7f86500ff2f0 msgr2=0x7f8650101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.800+0000 7f8656c00640 1 -- 172.21.15.59:0/3286493710 shutdown_connections 2024-04-02T17:10:22.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.800+0000 7f8656c00640 1 -- 172.21.15.59:0/3286493710 wait complete. 2024-04-02T17:10:22.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 Processor -- start 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 -- start start 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 0x7f865019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f86501a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f865010f9f0 con 0x7f86501058f0 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f865010f870 con 0x7f8650104f30 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f8656c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f865010fb70 con 0x7f8650103d40 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f864f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f864f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54380/0 (socket says 172.21.15.59:54380) 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.801+0000 7f864f400640 1 -- 172.21.15.59:0/2893224918 learned_addr learned my addr 172.21.15.59:0/2893224918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864fe00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 0x7f865019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f8654c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f86501a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 -- 172.21.15.59:0/2893224918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 msgr2=0x7f865019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:22.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 0x7f865019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 -- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 msgr2=0x7f86501a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f86501a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 -- 172.21.15.59:0/2893224918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501a1c80 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f8654c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f86501a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.802+0000 7f864f400640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.803+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c043020 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.803+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863c0054e0 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.803+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c003c40 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.803+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501a1f10 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.803+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501a2380 con 0x7f8650104f30 2024-04-02T17:10:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.804+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865010b060 con 0x7f8650104f30 2024-04-02T17:10:22.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.806+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f863c046020 con 0x7f8650104f30 2024-04-02T17:10:22.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.807+0000 7f864cc00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 0x7f86340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:22.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.807+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f863c0f26a0 con 0x7f8650104f30 2024-04-02T17:10:22.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.807+0000 7f864fe00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 0x7f86340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:22.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.808+0000 7f864fe00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 0x7f86340a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86400095f0 tx=0x7f8640005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:22.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:22.811+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f863c04d020 con 0x7f8650104f30 2024-04-02T17:10:23.049 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:22 smithi059 podman[71094]: 2024-04-02 17:10:22.818802393 +0000 UTC m=+0.149328316 container create 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:10:23.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.076+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865002d530 con 0x7f8650104f30 2024-04-02T17:10:23.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.077+0000 7f864cc00640 1 -- 172.21.15.59:0/2893224918 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f863c0bf480 con 0x7f8650104f30 2024-04-02T17:10:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:23.077 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T17:10:23.077 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 10 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 msgr2=0x7f86340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 0x7f86340a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86400095f0 tx=0x7f8640005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 msgr2=0x7f865019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c02c650 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 shutdown_connections 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f86340a3c30 0x7f86340a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86501058f0 0x7f86501a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8650104f30 0x7f865019a5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 --2- 172.21.15.59:0/2893224918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8650103d40 0x7f865019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 >> 172.21.15.59:0/2893224918 conn(0x7f86500ff2f0 msgr2=0x7f8650108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 shutdown_connections 2024-04-02T17:10:23.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:23.080+0000 7f8656c00640 1 -- 172.21.15.59:0/2893224918 wait complete. 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 podman[71094]: 2024-04-02 17:10:23.055169718 +0000 UTC m=+0.385695642 container init 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 podman[71094]: 2024-04-02 17:10:23.066377341 +0000 UTC m=+0.396903264 container start 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-mds[71141]: -- 172.21.15.59:0/1279826380 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae02e87a40 con 0x55ae02e5b000 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: starting mds.g at 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 bash[71094]: 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a 2024-04-02T17:10:23.314 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:10:23 smithi059 systemd[1]: Started Ceph mds.g for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:23.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:23 smithi164 ceph-mon[36499]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:23.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:23 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2893224918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:23.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:23.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:23 smithi154 ceph-mon[36479]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:23 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2893224918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:23.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-mon[31664]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:23.682 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2893224918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:23.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:23.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:boot 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: fsmap 10 up:standby 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:boot 2024-04-02T17:10:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: fsmap 10 up:standby 2024-04-02T17:10:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:10:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:24.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:24.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:24.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:boot 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: fsmap 10 up:standby 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:24.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:25.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 -- 172.21.15.59:0/3313921965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109330 msgr2=0x7f00b41097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 --2- 172.21.15.59:0/3313921965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109330 0x7f00b41097b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009900 tx=0x7f00a002bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 -- 172.21.15.59:0/3313921965 shutdown_connections 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 --2- 172.21.15.59:0/3313921965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109330 0x7f00b41097b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 --2- 172.21.15.59:0/3313921965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4109cf0 0x7f00b4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 --2- 172.21.15.59:0/3313921965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4108140 0x7f00b4108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 -- 172.21.15.59:0/3313921965 >> 172.21.15.59:0/3313921965 conn(0x7f00b4077110 msgr2=0x7f00b4079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 -- 172.21.15.59:0/3313921965 shutdown_connections 2024-04-02T17:10:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.344+0000 7f00ba200640 1 -- 172.21.15.59:0/3313921965 wait complete. 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.345+0000 7f00ba200640 1 Processor -- start 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.345+0000 7f00ba200640 1 -- start start 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4108140 0x7f00b419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 0x7f00b419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4113dd0 con 0x7f00b4109cf0 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4113c50 con 0x7f00b4109330 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00ba200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4113f50 con 0x7f00b4108140 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48284/0 (socket says 172.21.15.59:48284) 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00b3e00640 1 -- 172.21.15.59:0/2299098866 learned_addr learned my addr 172.21.15.59:0/2299098866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00b2a00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 0x7f00b419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.346+0000 7f00b3400640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4108140 0x7f00b419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 -- 172.21.15.59:0/2299098866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4108140 msgr2=0x7f00b419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4108140 0x7f00b419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 -- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 msgr2=0x7f00b41a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 -- 172.21.15.59:0/2299098866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b41a6010 con 0x7f00b4109330 2024-04-02T17:10:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b3e00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00b2a00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 0x7f00b419e930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009230 tx=0x7f00a002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0041070 con 0x7f00b4109330 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.347+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b41a62a0 con 0x7f00b4109330 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.348+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a0004450 con 0x7f00b4109330 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.348+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41a67e0 con 0x7f00b4109330 2024-04-02T17:10:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.348+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0003df0 con 0x7f00b4109330 2024-04-02T17:10:25.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.349+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b410f400 con 0x7f00b4109330 2024-04-02T17:10:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.351+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00a0044070 con 0x7f00b4109330 2024-04-02T17:10:25.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.353+0000 7f00abe00640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 0x7f00980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.353+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f00a00f1a20 con 0x7f00b4109330 2024-04-02T17:10:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.353+0000 7f00b3400640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 0x7f00980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:25.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.354+0000 7f00b3400640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 0x7f00980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f00b4109190 tx=0x7f00a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:25.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.357+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f00a00be1b0 con 0x7f00b4109330 2024-04-02T17:10:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:25 smithi164 ceph-mon[36499]: Deploying daemon mds.a on smithi059 2024-04-02T17:10:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:25 smithi164 ceph-mon[36499]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:25.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.630+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00b4062700 con 0x7f00b4109330 2024-04-02T17:10:25.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.631+0000 7f00abe00640 1 -- 172.21.15.59:0/2299098866 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f00a00c24b0 con 0x7f00b4109330 2024-04-02T17:10:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:25.631 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-02T17:10:25.631 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 11 2024-04-02T17:10:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 msgr2=0x7f00980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 0x7f00980a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f00b4109190 tx=0x7f00a4005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 msgr2=0x7f00b419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 0x7f00b419e930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009230 tx=0x7f00a002c650 comp rx=0 tx=0).stop 2024-04-02T17:10:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 shutdown_connections 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00980a3bb0 0x7f00980a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b4109cf0 0x7f00b41a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4109330 0x7f00b419e930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 --2- 172.21.15.59:0/2299098866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00b4108140 0x7f00b419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 >> 172.21.15.59:0/2299098866 conn(0x7f00b4077110 msgr2=0x7f00b410c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 shutdown_connections 2024-04-02T17:10:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:25.634+0000 7f00ba200640 1 -- 172.21.15.59:0/2299098866 wait complete. 2024-04-02T17:10:25.648 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:25 smithi059 ceph-mon[31664]: Deploying daemon mds.a on smithi059 2024-04-02T17:10:25.649 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:25 smithi059 ceph-mon[31664]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:25 smithi154 ceph-mon[36479]: Deploying daemon mds.a on smithi059 2024-04-02T17:10:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:25 smithi154 ceph-mon[36479]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:26.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:26 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2299098866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:26.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:26 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2299098866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:26.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:26 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2299098866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:26.927 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:26 smithi059 podman[71468]: 2024-04-02 17:10:26.697360511 +0000 UTC m=+0.162166594 container create 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-04-02T17:10:26.928 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:26 smithi059 podman[71468]: 2024-04-02 17:10:26.926499829 +0000 UTC m=+0.391305932 container init 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T17:10:27.299 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:26 smithi059 podman[71468]: 2024-04-02 17:10:26.937043846 +0000 UTC m=+0.401849902 container start 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T17:10:27.299 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:26 smithi059 ceph-mds[71487]: -- 172.21.15.59:0/3255399027 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561335039a40 con 0x56133500cc00 2024-04-02T17:10:27.299 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:26 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a[71483]: starting mds.a at 2024-04-02T17:10:27.299 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:27 smithi059 bash[71468]: 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee 2024-04-02T17:10:27.299 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:10:27 smithi059 systemd[1]: Started Ceph mds.a for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:27.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:27 smithi154 ceph-mon[36479]: Deploying daemon mds.b on smithi154 2024-04-02T17:10:27.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:27.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:27.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:27.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:27.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:27.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:27 smithi059 ceph-mon[31664]: Deploying daemon mds.b on smithi154 2024-04-02T17:10:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:27.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:27 smithi164 ceph-mon[36499]: Deploying daemon mds.b on smithi154 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 -- 172.21.15.59:0/3211738606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8103b50 msgr2=0x7f33c8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 --2- 172.21.15.59:0/3211738606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8103b50 0x7f33c8103f70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009900 tx=0x7f33b802bc40 comp rx=0 tx=0).stop 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 -- 172.21.15.59:0/3211738606 shutdown_connections 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 --2- 172.21.15.59:0/3211738606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8104d40 0x7f33c81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 --2- 172.21.15.59:0/3211738606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 0x7f33c810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 --2- 172.21.15.59:0/3211738606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8103b50 0x7f33c8103f70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 -- 172.21.15.59:0/3211738606 >> 172.21.15.59:0/3211738606 conn(0x7f33c80ff0e0 msgr2=0x7f33c8101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 -- 172.21.15.59:0/3211738606 shutdown_connections 2024-04-02T17:10:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.898+0000 7f33cd400640 1 -- 172.21.15.59:0/3211738606 wait complete. 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 Processor -- start 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 -- start start 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8103b50 0x7f33c8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 0x7f33c81a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810f810 con 0x7f33c8103b50 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810f690 con 0x7f33c8105700 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.900+0000 7f33cd400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810f990 con 0x7f33c8104d40 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8103b50 0x7f33c8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47096/0 (socket says 172.21.15.59:47096) 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 -- 172.21.15.59:0/2417192595 learned_addr learned my addr 172.21.15.59:0/2417192595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33cca00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 0x7f33c81a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 -- 172.21.15.59:0/2417192595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 msgr2=0x7f33c81a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 0x7f33c81a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 -- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8103b50 msgr2=0x7f33c8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8103b50 0x7f33c8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 -- 172.21.15.59:0/2417192595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c81a1ae0 con 0x7f33c8104d40 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.901+0000 7f33bfe00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.902+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000cd80 con 0x7f33c8104d40 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.902+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0005500 con 0x7f33c8104d40 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.902+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0010690 con 0x7f33c8104d40 2024-04-02T17:10:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.902+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c81a1dd0 con 0x7f33c8104d40 2024-04-02T17:10:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.902+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81a2290 con 0x7f33c8104d40 2024-04-02T17:10:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.903+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c810ae90 con 0x7f33c8104d40 2024-04-02T17:10:27.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.905+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f33b00046c0 con 0x7f33c8104d40 2024-04-02T17:10:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.906+0000 7f33c5600640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 0x7f33ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.907+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f33b00d02e0 con 0x7f33c8104d40 2024-04-02T17:10:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.907+0000 7f33c7e00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 0x7f33ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.907+0000 7f33c7e00640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 0x7f33ac0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f33b802c180 tx=0x7f33b802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:27.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:27.909+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f33b009d0c0 con 0x7f33c8104d40 2024-04-02T17:10:28.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.154+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c802d5c0 con 0x7f33c8104d40 2024-04-02T17:10:28.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.155+0000 7f33c5600640 1 -- 172.21.15.59:0/2417192595 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f33b009c9c0 con 0x7f33c8104d40 2024-04-02T17:10:28.155 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:28.155 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-02T17:10:28.155 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 12 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 msgr2=0x7f33ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 0x7f33ac0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f33b802c180 tx=0x7f33b802cf90 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 msgr2=0x7f33c819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000edf0 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 shutdown_connections 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33c8105700 0x7f33c81a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c8104d40 0x7f33c819a400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f33ac0a3c30 0x7f33ac0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 --2- 172.21.15.59:0/2417192595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c8103b50 0x7f33c8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 >> 172.21.15.59:0/2417192595 conn(0x7f33c80ff0e0 msgr2=0x7f33c8108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 shutdown_connections 2024-04-02T17:10:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:28.159+0000 7f33cd400640 1 -- 172.21.15.59:0/2417192595 wait complete. 2024-04-02T17:10:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:28 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6840/195975414,v1:172.21.15.59:6841/195975414] up:boot 2024-04-02T17:10:28.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:28 smithi059 ceph-mon[31664]: fsmap 11 up:standby 2024-04-02T17:10:28.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T17:10:28.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:28 smithi059 ceph-mon[31664]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:28.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:28 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2417192595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:28.621 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:28 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6840/195975414,v1:172.21.15.59:6841/195975414] up:boot 2024-04-02T17:10:28.621 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:28 smithi154 ceph-mon[36479]: fsmap 11 up:standby 2024-04-02T17:10:28.621 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T17:10:28.621 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:28 smithi154 ceph-mon[36479]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:28.621 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:28 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2417192595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:28.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:28 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6840/195975414,v1:172.21.15.59:6841/195975414] up:boot 2024-04-02T17:10:28.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:28 smithi164 ceph-mon[36499]: fsmap 11 up:standby 2024-04-02T17:10:28.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T17:10:28.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:28 smithi164 ceph-mon[36499]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:28.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:28 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2417192595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:29.590 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 systemd[1]: Starting Ceph mds.b for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 podman[66802]: 2024-04-02 17:10:29.591027207 +0000 UTC m=+0.107512093 container create 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 podman[66802]: 2024-04-02 17:10:29.501387192 +0000 UTC m=+0.017872079 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 podman[66802]: 2024-04-02 17:10:29.793332006 +0000 UTC m=+0.309816892 container init 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 podman[66802]: 2024-04-02 17:10:29.795966749 +0000 UTC m=+0.312451643 container start 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 ceph-mds[66821]: -- 172.21.15.154:0/3052982454 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587ae939a40 con 0x5587ae90cc00 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: starting mds.b at 2024-04-02T17:10:29.876 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 bash[66802]: 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc 2024-04-02T17:10:30.149 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:29 smithi154 systemd[1]: Started Ceph mds.b for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:10:30.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 -- 172.21.15.59:0/3357410517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 msgr2=0x7f8e68078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 --2- 172.21.15.59:0/3357410517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e68078940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009be0 tx=0x7f8e6002bef0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 -- 172.21.15.59:0/3357410517 shutdown_connections 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 --2- 172.21.15.59:0/3357410517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e68078940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 --2- 172.21.15.59:0/3357410517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e68077700 0x7f8e68077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 --2- 172.21.15.59:0/3357410517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e680764c0 0x7f8e680768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 -- 172.21.15.59:0/3357410517 >> 172.21.15.59:0/3357410517 conn(0x7f8e680ff2f0 msgr2=0x7f8e68101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 -- 172.21.15.59:0/3357410517 shutdown_connections 2024-04-02T17:10:30.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.426+0000 7f8e6e200640 1 -- 172.21.15.59:0/3357410517 wait complete. 2024-04-02T17:10:30.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.427+0000 7f8e6e200640 1 Processor -- start 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 -- start start 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e68077700 0x7f8e6819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e681a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68113e10 con 0x7f8e680780a0 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68113c90 con 0x7f8e680764c0 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e6e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68113f90 con 0x7f8e68077700 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e67400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e67400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54482/0 (socket says 172.21.15.59:54482) 2024-04-02T17:10:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e67e00640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e681a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e67400640 1 -- 172.21.15.59:0/2730057616 learned_addr learned my addr 172.21.15.59:0/2730057616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.428+0000 7f8e66a00640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e68077700 0x7f8e6819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 -- 172.21.15.59:0/2730057616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e68077700 msgr2=0x7f8e6819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e68077700 0x7f8e6819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 -- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 msgr2=0x7f8e681a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e681a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 -- 172.21.15.59:0/2730057616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e681a5fb0 con 0x7f8e680764c0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e67400640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9c0 tx=0x7f8e5800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800ccc0 con 0x7f8e680764c0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58005500 con 0x7f8e680764c0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58010430 con 0x7f8e680764c0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.429+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e681a62a0 con 0x7f8e680764c0 2024-04-02T17:10:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.430+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e681a67a0 con 0x7f8e680764c0 2024-04-02T17:10:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.431+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e681107c0 con 0x7f8e680764c0 2024-04-02T17:10:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.432+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e580046c0 con 0x7f8e680764c0 2024-04-02T17:10:30.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.434+0000 7f8e5fe00640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 0x7f8e4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.434+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f8e580d0240 con 0x7f8e680764c0 2024-04-02T17:10:30.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.434+0000 7f8e66a00640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 0x7f8e4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.435+0000 7f8e66a00640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 0x7f8e4c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e6819f8a0 tx=0x7f8e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:30.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.438+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e5809d020 con 0x7f8e680764c0 2024-04-02T17:10:30.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.693+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e68062730 con 0x7f8e680764c0 2024-04-02T17:10:30.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.694+0000 7f8e5fe00640 1 -- 172.21.15.59:0/2730057616 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f8e5809c920 con 0x7f8e680764c0 2024-04-02T17:10:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:30.694 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-02T17:10:30.694 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 13 2024-04-02T17:10:30.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 msgr2=0x7f8e4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 0x7f8e4c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e6819f8a0 tx=0x7f8e54005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 msgr2=0x7f8e6819e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9c0 tx=0x7f8e5800ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 shutdown_connections 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8e4c0a3c30 0x7f8e4c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e680780a0 0x7f8e681a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e68077700 0x7f8e6819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 --2- 172.21.15.59:0/2730057616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e680764c0 0x7f8e6819e390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.697+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 >> 172.21.15.59:0/2730057616 conn(0x7f8e680ff2f0 msgr2=0x7f8e68100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:30.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.698+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 shutdown_connections 2024-04-02T17:10:30.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.698+0000 7f8e6e200640 1 -- 172.21.15.59:0/2730057616 wait complete. 2024-04-02T17:10:30.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:10:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 -- 172.21.15.59:0/4275419899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110104d60 msgr2=0x7fe110107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 --2- 172.21.15.59:0/4275419899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110104d60 0x7fe110107150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009ca0 tx=0x7fe0fc02c100 comp rx=0 tx=0).stop 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 -- 172.21.15.59:0/4275419899 shutdown_connections 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 --2- 172.21.15.59:0/4275419899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110104d60 0x7fe110107150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 --2- 172.21.15.59:0/4275419899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110107690 0x7fe110109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 --2- 172.21.15.59:0/4275419899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110102430 0x7fe110104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 -- 172.21.15.59:0/4275419899 >> 172.21.15.59:0/4275419899 conn(0x7fe1100fbde0 msgr2=0x7fe1100fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 -- 172.21.15.59:0/4275419899 shutdown_connections 2024-04-02T17:10:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.918+0000 7fe116e00640 1 -- 172.21.15.59:0/4275419899 wait complete. 2024-04-02T17:10:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 Processor -- start 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 -- start start 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 0x7fe1101981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 0x7fe11019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010d5c0 con 0x7fe110102430 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010d440 con 0x7fe110104d60 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.919+0000 7fe116e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe11010d740 con 0x7fe110107690 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40036/0 (socket says 172.21.15.59:40036) 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 0x7fe1101981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115a00640 1 -- 172.21.15.59:0/602433876 learned_addr learned my addr 172.21.15.59:0/602433876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe116400640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 0x7fe11019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 -- 172.21.15.59:0/602433876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 msgr2=0x7fe11019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 0x7fe11019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 -- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 msgr2=0x7fe110197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 -- 172.21.15.59:0/602433876 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11019f890 con 0x7fe110104d60 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe116400640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 0x7fe11019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115a00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.920+0000 7fe115000640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 0x7fe1101981c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02c640 tx=0x7fe0fc035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.921+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc039040 con 0x7fe110104d60 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.921+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc003970 con 0x7fe110104d60 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.921+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004310 con 0x7fe110104d60 2024-04-02T17:10:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.921+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11019fb20 con 0x7fe110104d60 2024-04-02T17:10:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.921+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11019ffc0 con 0x7fe110104d60 2024-04-02T17:10:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.922+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1100fd8d0 con 0x7fe110104d60 2024-04-02T17:10:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.924+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe0fc046030 con 0x7fe110104d60 2024-04-02T17:10:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.925+0000 7fe10e000640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 0x7fe0f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.925+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe0fc0f1d50 con 0x7fe110104d60 2024-04-02T17:10:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.925+0000 7fe115a00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 0x7fe0f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:30.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.926+0000 7fe115a00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 0x7fe0f40a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe110199150 tx=0x7fe100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:30.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:30.929+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe0fc04d060 con 0x7fe110104d60 2024-04-02T17:10:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:boot 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: fsmap 12 up:standby 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:10:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:30 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2730057616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.176+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe110062700 con 0x7fe110104d60 2024-04-02T17:10:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.177+0000 7fe10e000640 1 -- 172.21.15.59:0/602433876 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18458 (secure 0 0 0) 0x7fe0fc0beb30 con 0x7fe110104d60 2024-04-02T17:10:31.177 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:31.177 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:09:37.821176+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:10:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 msgr2=0x7fe0f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 0x7fe0f40a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe110199150 tx=0x7fe100005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 msgr2=0x7fe1101981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 0x7fe1101981c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02c640 tx=0x7fe0fc035d90 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 shutdown_connections 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe110107690 0x7fe11019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe110104d60 0x7fe1101981c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0f40a3bb0 0x7fe0f40a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 --2- 172.21.15.59:0/602433876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe110102430 0x7fe110197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 >> 172.21.15.59:0/602433876 conn(0x7fe1100fbde0 msgr2=0x7fe110102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 shutdown_connections 2024-04-02T17:10:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.180+0000 7fe116e00640 1 -- 172.21.15.59:0/602433876 wait complete. 2024-04-02T17:10:31.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:10:31.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:boot 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: fsmap 12 up:standby 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:10:31.206 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:30 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2730057616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.243 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:boot 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: fsmap 12 up:standby 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:10:31.244 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:30 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2730057616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:31.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 -- 172.21.15.59:0/4070406979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec104d00 msgr2=0x7f28ec105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 --2- 172.21.15.59:0/4070406979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec104d00 0x7f28ec105180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009860 tx=0x7f28d802bb70 comp rx=0 tx=0).stop 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 -- 172.21.15.59:0/4070406979 shutdown_connections 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 --2- 172.21.15.59:0/4070406979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec104d00 0x7f28ec105180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 --2- 172.21.15.59:0/4070406979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec1056c0 0x7f28ec10bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 --2- 172.21.15.59:0/4070406979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 -- 172.21.15.59:0/4070406979 >> 172.21.15.59:0/4070406979 conn(0x7f28ec0ff080 msgr2=0x7f28ec1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 -- 172.21.15.59:0/4070406979 shutdown_connections 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.390+0000 7f28ebe00640 1 -- 172.21.15.59:0/4070406979 wait complete. 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.391+0000 7f28ebe00640 1 Processor -- start 2024-04-02T17:10:31.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 -- start start 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec104d00 0x7f28ec19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1056c0 0x7f28ec1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec10f790 con 0x7f28ec1056c0 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec10f610 con 0x7f28ec104d00 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ebe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec10f910 con 0x7f28ec103b10 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48166/0 (socket says 172.21.15.59:48166) 2024-04-02T17:10:31.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eaa00640 1 -- 172.21.15.59:0/1069719628 learned_addr learned my addr 172.21.15.59:0/1069719628 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eb400640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1056c0 0x7f28ec1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28ea000640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec104d00 0x7f28ec19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eaa00640 1 -- 172.21.15.59:0/1069719628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec104d00 msgr2=0x7f28ec19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.392+0000 7f28eaa00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec104d00 0x7f28ec19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28eaa00640 1 -- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1056c0 msgr2=0x7f28ec1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28eaa00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1056c0 0x7f28ec1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28eaa00640 1 -- 172.21.15.59:0/1069719628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec1a1a80 con 0x7f28ec103b10 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28eaa00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9c0 tx=0x7f28dc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00cd80 con 0x7f28ec103b10 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28dc0035f0 con 0x7f28ec103b10 2024-04-02T17:10:31.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec1a1d70 con 0x7f28ec103b10 2024-04-02T17:10:31.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc010690 con 0x7f28ec103b10 2024-04-02T17:10:31.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.393+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a2180 con 0x7f28ec103b10 2024-04-02T17:10:31.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.395+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec10ae80 con 0x7f28ec103b10 2024-04-02T17:10:31.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.397+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f28dc0037a0 con 0x7f28ec103b10 2024-04-02T17:10:31.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.398+0000 7f28f0a00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 0x7f28d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.398+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f28dc0d0360 con 0x7f28ec103b10 2024-04-02T17:10:31.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.398+0000 7f28ea000640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 0x7f28d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.399+0000 7f28ea000640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 0x7f28d00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009830 tx=0x7f28d8034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:31.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.402+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f28dc09d140 con 0x7f28ec103b10 2024-04-02T17:10:31.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.612+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28ec0627a0 con 0x7f28ec103b10 2024-04-02T17:10:31.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.613+0000 7f28f0a00640 1 -- 172.21.15.59:0/1069719628 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f28dc09ca40 con 0x7f28ec103b10 2024-04-02T17:10:31.612 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:10:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 msgr2=0x7f28d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 0x7f28d00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009830 tx=0x7f28d8034ab0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 msgr2=0x7f28ec199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9c0 tx=0x7f28dc00ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 shutdown_connections 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f28d00a3bb0 0x7f28d00a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1056c0 0x7f28ec1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ec104d00 0x7f28ec19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 --2- 172.21.15.59:0/1069719628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec103b10 0x7f28ec199e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 >> 172.21.15.59:0/1069719628 conn(0x7f28ec0ff080 msgr2=0x7f28ec108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 shutdown_connections 2024-04-02T17:10:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.617+0000 7f28ebe00640 1 -- 172.21.15.59:0/1069719628 wait complete. 2024-04-02T17:10:31.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 -- 172.21.15.59:0/4083278196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 msgr2=0x7f6264078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 --2- 172.21.15.59:0/4083278196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f6264078940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f625c009ca0 tx=0x7f625c02c100 comp rx=0 tx=0).stop 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 -- 172.21.15.59:0/4083278196 shutdown_connections 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 --2- 172.21.15.59:0/4083278196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f6264078940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 --2- 172.21.15.59:0/4083278196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6264077700 0x7f6264077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 --2- 172.21.15.59:0/4083278196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62640764c0 0x7f62640768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 -- 172.21.15.59:0/4083278196 >> 172.21.15.59:0/4083278196 conn(0x7f62640ff0e0 msgr2=0x7f6264101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 -- 172.21.15.59:0/4083278196 shutdown_connections 2024-04-02T17:10:31.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.837+0000 7f6263200640 1 -- 172.21.15.59:0/4083278196 wait complete. 2024-04-02T17:10:31.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.838+0000 7f6263200640 1 Processor -- start 2024-04-02T17:10:31.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.838+0000 7f6263200640 1 -- start start 2024-04-02T17:10:31.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 0x7f626419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6264077700 0x7f626419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f62641a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6264113be0 con 0x7f62640780a0 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6264113a60 con 0x7f6264077700 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6263200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6264113d60 con 0x7f62640764c0 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f62641a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6261e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 0x7f626419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6261e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 0x7f626419e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48174/0 (socket says 172.21.15.59:48174) 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6261e00640 1 -- 172.21.15.59:0/1885732771 learned_addr learned my addr 172.21.15.59:0/1885732771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6261400640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6264077700 0x7f626419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 -- 172.21.15.59:0/1885732771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 msgr2=0x7f626419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 0x7f626419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 -- 172.21.15.59:0/1885732771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6264077700 msgr2=0x7f626419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6264077700 0x7f626419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.839+0000 7f6262800640 1 -- 172.21.15.59:0/1885732771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62641a5dd0 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f6262800640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f62641a5760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f625c037040 tx=0x7f625c035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c039040 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c044070 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c004350 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62641a6060 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.840+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a64d0 con 0x7f62640780a0 2024-04-02T17:10:31.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.841+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6264110600 con 0x7f62640780a0 2024-04-02T17:10:31.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.843+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f625c0044f0 con 0x7f62640780a0 2024-04-02T17:10:31.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.844+0000 7f625aa00640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 0x7f62480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.845+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f625c0f24d0 con 0x7f62640780a0 2024-04-02T17:10:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.845+0000 7f6261e00640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 0x7f62480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.845+0000 7f6261e00640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 0x7f62480a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6254009c90 tx=0x7f6254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:31.850+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f625c0bf2b0 con 0x7f62640780a0 2024-04-02T17:10:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.079+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f62640627a0 con 0x7f62640780a0 2024-04-02T17:10:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.080+0000 7f625aa00640 1 -- 172.21.15.59:0/1885732771 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f625c0098b0 con 0x7f62640780a0 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:10:32.080 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 msgr2=0x7f62480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 0x7f62480a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6254009c90 tx=0x7f6254005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 msgr2=0x7f62641a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f62641a5760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f625c037040 tx=0x7f625c035d10 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 shutdown_connections 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62480a3c30 0x7f62480a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62640780a0 0x7f62641a5760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6264077700 0x7f626419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 --2- 172.21.15.59:0/1885732771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62640764c0 0x7f626419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 >> 172.21.15.59:0/1885732771 conn(0x7f62640ff0e0 msgr2=0x7f6264100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 shutdown_connections 2024-04-02T17:10:32.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.083+0000 7f6263200640 1 -- 172.21.15.59:0/1885732771 wait complete. 2024-04-02T17:10:32.103 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T17:10:32.103 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T17:10:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:31 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/602433876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:31 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1069719628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:10:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:32.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:31 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/602433876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:32.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:31 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1069719628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:10:32.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 -- 172.21.15.59:0/4066406921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 msgr2=0x7f5b6c073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 --2- 172.21.15.59:0/4066406921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c073f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009860 tx=0x7f5b5c02bc00 comp rx=0 tx=0).stop 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 -- 172.21.15.59:0/4066406921 shutdown_connections 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 --2- 172.21.15.59:0/4066406921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 --2- 172.21.15.59:0/4066406921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 --2- 172.21.15.59:0/4066406921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c073f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 -- 172.21.15.59:0/4066406921 >> 172.21.15.59:0/4066406921 conn(0x7f5b6c0fcf70 msgr2=0x7f5b6c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 -- 172.21.15.59:0/4066406921 shutdown_connections 2024-04-02T17:10:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.335+0000 7f5b71a00640 1 -- 172.21.15.59:0/4066406921 wait complete. 2024-04-02T17:10:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.336+0000 7f5b71a00640 1 Processor -- start 2024-04-02T17:10:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.336+0000 7f5b71a00640 1 -- start start 2024-04-02T17:10:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10f880 con 0x7f5b6c0744e0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10f700 con 0x7f5b6c1098b0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fa00 con 0x7f5b6c075ac0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40096/0 (socket says 172.21.15.59:40096) 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6be00640 1 -- 172.21.15.59:0/3117076941 learned_addr learned my addr 172.21.15.59:0/3117076941 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71000640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 -- 172.21.15.59:0/3117076941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 msgr2=0x7f5b6c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 -- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 msgr2=0x7f5b6c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6b400640 1 -- 172.21.15.59:0/3117076941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1a1b70 con 0x7f5b6c075ac0 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b6be00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.337+0000 7f5b71000640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b6b400640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c19a490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9f0 tx=0x7f5b5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800ccc0 con 0x7f5b6c075ac0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b58005020 con 0x7f5b6c075ac0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580049e0 con 0x7f5b6c075ac0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a1e60 con 0x7f5b6c075ac0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.338+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a22d0 con 0x7f5b6c075ac0 2024-04-02T17:10:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.339+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c108670 con 0x7f5b6c075ac0 2024-04-02T17:10:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.343+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b58005560 con 0x7f5b6c075ac0 2024-04-02T17:10:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.344+0000 7f5b68c00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 0x7f5b500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.345+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f5b580cff80 con 0x7f5b6c075ac0 2024-04-02T17:10:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.345+0000 7f5b6be00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 0x7f5b500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.345+0000 7f5b6be00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 0x7f5b500a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0095f0 tx=0x7f5b5c034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:32.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.349+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5b580051d0 con 0x7f5b6c075ac0 2024-04-02T17:10:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:31 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/602433876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:31 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1069719628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:10:32.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.623+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5b6c062700 con 0x7f5b6c075ac0 2024-04-02T17:10:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:32.994+0000 7f5b68c00640 1 -- 172.21.15.59:0/3117076941 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f5b5809cd60 con 0x7f5b6c075ac0 2024-04-02T17:10:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-02T17:10:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.002+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 msgr2=0x7f5b500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.002+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 0x7f5b500a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0095f0 tx=0x7f5b5c034b50 comp rx=0 tx=0).stop 2024-04-02T17:10:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.002+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 msgr2=0x7f5b6c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.002+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c19a490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9f0 tx=0x7f5b5800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 shutdown_connections 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c1098b0 0x7f5b6c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c075ac0 0x7f5b6c19a490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b500a3c30 0x7f5b500a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 --2- 172.21.15.59:0/3117076941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b6c0744e0 0x7f5b6c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 >> 172.21.15.59:0/3117076941 conn(0x7f5b6c0fcf70 msgr2=0x7f5b6c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 shutdown_connections 2024-04-02T17:10:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.003+0000 7f5b71a00640 1 -- 172.21.15.59:0/3117076941 wait complete. 2024-04-02T17:10:33.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T17:10:33.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 -- 172.21.15.59:0/139784992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 msgr2=0x7f5d1c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 --2- 172.21.15.59:0/139784992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c101e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d1002bc40 comp rx=0 tx=0).stop 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 -- 172.21.15.59:0/139784992 shutdown_connections 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 --2- 172.21.15.59:0/139784992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 --2- 172.21.15.59:0/139784992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c102bf0 0x7f5d1c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 --2- 172.21.15.59:0/139784992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c101e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 -- 172.21.15.59:0/139784992 >> 172.21.15.59:0/139784992 conn(0x7f5d1c0fcf70 msgr2=0x7f5d1c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 -- 172.21.15.59:0/139784992 shutdown_connections 2024-04-02T17:10:33.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.232+0000 7f5d22a00640 1 -- 172.21.15.59:0/139784992 wait complete. 2024-04-02T17:10:33.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 Processor -- start 2024-04-02T17:10:33.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 -- start start 2024-04-02T17:10:33.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c102bf0 0x7f5d1c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c10d6a0 con 0x7f5d1c102bf0 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c10d520 con 0x7f5d1c1035b0 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.233+0000 7f5d22a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c10d820 con 0x7f5d1c101a00 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d20c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c102bf0 0x7f5d1c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54040/0 (socket says 172.21.15.59:54040) 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 -- 172.21.15.59:0/825832468 learned_addr learned my addr 172.21.15.59:0/825832468 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d21600640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 -- 172.21.15.59:0/825832468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 msgr2=0x7f5d1c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 -- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c102bf0 msgr2=0x7f5d1c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c102bf0 0x7f5d1c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 -- 172.21.15.59:0/825832468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1c1a1b20 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.234+0000 7f5d22000640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400eb10 tx=0x7f5d1400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.235+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004b20 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.235+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d14004cc0 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.235+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14010760 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.235+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1c1a1e10 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.235+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1c1a2280 con 0x7f5d1c1035b0 2024-04-02T17:10:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.237+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1c1a2540 con 0x7f5d1c1035b0 2024-04-02T17:10:33.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.238+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d14005020 con 0x7f5d1c1035b0 2024-04-02T17:10:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.239+0000 7f5d1a000640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 0x7f5d040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.240+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f5d140d0660 con 0x7f5d1c1035b0 2024-04-02T17:10:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.240+0000 7f5d21600640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 0x7f5d040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.241+0000 7f5d21600640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 0x7f5d040a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d1002c180 tx=0x7f5d1002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.243+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5d1409d2d0 con 0x7f5d1c1035b0 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1885732771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T17:10:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3117076941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:32 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1885732771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T17:10:33.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3117076941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:32 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1885732771' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T17:10:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3117076941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:32 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T17:10:33.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:33.453+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5d1c1a2710 con 0x7f5d1c1035b0 2024-04-02T17:10:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.011+0000 7f5d1a000640 1 -- 172.21.15.59:0/825832468 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f5d1409cbd0 con 0x7f5d1c1035b0 2024-04-02T17:10:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-02T17:10:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 msgr2=0x7f5d040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 0x7f5d040a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d1002c180 tx=0x7f5d1002cf90 comp rx=0 tx=0).stop 2024-04-02T17:10:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 msgr2=0x7f5d1c1a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400eb10 tx=0x7f5d1400c2d0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 shutdown_connections 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d040a3c30 0x7f5d040a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d1c102bf0 0x7f5d1c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c1035b0 0x7f5d1c1a14b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 --2- 172.21.15.59:0/825832468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d1c101a00 0x7f5d1c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.019+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 >> 172.21.15.59:0/825832468 conn(0x7f5d1c0fcf70 msgr2=0x7f5d1c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.020+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 shutdown_connections 2024-04-02T17:10:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.020+0000 7f5d22a00640 1 -- 172.21.15.59:0/825832468 wait complete. 2024-04-02T17:10:34.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T17:10:34.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T17:10:34.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T17:10:34.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/825832468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:33 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 -- 172.21.15.59:0/2368766994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf8104f10 msgr2=0x7f6cf8105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2368766994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf8104f10 0x7f6cf8105390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009900 tx=0x7f6ce402bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 -- 172.21.15.59:0/2368766994 shutdown_connections 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2368766994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf81058d0 0x7f6cf810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2368766994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf8104f10 0x7f6cf8105390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2368766994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 0x7f6cf8104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 -- 172.21.15.59:0/2368766994 >> 172.21.15.59:0/2368766994 conn(0x7f6cf80ff2b0 msgr2=0x7f6cf81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 -- 172.21.15.59:0/2368766994 shutdown_connections 2024-04-02T17:10:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.247+0000 7f6cfd600640 1 -- 172.21.15.59:0/2368766994 wait complete. 2024-04-02T17:10:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.249+0000 7f6cfd600640 1 Processor -- start 2024-04-02T17:10:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.249+0000 7f6cfd600640 1 -- start start 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 0x7f6cf8199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf8104f10 0x7f6cf819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810f9f0 con 0x7f6cf8103d20 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810f870 con 0x7f6cf8104f10 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cfd600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810fb70 con 0x7f6cf81058d0 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 0x7f6cf8199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48216/0 (socket says 172.21.15.59:48216) 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 -- 172.21.15.59:0/2867834440 learned_addr learned my addr 172.21.15.59:0/2867834440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf6000640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf8104f10 0x7f6cf819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 -- 172.21.15.59:0/2867834440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf8104f10 msgr2=0x7f6cf819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf8104f10 0x7f6cf819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 -- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 msgr2=0x7f6cf8199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 0x7f6cf8199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.250+0000 7f6cf7400640 1 -- 172.21.15.59:0/2867834440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf81a1c00 con 0x7f6cf81058d0 2024-04-02T17:10:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cf7400640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000eb10 tx=0x7f6cf000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004b20 con 0x7f6cf81058d0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0004cc0 con 0x7f6cf81058d0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0010760 con 0x7f6cf81058d0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf81a1ef0 con 0x7f6cf81058d0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.251+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf81a2300 con 0x7f6cf81058d0 2024-04-02T17:10:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.252+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf810afe0 con 0x7f6cf81058d0 2024-04-02T17:10:34.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.254+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6cf0005020 con 0x7f6cf81058d0 2024-04-02T17:10:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.255+0000 7f6cef400640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 0x7f6cdc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.256+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f6cf00d0910 con 0x7f6cf81058d0 2024-04-02T17:10:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.256+0000 7f6cf6a00640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 0x7f6cdc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.257+0000 7f6cf6a00640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 0x7f6cdc0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8104d70 tx=0x7f6ce8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:34.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.259+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6cf009d400 con 0x7f6cf81058d0 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/825832468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:33 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/825832468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T17:10:34.316 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:33 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:10:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:34.527+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6cf8062810 con 0x7f6cf81058d0 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: pgmap v306: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2867834440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:35 smithi059 ceph-mon[31664]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T17:10:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: pgmap v306: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T17:10:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T17:10:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T17:10:35.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:35.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2867834440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:35 smithi164 ceph-mon[36499]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T17:10:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: pgmap v306: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2867834440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T17:10:35.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:35 smithi154 ceph-mon[36479]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T17:10:35.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.483+0000 7f6cef400640 1 -- 172.21.15.59:0/2867834440 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6cf009cd00 con 0x7f6cf81058d0 2024-04-02T17:10:35.483 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T17:10:35.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.486+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 msgr2=0x7f6cdc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.486+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 0x7f6cdc0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8104d70 tx=0x7f6ce8005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:35.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.486+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 msgr2=0x7f6cf81a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.486+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000eb10 tx=0x7f6cf000c2d0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 shutdown_connections 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cf81058d0 0x7f6cf81a1590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cf8104f10 0x7f6cf819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cdc0a3c30 0x7f6cdc0a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 --2- 172.21.15.59:0/2867834440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf8103d20 0x7f6cf8199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 >> 172.21.15.59:0/2867834440 conn(0x7f6cf80ff2b0 msgr2=0x7f6cf81081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.487+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 shutdown_connections 2024-04-02T17:10:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.488+0000 7f6cfd600640 1 -- 172.21.15.59:0/2867834440 wait complete. 2024-04-02T17:10:35.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3763568083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc4107630 msgr2=0x7f3fc4109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3763568083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc4107630 0x7f3fc4109a80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009be0 tx=0x7f3fbc02bef0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3763568083 shutdown_connections 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3763568083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc41070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3763568083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc4107630 0x7f3fc4109a80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3763568083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc41023d0 0x7f3fc41047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3763568083 >> 172.21.15.59:0/3763568083 conn(0x7f3fc40fbde0 msgr2=0x7f3fc40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3763568083 shutdown_connections 2024-04-02T17:10:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.704+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3763568083 wait complete. 2024-04-02T17:10:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.705+0000 7f3fcaa00640 1 Processor -- start 2024-04-02T17:10:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 -- start start 2024-04-02T17:10:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc41023d0 0x7f3fc4199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc4107630 0x7f3fc41a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410d640 con 0x7f3fc4104d00 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410d4c0 con 0x7f3fc41023d0 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fcaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc410d7c0 con 0x7f3fc4107630 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fc8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fc8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40154/0 (socket says 172.21.15.59:40154) 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc4107630 0x7f3fc41a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fc8c00640 1 -- 172.21.15.59:0/3010998288 learned_addr learned my addr 172.21.15.59:0/3010998288 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.706+0000 7f3fc9600640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc41023d0 0x7f3fc4199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 -- 172.21.15.59:0/3010998288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc4107630 msgr2=0x7f3fc41a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc4107630 0x7f3fc41a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 -- 172.21.15.59:0/3010998288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc41023d0 msgr2=0x7f3fc4199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc41023d0 0x7f3fc4199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 -- 172.21.15.59:0/3010998288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a1a10 con 0x7f3fc4104d00 2024-04-02T17:10:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc8c00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000d770 tx=0x7f3fb000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.707+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0017070 con 0x7f3fc4104d00 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.708+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a1d00 con 0x7f3fc4104d00 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.708+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb0010460 con 0x7f3fc4104d00 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.708+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000f620 con 0x7f3fc4104d00 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.708+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a2240 con 0x7f3fc4104d00 2024-04-02T17:10:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.709+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc4109120 con 0x7f3fc4104d00 2024-04-02T17:10:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.711+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3fb001a070 con 0x7f3fc4104d00 2024-04-02T17:10:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.713+0000 7f3fc2000640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 0x7f3fac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.713+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3fb00cfd00 con 0x7f3fc4104d00 2024-04-02T17:10:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.713+0000 7f3fc9600640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 0x7f3fac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.714+0000 7f3fc9600640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 0x7f3fac0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005fd0 tx=0x7f3fb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.716+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3fb009c790 con 0x7f3fc4104d00 2024-04-02T17:10:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.927+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fc402d4f0 con 0x7f3fc4104d00 2024-04-02T17:10:35.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.931+0000 7f3fc2000640 1 -- 172.21.15.59:0/3010998288 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22234 (secure 0 0 0) 0x7f3fb009c090 con 0x7f3fc4104d00 2024-04-02T17:10:35.931 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:35.931 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:10:34.529467+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:10:35.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 msgr2=0x7f3fac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 0x7f3fac0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005fd0 tx=0x7f3fb8009260 comp rx=0 tx=0).stop 2024-04-02T17:10:35.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 msgr2=0x7f3fc419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:35.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000d770 tx=0x7f3fb000db70 comp rx=0 tx=0).stop 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 shutdown_connections 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fac0a3bb0 0x7f3fac0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc4104d00 0x7f3fc419a330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fc4107630 0x7f3fc41a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 --2- 172.21.15.59:0/3010998288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fc41023d0 0x7f3fc4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 >> 172.21.15.59:0/3010998288 conn(0x7f3fc40fbde0 msgr2=0x7f3fc4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 shutdown_connections 2024-04-02T17:10:35.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:35.933+0000 7f3fcaa00640 1 -- 172.21.15.59:0/3010998288 wait complete. 2024-04-02T17:10:35.950 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:10:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T17:10:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T17:10:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T17:10:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T17:10:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:36 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3010998288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:36.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 -- 172.21.15.59:0/712325904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794108140 msgr2=0x7f6794108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 --2- 172.21.15.59:0/712325904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794108140 0x7f6794108560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6788009860 tx=0x7f678802bb70 comp rx=0 tx=0).stop 2024-04-02T17:10:36.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 -- 172.21.15.59:0/712325904 shutdown_connections 2024-04-02T17:10:36.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 --2- 172.21.15.59:0/712325904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794109330 0x7f67941097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 --2- 172.21.15.59:0/712325904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 0x7f6794110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 --2- 172.21.15.59:0/712325904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794108140 0x7f6794108560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.143+0000 7f679be00640 1 -- 172.21.15.59:0/712325904 >> 172.21.15.59:0/712325904 conn(0x7f6794077110 msgr2=0x7f6794079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:36.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.144+0000 7f679be00640 1 -- 172.21.15.59:0/712325904 shutdown_connections 2024-04-02T17:10:36.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.144+0000 7f679be00640 1 -- 172.21.15.59:0/712325904 wait complete. 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 Processor -- start 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 -- start start 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794109330 0x7f679419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 0x7f67941a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6794113df0 con 0x7f6794108140 2024-04-02T17:10:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6794113c70 con 0x7f6794109330 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f679be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6794113f70 con 0x7f6794109cf0 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f6799400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f6799400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40166/0 (socket says 172.21.15.59:40166) 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.145+0000 7f6799400640 1 -- 172.21.15.59:0/953875709 learned_addr learned my addr 172.21.15.59:0/953875709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799e00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 0x7f67941a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6798a00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794109330 0x7f679419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 -- 172.21.15.59:0/953875709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 msgr2=0x7f67941a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 0x7f67941a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 -- 172.21.15.59:0/953875709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794109330 msgr2=0x7f679419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794109330 0x7f679419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 -- 172.21.15.59:0/953875709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941a6040 con 0x7f6794108140 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6799400640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67880349e0 tx=0x7f6788032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.146+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788041020 con 0x7f6794108140 2024-04-02T17:10:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.147+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f678802c9f0 con 0x7f6794108140 2024-04-02T17:10:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.147+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788039040 con 0x7f6794108140 2024-04-02T17:10:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.147+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67941a62d0 con 0x7f6794108140 2024-04-02T17:10:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.147+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67941a6770 con 0x7f6794108140 2024-04-02T17:10:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.148+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679410f460 con 0x7f6794108140 2024-04-02T17:10:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.150+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6788044020 con 0x7f6794108140 2024-04-02T17:10:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.151+0000 7f6792000640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 0x7f677c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.151+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f67880f1f30 con 0x7f6794108140 2024-04-02T17:10:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.151+0000 7f6798a00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 0x7f677c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.152+0000 7f6798a00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 0x7f677c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f679419f930 tx=0x7f6780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.154+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f678804b020 con 0x7f6794108140 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T17:10:36.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T17:10:36.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T17:10:36.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:36 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3010998288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:36.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.361+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f679402d540 con 0x7f6794108140 2024-04-02T17:10:36.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.365+0000 7f6792000640 1 -- 172.21.15.59:0/953875709 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22221 (secure 0 0 0) 0x7f67880be9c0 con 0x7f6794108140 2024-04-02T17:10:36.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:36.365 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:10:36.009161+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:10:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 msgr2=0x7f677c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 0x7f677c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f679419f930 tx=0x7f6780009260 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 msgr2=0x7f679419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67880349e0 tx=0x7f6788032430 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 shutdown_connections 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6794109cf0 0x7f67941a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6794109330 0x7f679419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f677c0a3bb0 0x7f677c0a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 --2- 172.21.15.59:0/953875709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6794108140 0x7f679419e420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.367+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 >> 172.21.15.59:0/953875709 conn(0x7f6794077110 msgr2=0x7f679410c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.368+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 shutdown_connections 2024-04-02T17:10:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.368+0000 7f679be00640 1 -- 172.21.15.59:0/953875709 wait complete. 2024-04-02T17:10:36.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T17:10:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T17:10:36.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3010998288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 -- 172.21.15.59:0/954068188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd810dfd0 msgr2=0x7f4cd81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 --2- 172.21.15.59:0/954068188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81103c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009cd0 tx=0x7f4cd002c0f0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 -- 172.21.15.59:0/954068188 shutdown_connections 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 --2- 172.21.15.59:0/954068188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81103c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 --2- 172.21.15.59:0/954068188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd8074560 0x7f4cd80749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 --2- 172.21.15.59:0/954068188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 -- 172.21.15.59:0/954068188 >> 172.21.15.59:0/954068188 conn(0x7f4cd80fcf70 msgr2=0x7f4cd80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 -- 172.21.15.59:0/954068188 shutdown_connections 2024-04-02T17:10:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.606+0000 7f4cd7200640 1 -- 172.21.15.59:0/954068188 wait complete. 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.607+0000 7f4cd7200640 1 Processor -- start 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 -- start start 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8113c90 con 0x7f4cd8074560 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8113b10 con 0x7f4cd8075b40 2024-04-02T17:10:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8113e10 con 0x7f4cd810dfd0 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40184/0 (socket says 172.21.15.59:40184) 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd5e00640 1 -- 172.21.15.59:0/645392458 learned_addr learned my addr 172.21.15.59:0/645392458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4cd6800640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.608+0000 7f4ccfe00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 -- 172.21.15.59:0/645392458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd810dfd0 msgr2=0x7f4cd81a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 -- 172.21.15.59:0/645392458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 msgr2=0x7f4cd819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 -- 172.21.15.59:0/645392458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd81a5ea0 con 0x7f4cd8074560 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4ccfe00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd5e00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800e9f0 tx=0x7f4cc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc800cd80 con 0x7f4cd8074560 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc8010910 con 0x7f4cd8074560 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8018990 con 0x7f4cd8074560 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81a6190 con 0x7f4cd8074560 2024-04-02T17:10:36.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.609+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a66e0 con 0x7f4cd8074560 2024-04-02T17:10:36.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.611+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd81041a0 con 0x7f4cd8074560 2024-04-02T17:10:36.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.613+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4cc8010ac0 con 0x7f4cd8074560 2024-04-02T17:10:36.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.615+0000 7f4ccea00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 0x7f4cbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:36.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.615+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f4cc80d05d0 con 0x7f4cd8074560 2024-04-02T17:10:36.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.615+0000 7f4ccfe00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 0x7f4cbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:36.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.616+0000 7f4ccfe00640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 0x7f4cbc0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cd819f790 tx=0x7f4cc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:36.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.620+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4cc809cab0 con 0x7f4cd8074560 2024-04-02T17:10:36.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:36.872+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4cd802d370 con 0x7f4cd8074560 2024-04-02T17:10:36.899 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:10:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: 2024-04-02T17:10:36.549+0000 7ff21fdfe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T17:10:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: pgmap v309: 97 pgs: 19 creating+peering, 20 active+clean, 58 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T17:10:37.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/953875709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: Cluster is now healthy 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:active 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T17:10:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: pgmap v309: 97 pgs: 19 creating+peering, 20 active+clean, 58 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T17:10:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/953875709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: Cluster is now healthy 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:active 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:37 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: pgmap v309: 97 pgs: 19 creating+peering, 20 active+clean, 58 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/953875709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: Cluster is now healthy 2024-04-02T17:10:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] up:active 2024-04-02T17:10:37.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:37.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:37 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T17:10:37.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.508+0000 7f4ccea00640 1 -- 172.21.15.59:0/645392458 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4cc80a0db0 con 0x7f4cd8074560 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 msgr2=0x7f4cbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 0x7f4cbc0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cd819f790 tx=0x7f4cc0005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 msgr2=0x7f4cd819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800e9f0 tx=0x7f4cc800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 shutdown_connections 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd810dfd0 0x7f4cd81a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd8075b40 0x7f4cd819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4cbc0a3bb0 0x7f4cbc0a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 --2- 172.21.15.59:0/645392458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd8074560 0x7f4cd819e280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.511+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 >> 172.21.15.59:0/645392458 conn(0x7f4cd80fcf70 msgr2=0x7f4cd8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.512+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 shutdown_connections 2024-04-02T17:10:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.512+0000 7f4cd7200640 1 -- 172.21.15.59:0/645392458 wait complete. 2024-04-02T17:10:37.536 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-02T17:10:37.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-02T17:10:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 -- 172.21.15.59:0/1968662313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d481036c0 msgr2=0x7f6d48109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1968662313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d481036c0 0x7f6d48109f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009860 tx=0x7f6d4002bb70 comp rx=0 tx=0).stop 2024-04-02T17:10:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 -- 172.21.15.59:0/1968662313 shutdown_connections 2024-04-02T17:10:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1968662313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d481036c0 0x7f6d48109f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1968662313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48102d00 0x7f6d48103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1968662313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48101b10 0x7f6d48101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 -- 172.21.15.59:0/1968662313 >> 172.21.15.59:0/1968662313 conn(0x7f6d480fd080 msgr2=0x7f6d480ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.766+0000 7f6d4e400640 1 -- 172.21.15.59:0/1968662313 shutdown_connections 2024-04-02T17:10:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.767+0000 7f6d4e400640 1 -- 172.21.15.59:0/1968662313 wait complete. 2024-04-02T17:10:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.767+0000 7f6d4e400640 1 Processor -- start 2024-04-02T17:10:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 -- start start 2024-04-02T17:10:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48101b10 0x7f6d480758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 0x7f6d48073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4810d790 con 0x7f6d481036c0 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4810d610 con 0x7f6d48101b10 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d4e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4810d910 con 0x7f6d48102d00 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40192/0 (socket says 172.21.15.59:40192) 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d47e00640 1 -- 172.21.15.59:0/1710199521 learned_addr learned my addr 172.21.15.59:0/1710199521 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d3ea00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 0x7f6d48073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.768+0000 7f6d47400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48101b10 0x7f6d480758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 -- 172.21.15.59:0/1710199521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 msgr2=0x7f6d48073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 0x7f6d48073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 -- 172.21.15.59:0/1710199521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48101b10 msgr2=0x7f6d480758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48101b10 0x7f6d480758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 -- 172.21.15.59:0/1710199521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d48074cc0 con 0x7f6d481036c0 2024-04-02T17:10:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d3ea00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 0x7f6d48073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d47e00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002c180 tx=0x7f6d40032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40041020 con 0x7f6d481036c0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.769+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d4002c9f0 con 0x7f6d481036c0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.770+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d481a1ec0 con 0x7f6d481036c0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.770+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40039040 con 0x7f6d481036c0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.770+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a2380 con 0x7f6d481036c0 2024-04-02T17:10:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.771+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d48108e30 con 0x7f6d481036c0 2024-04-02T17:10:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.773+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6d40044020 con 0x7f6d481036c0 2024-04-02T17:10:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.774+0000 7f6d44c00640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 0x7f6d2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.774+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6d400f1bd0 con 0x7f6d481036c0 2024-04-02T17:10:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.774+0000 7f6d47400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 0x7f6d2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.775+0000 7f6d47400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 0x7f6d2c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d48102b60 tx=0x7f6d38010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:37.778+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6d4004b020 con 0x7f6d481036c0 2024-04-02T17:10:38.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:38.029+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6d480627a0 con 0x7f6d481036c0 2024-04-02T17:10:38.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:38 smithi059 ceph-mon[31664]: pgmap v311: 97 pgs: 19 creating+peering, 37 active+clean, 41 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 227 B/s wr, 1 op/s 2024-04-02T17:10:38.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:38 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T17:10:38.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:38 smithi059 ceph-mon[31664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:38.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:38 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T17:10:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:38 smithi164 ceph-mon[36499]: pgmap v311: 97 pgs: 19 creating+peering, 37 active+clean, 41 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 227 B/s wr, 1 op/s 2024-04-02T17:10:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:38 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T17:10:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:38 smithi164 ceph-mon[36499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:38.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:38 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T17:10:38.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:38 smithi154 ceph-mon[36479]: pgmap v311: 97 pgs: 19 creating+peering, 37 active+clean, 41 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 227 B/s wr, 1 op/s 2024-04-02T17:10:38.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:38 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/645392458' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T17:10:38.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:38 smithi154 ceph-mon[36479]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:38.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:38 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T17:10:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.023+0000 7f6d44c00640 1 -- 172.21.15.59:0/1710199521 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6d400be660 con 0x7f6d481036c0 2024-04-02T17:10:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 msgr2=0x7f6d2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 0x7f6d2c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d48102b60 tx=0x7f6d38010040 comp rx=0 tx=0).stop 2024-04-02T17:10:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 msgr2=0x7f6d48074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002c180 tx=0x7f6d40032460 comp rx=0 tx=0).stop 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 shutdown_connections 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3c30 0x7f6d2c0a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d481036c0 0x7f6d48074430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48102d00 0x7f6d48073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 --2- 172.21.15.59:0/1710199521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48101b10 0x7f6d480758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.027+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 >> 172.21.15.59:0/1710199521 conn(0x7f6d480fd080 msgr2=0x7f6d48106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.028+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 shutdown_connections 2024-04-02T17:10:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.028+0000 7f6d4e400640 1 -- 172.21.15.59:0/1710199521 wait complete. 2024-04-02T17:10:39.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-02T17:10:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 -- 172.21.15.59:0/3388547025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103d40 msgr2=0x7f6a14104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 --2- 172.21.15.59:0/3388547025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103d40 0x7f6a14104160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009900 tx=0x7f6a0402bc50 comp rx=0 tx=0).stop 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 -- 172.21.15.59:0/3388547025 shutdown_connections 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 --2- 172.21.15.59:0/3388547025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 0x7f6a1410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 --2- 172.21.15.59:0/3388547025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14104f30 0x7f6a141053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 --2- 172.21.15.59:0/3388547025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103d40 0x7f6a14104160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 -- 172.21.15.59:0/3388547025 >> 172.21.15.59:0/3388547025 conn(0x7f6a140ff2b0 msgr2=0x7f6a141016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 -- 172.21.15.59:0/3388547025 shutdown_connections 2024-04-02T17:10:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.261+0000 7f6a19600640 1 -- 172.21.15.59:0/3388547025 wait complete. 2024-04-02T17:10:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 Processor -- start 2024-04-02T17:10:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 -- start start 2024-04-02T17:10:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14103d40 0x7f6a1419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 0x7f6a141a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410f9c0 con 0x7f6a14104f30 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410f840 con 0x7f6a14103d40 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.262+0000 7f6a19600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410fb40 con 0x7f6a141058f0 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40216/0 (socket says 172.21.15.59:40216) 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 -- 172.21.15.59:0/2859010160 learned_addr learned my addr 172.21.15.59:0/2859010160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12a00640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14103d40 0x7f6a1419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a13400640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 0x7f6a141a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 -- 172.21.15.59:0/2859010160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 msgr2=0x7f6a141a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 0x7f6a141a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 -- 172.21.15.59:0/2859010160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14103d40 msgr2=0x7f6a1419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14103d40 0x7f6a1419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 -- 172.21.15.59:0/2859010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a141a1d20 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.263+0000 7f6a12000640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000ea60 tx=0x7f6a0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.264+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000cde0 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.264+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a000035f0 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.264+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00010670 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.264+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a141a1fb0 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.264+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141a2520 con 0x7f6a14104f30 2024-04-02T17:10:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.265+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1410b060 con 0x7f6a14104f30 2024-04-02T17:10:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.268+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a000037a0 con 0x7f6a14104f30 2024-04-02T17:10:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.269+0000 7f6a0b400640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 0x7f69f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.269+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6a000d04f0 con 0x7f6a14104f30 2024-04-02T17:10:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.269+0000 7f6a12a00640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 0x7f69f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.270+0000 7f6a12a00640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 0x7f69f80a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a04000c00 tx=0x7f6a04005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.273+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6a0009d020 con 0x7f6a14104f30 2024-04-02T17:10:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:39.528+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6a1402d5c0 con 0x7f6a14104f30 2024-04-02T17:10:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T17:10:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T17:10:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T17:10:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-02T17:10:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:39 smithi059 ceph-mon[31664]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-02T17:10:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T17:10:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T17:10:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T17:10:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-02T17:10:39.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:39 smithi164 ceph-mon[36499]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-02T17:10:39.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T17:10:39.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1710199521' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T17:10:39.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T17:10:39.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T17:10:39.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-02T17:10:39.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:39 smithi154 ceph-mon[36479]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-02T17:10:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.035+0000 7f6a0b400640 1 -- 172.21.15.59:0/2859010160 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f6a0009c920 con 0x7f6a14104f30 2024-04-02T17:10:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 msgr2=0x7f69f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 0x7f69f80a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a04000c00 tx=0x7f6a04005fd0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 msgr2=0x7f6a1419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000ea60 tx=0x7f6a0000ef20 comp rx=0 tx=0).stop 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 shutdown_connections 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69f80a3c30 0x7f69f80a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14104f30 0x7f6a1419a640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a141058f0 0x7f6a141a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 --2- 172.21.15.59:0/2859010160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a14103d40 0x7f6a1419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 >> 172.21.15.59:0/2859010160 conn(0x7f6a140ff2b0 msgr2=0x7f6a14108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 shutdown_connections 2024-04-02T17:10:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.040+0000 7f6a19600640 1 -- 172.21.15.59:0/2859010160 wait complete. 2024-04-02T17:10:40.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-02T17:10:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 -- 172.21.15.59:0/3534014827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 msgr2=0x7f8974109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 --2- 172.21.15.59:0/3534014827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f8974109790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8960009900 tx=0x7f896002bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 -- 172.21.15.59:0/3534014827 shutdown_connections 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 --2- 172.21.15.59:0/3534014827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974109cd0 0x7f8974110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 --2- 172.21.15.59:0/3534014827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f8974109790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 --2- 172.21.15.59:0/3534014827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974108120 0x7f8974108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 -- 172.21.15.59:0/3534014827 >> 172.21.15.59:0/3534014827 conn(0x7f89740770c0 msgr2=0x7f8974079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 -- 172.21.15.59:0/3534014827 shutdown_connections 2024-04-02T17:10:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.272+0000 7f897b200640 1 -- 172.21.15.59:0/3534014827 wait complete. 2024-04-02T17:10:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.273+0000 7f897b200640 1 Processor -- start 2024-04-02T17:10:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.273+0000 7f897b200640 1 -- start start 2024-04-02T17:10:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.273+0000 7f897b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974108120 0x7f897419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f897b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f897b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974109cd0 0x7f89741a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f897b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8974113e00 con 0x7f8974109cd0 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f897b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8974113c80 con 0x7f8974109310 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f897b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8974113f80 con 0x7f8974108120 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54170/0 (socket says 172.21.15.59:54170) 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 -- 172.21.15.59:0/3489799399 learned_addr learned my addr 172.21.15.59:0/3489799399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8979200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974109cd0 0x7f89741a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973e00640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974108120 0x7f897419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 -- 172.21.15.59:0/3489799399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974108120 msgr2=0x7f897419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974108120 0x7f897419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 -- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974109cd0 msgr2=0x7f89741a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974109cd0 0x7f89741a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.274+0000 7f8973400640 1 -- 172.21.15.59:0/3489799399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89741a6020 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f8973400640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8960000c00 tx=0x7f896002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960043020 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89600054e0 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960003c40 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89741a62b0 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.275+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89741a67b0 con 0x7f8974109310 2024-04-02T17:10:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.276+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897410f440 con 0x7f8974109310 2024-04-02T17:10:40.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.279+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8960046020 con 0x7f8974109310 2024-04-02T17:10:40.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.280+0000 7f8970c00640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 0x7f89580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:40.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.280+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f89600f2300 con 0x7f8974109310 2024-04-02T17:10:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.281+0000 7f8973e00640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 0x7f89580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.281+0000 7f8973e00640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 0x7f89580a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8974109170 tx=0x7f8964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.284+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f896004d020 con 0x7f8974109310 2024-04-02T17:10:40.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:40.536+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f897402d370 con 0x7f8974109310 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: pgmap v312: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 6 op/s 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-02T17:10:40.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:40 smithi059 ceph-mon[31664]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: pgmap v312: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 6 op/s 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-02T17:10:40.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-02T17:10:40.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:40 smithi164 ceph-mon[36499]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-02T17:10:40.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: pgmap v312: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 6 op/s 2024-04-02T17:10:40.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T17:10:40.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2859010160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T17:10:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:10:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T17:10:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-02T17:10:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-04-02T17:10:40.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:40 smithi154 ceph-mon[36479]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-02T17:10:41.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.051+0000 7f8970c00640 1 -- 172.21.15.59:0/3489799399 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f89600bed90 con 0x7f8974109310 2024-04-02T17:10:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.056+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 msgr2=0x7f89580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.056+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 0x7f89580a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8974109170 tx=0x7f8964005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.056+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 msgr2=0x7f897419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8960000c00 tx=0x7f896002c650 comp rx=0 tx=0).stop 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 shutdown_connections 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89580a3c30 0x7f89580a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8974109cd0 0x7f89741a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8974109310 0x7f897419e940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 --2- 172.21.15.59:0/3489799399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8974108120 0x7f897419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 >> 172.21.15.59:0/3489799399 conn(0x7f89740770c0 msgr2=0x7f897410c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 shutdown_connections 2024-04-02T17:10:41.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.057+0000 7f897b200640 1 -- 172.21.15.59:0/3489799399 wait complete. 2024-04-02T17:10:41.085 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-02T17:10:41.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-02T17:10:41.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.292+0000 7facc7200640 1 -- 172.21.15.59:0/1568820455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0104f30 msgr2=0x7facc01053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:41.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.292+0000 7facc7200640 1 --2- 172.21.15.59:0/1568820455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0104f30 0x7facc01053b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faca8009900 tx=0x7faca802bc40 comp rx=0 tx=0).stop 2024-04-02T17:10:41.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 -- 172.21.15.59:0/1568820455 shutdown_connections 2024-04-02T17:10:41.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 --2- 172.21.15.59:0/1568820455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 --2- 172.21.15.59:0/1568820455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0104f30 0x7facc01053b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 --2- 172.21.15.59:0/1568820455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0103d40 0x7facc0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 -- 172.21.15.59:0/1568820455 >> 172.21.15.59:0/1568820455 conn(0x7facc00ff2b0 msgr2=0x7facc01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:41.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 -- 172.21.15.59:0/1568820455 shutdown_connections 2024-04-02T17:10:41.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.293+0000 7facc7200640 1 -- 172.21.15.59:0/1568820455 wait complete. 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.294+0000 7facc7200640 1 Processor -- start 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.294+0000 7facc7200640 1 -- start start 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.294+0000 7facc7200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0104f30 0x7facc019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc01a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:41.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7facc010f9a0 con 0x7facc0104f30 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc7200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7facc010f820 con 0x7facc0103d40 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7facc010fb20 con 0x7facc01058f0 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56436/0 (socket says 172.21.15.59:56436) 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 -- 172.21.15.59:0/3887762324 learned_addr learned my addr 172.21.15.59:0/3887762324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facb7400640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0104f30 0x7facc019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc5200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc01a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 -- 172.21.15.59:0/3887762324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 msgr2=0x7facc01a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc01a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 -- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0104f30 msgr2=0x7facc019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0104f30 0x7facc019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facbfe00640 1 -- 172.21.15.59:0/3887762324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01a1c30 con 0x7facc0103d40 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.295+0000 7facc5200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facbfe00640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facb000ea80 tx=0x7facb000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:41.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb000ce40 con 0x7facc0103d40 2024-04-02T17:10:41.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb0011950 con 0x7facc0103d40 2024-04-02T17:10:41.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb001ab80 con 0x7facc0103d40 2024-04-02T17:10:41.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01a1f20 con 0x7facc0103d40 2024-04-02T17:10:41.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.296+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a2390 con 0x7facc0103d40 2024-04-02T17:10:41.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.297+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc010b060 con 0x7facc0103d40 2024-04-02T17:10:41.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.299+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7facb0011b00 con 0x7facc0103d40 2024-04-02T17:10:41.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.300+0000 7facbd600640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 0x7faca40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:41.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.301+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7facb00d08f0 con 0x7facc0103d40 2024-04-02T17:10:41.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.301+0000 7facb7400640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 0x7faca40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:41.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.301+0000 7facb7400640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 0x7faca40a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faca802c7c0 tx=0x7faca8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:41.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.305+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7facb0016030 con 0x7facc0103d40 2024-04-02T17:10:41.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:41.544+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7facc0061f40 con 0x7faca40a3c30 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3489799399' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:active 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-04-02T17:10:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-mon[31664]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T17:10:41.800 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:10:41.555+0000 7f4dc5a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T17:10:41.800 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:10:41.555+0000 7f4dc5a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T17:10:41.800 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:10:41.555+0000 7f4dc5a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T17:10:41.800 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:10:41.555+0000 7f4dc5a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T17:10:41.800 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:10:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:10:41.555+0000 7f4dc5a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3489799399' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:active 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-04-02T17:10:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:41 smithi164 ceph-mon[36499]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T17:10:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3489799399' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T17:10:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T17:10:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] up:active 2024-04-02T17:10:41.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T17:10:41.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-04-02T17:10:41.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-04-02T17:10:41.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:41 smithi154 ceph-mon[36479]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T17:10:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.111+0000 7facbd600640 1 -- 172.21.15.59:0/3887762324 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facc0061f40 con 0x7faca40a3c30 2024-04-02T17:10:42.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 msgr2=0x7faca40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 0x7faca40a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faca802c7c0 tx=0x7faca8005fd0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 msgr2=0x7facc019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facb000ea80 tx=0x7facb000ef40 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 shutdown_connections 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faca40a3c30 0x7faca40a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facc0104f30 0x7facc019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facc01058f0 0x7facc01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 --2- 172.21.15.59:0/3887762324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facc0103d40 0x7facc019a010 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 >> 172.21.15.59:0/3887762324 conn(0x7facc00ff2b0 msgr2=0x7facc0108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 shutdown_connections 2024-04-02T17:10:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.114+0000 7facc7200640 1 -- 172.21.15.59:0/3887762324 wait complete. 2024-04-02T17:10:42.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 -- 172.21.15.59:0/3430042216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 msgr2=0x7fef4c109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 --2- 172.21.15.59:0/3430042216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c109790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef34009900 tx=0x7fef3402bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 -- 172.21.15.59:0/3430042216 shutdown_connections 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 --2- 172.21.15.59:0/3430042216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c109cd0 0x7fef4c110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 --2- 172.21.15.59:0/3430042216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c109790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 --2- 172.21.15.59:0/3430042216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c108120 0x7fef4c108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 -- 172.21.15.59:0/3430042216 >> 172.21.15.59:0/3430042216 conn(0x7fef4c0770c0 msgr2=0x7fef4c079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 -- 172.21.15.59:0/3430042216 shutdown_connections 2024-04-02T17:10:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.350+0000 7fef50e00640 1 -- 172.21.15.59:0/3430042216 wait complete. 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.351+0000 7fef50e00640 1 Processor -- start 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.351+0000 7fef50e00640 1 -- start start 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c108120 0x7fef4c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c109cd0 0x7fef4c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c113db0 con 0x7fef4c109cd0 2024-04-02T17:10:42.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c113c30 con 0x7fef4c109310 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef50e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c113f30 con 0x7fef4c108120 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef49e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c108120 0x7fef4c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c109cd0 0x7fef4c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56456/0 (socket says 172.21.15.59:56456) 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 -- 172.21.15.59:0/3642093378 learned_addr learned my addr 172.21.15.59:0/3642093378 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 -- 172.21.15.59:0/3642093378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c108120 msgr2=0x7fef4c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c108120 0x7fef4c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 -- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c109cd0 msgr2=0x7fef4c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c109cd0 0x7fef4c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.352+0000 7fef41400640 1 -- 172.21.15.59:0/3642093378 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1a6050 con 0x7fef4c109310 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef41400640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef34009230 tx=0x7fef3402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34041070 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef34004450 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34003e20 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1a62e0 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.353+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1a6780 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.354+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c10f440 con 0x7fef4c109310 2024-04-02T17:10:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.356+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef34044070 con 0x7fef4c109310 2024-04-02T17:10:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.358+0000 7fef43400640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.358+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fef340f1e20 con 0x7fef4c109310 2024-04-02T17:10:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.358+0000 7fef49e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.359+0000 7fef49e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 0x7fef300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef4c109170 tx=0x7fef3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.362+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fef340be260 con 0x7fef4c109310 2024-04-02T17:10:42.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.605+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fef4c061fd0 con 0x7fef300a3c30 2024-04-02T17:10:42.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.648+0000 7fef43400640 1 -- 172.21.15.59:0/3642093378 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef4c061fd0 con 0x7fef300a3c30 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 msgr2=0x7fef300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 0x7fef300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef4c109170 tx=0x7fef3c005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 msgr2=0x7fef4c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef34009230 tx=0x7fef3402c650 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 shutdown_connections 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fef300a3c30 0x7fef300a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4c109cd0 0x7fef4c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c109310 0x7fef4c19e970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 --2- 172.21.15.59:0/3642093378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c108120 0x7fef4c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 >> 172.21.15.59:0/3642093378 conn(0x7fef4c0770c0 msgr2=0x7fef4c10c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 shutdown_connections 2024-04-02T17:10:42.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.651+0000 7fef50e00640 1 -- 172.21.15.59:0/3642093378 wait complete. 2024-04-02T17:10:42.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-02T17:10:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 -- 172.21.15.59:0/414005228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 msgr2=0x7f7d28105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/414005228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d28105150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009900 tx=0x7f7d1002bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 -- 172.21.15.59:0/414005228 shutdown_connections 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/414005228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 0x7f7d28105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/414005228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d28105150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/414005228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d281027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 -- 172.21.15.59:0/414005228 >> 172.21.15.59:0/414005228 conn(0x7f7d280fbf90 msgr2=0x7f7d280fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 -- 172.21.15.59:0/414005228 shutdown_connections 2024-04-02T17:10:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.905+0000 7f7d2cc00640 1 -- 172.21.15.59:0/414005228 wait complete. 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.906+0000 7f7d2cc00640 1 Processor -- start 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.906+0000 7f7d2cc00640 1 -- start start 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 0x7f7d281a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28113d70 con 0x7f7d28105690 2024-04-02T17:10:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28113bf0 con 0x7f7d28100370 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28113ef0 con 0x7f7d28102ce0 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d25e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d25e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56480/0 (socket says 172.21.15.59:56480) 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60560/0 (socket says 172.21.15.59:60560) 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d26800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 0x7f7d281a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d25e00640 1 -- 172.21.15.59:0/857172024 learned_addr learned my addr 172.21.15.59:0/857172024 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.907+0000 7f7d25e00640 1 -- 172.21.15.59:0/857172024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 msgr2=0x7f7d2819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d25e00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d25e00640 1 -- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 msgr2=0x7f7d281a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d25e00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 0x7f7d281a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d25e00640 1 -- 172.21.15.59:0/857172024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d281a6010 con 0x7f7d28100370 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d1fe00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d25e00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e9f0 tx=0x7f7d1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800cd80 con 0x7f7d28100370 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d18010910 con 0x7f7d28100370 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18018990 con 0x7f7d28100370 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d281a6300 con 0x7f7d28100370 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.908+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d281a67d0 con 0x7f7d28100370 2024-04-02T17:10:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.910+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d280fda80 con 0x7f7d28100370 2024-04-02T17:10:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.911+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7d18010ac0 con 0x7f7d28100370 2024-04-02T17:10:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.913+0000 7f7d1ea00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 0x7f7d0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.913+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7d180d05c0 con 0x7f7d28100370 2024-04-02T17:10:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.913+0000 7f7d1fe00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 0x7f7d0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.914+0000 7f7d1fe00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 0x7f7d0c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002c7b0 tx=0x7f7d10005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:42.917+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d1809ca00 con 0x7f7d28100370 2024-04-02T17:10:43.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.168+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7d28061fd0 con 0x7f7d0c0a3c30 2024-04-02T17:10:43.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.186+0000 7f7d1ea00640 1 -- 172.21.15.59:0/857172024 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7d28061fd0 con 0x7f7d0c0a3c30 2024-04-02T17:10:43.185 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_0/3ef474e3-627c-47dd-9089-3f22eb54e744 2024-04-02T17:10:43.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.188+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 msgr2=0x7f7d0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.188+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 0x7f7d0c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002c7b0 tx=0x7f7d10005fd0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.188+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 msgr2=0x7f7d2819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.188+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e9f0 tx=0x7f7d1800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 shutdown_connections 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d0c0a3c30 0x7f7d0c0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d28105690 0x7f7d281a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d28102ce0 0x7f7d2819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 --2- 172.21.15.59:0/857172024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d28100370 0x7f7d2819e3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 >> 172.21.15.59:0/857172024 conn(0x7f7d280fbf90 msgr2=0x7f7d28100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:43.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 shutdown_connections 2024-04-02T17:10:43.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.189+0000 7f7d2cc00640 1 -- 172.21.15.59:0/857172024 wait complete. 2024-04-02T17:10:43.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-02T17:10:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: pgmap v313: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-04-02T17:10:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:active 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: Cluster is now healthy 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T17:10:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:43 smithi164 ceph-mon[36499]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-02T17:10:43.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: pgmap v313: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:active 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: Cluster is now healthy 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T17:10:43.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:43 smithi154 ceph-mon[36479]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-02T17:10:43.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 -- 172.21.15.59:0/881187483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c104e10 msgr2=0x7f926c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 --2- 172.21.15.59:0/881187483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c104e10 0x7f926c105290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9258009900 tx=0x7f925802bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 -- 172.21.15.59:0/881187483 shutdown_connections 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 --2- 172.21.15.59:0/881187483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 0x7f926c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 --2- 172.21.15.59:0/881187483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c104e10 0x7f926c105290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 --2- 172.21.15.59:0/881187483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c103c20 0x7f926c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 -- 172.21.15.59:0/881187483 >> 172.21.15.59:0/881187483 conn(0x7f926c0ff1b0 msgr2=0x7f926c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 -- 172.21.15.59:0/881187483 shutdown_connections 2024-04-02T17:10:43.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.437+0000 7f926b200640 1 -- 172.21.15.59:0/881187483 wait complete. 2024-04-02T17:10:43.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.438+0000 7f926b200640 1 Processor -- start 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 -- start start 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 0x7f926c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 0x7f926c1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10f8f0 con 0x7f926c1057d0 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10f770 con 0x7f926c103c20 2024-04-02T17:10:43.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f926c10fa70 con 0x7f926c104e10 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f9269e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f9269e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56496/0 (socket says 172.21.15.59:56496) 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f9269e00640 1 -- 172.21.15.59:0/967209770 learned_addr learned my addr 172.21.15.59:0/967209770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f926a800640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 0x7f926c1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.439+0000 7f9269400640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 0x7f926c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 -- 172.21.15.59:0/967209770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 msgr2=0x7f926c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 0x7f926c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 -- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 msgr2=0x7f926c1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 0x7f926c1a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 -- 172.21.15.59:0/967209770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1a1b20 con 0x7f926c103c20 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269400640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 0x7f926c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:10:43.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9269e00640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9c0 tx=0x7f925c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c00cd80 con 0x7f926c103c20 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c0035f0 con 0x7f926c103c20 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1a1e10 con 0x7f926c103c20 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c010690 con 0x7f926c103c20 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.440+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1a2220 con 0x7f926c103c20 2024-04-02T17:10:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.442+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c10aee0 con 0x7f926c103c20 2024-04-02T17:10:43.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.443+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f925c0037a0 con 0x7f926c103c20 2024-04-02T17:10:43.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.445+0000 7f9262a00640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 0x7f92500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.445+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f925c0d07e0 con 0x7f926c103c20 2024-04-02T17:10:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.445+0000 7f9269400640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 0x7f92500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.446+0000 7f9269400640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 0x7f92500a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f925802c240 tx=0x7f925802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.449+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f925c09d270 con 0x7f926c103c20 2024-04-02T17:10:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: pgmap v313: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 11 op/s 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] up:active 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: Cluster is now healthy 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T17:10:43.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:43 smithi059 ceph-mon[31664]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-02T17:10:43.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.697+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f926c061f40 con 0x7f92500a3bb0 2024-04-02T17:10:43.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.739+0000 7f9262a00640 1 -- 172.21.15.59:0/967209770 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f926c061f40 con 0x7f92500a3bb0 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 msgr2=0x7f92500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 0x7f92500a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f925802c240 tx=0x7f925802df90 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 msgr2=0x7f926c199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9c0 tx=0x7f925c00ee80 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 shutdown_connections 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3bb0 0x7f92500a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c1057d0 0x7f926c1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c104e10 0x7f926c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 --2- 172.21.15.59:0/967209770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c103c20 0x7f926c199f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 >> 172.21.15.59:0/967209770 conn(0x7f926c0ff1b0 msgr2=0x7f926c1080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 shutdown_connections 2024-04-02T17:10:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.742+0000 7f926b200640 1 -- 172.21.15.59:0/967209770 wait complete. 2024-04-02T17:10:43.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 -- 172.21.15.59:0/1020729821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c1058c0 msgr2=0x7f117c10c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 --2- 172.21.15.59:0/1020729821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c1058c0 0x7f117c10c150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1174009be0 tx=0x7f117402bef0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 -- 172.21.15.59:0/1020729821 shutdown_connections 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 --2- 172.21.15.59:0/1020729821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c1058c0 0x7f117c10c150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 --2- 172.21.15.59:0/1020729821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c104f00 0x7f117c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 --2- 172.21.15.59:0/1020729821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c103d10 0x7f117c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 -- 172.21.15.59:0/1020729821 >> 172.21.15.59:0/1020729821 conn(0x7f117c0ff2c0 msgr2=0x7f117c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 -- 172.21.15.59:0/1020729821 shutdown_connections 2024-04-02T17:10:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.998+0000 7f1184000640 1 -- 172.21.15.59:0/1020729821 wait complete. 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 Processor -- start 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 -- start start 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 0x7f117c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c1058c0 0x7f117c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10f9e0 con 0x7f117c104f00 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10f860 con 0x7f117c103d10 2024-04-02T17:10:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:43.999+0000 7f1184000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10fb60 con 0x7f117c1058c0 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39354/0 (socket says 172.21.15.59:39354) 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 -- 172.21.15.59:0/2059705365 learned_addr learned my addr 172.21.15.59:0/2059705365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1182000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c1058c0 0x7f117c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1181600640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 0x7f117c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 -- 172.21.15.59:0/2059705365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c1058c0 msgr2=0x7f117c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c1058c0 0x7f117c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 -- 172.21.15.59:0/2059705365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 msgr2=0x7f117c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 0x7f117c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 -- 172.21.15.59:0/2059705365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c1a1bf0 con 0x7f117c104f00 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1181600640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 0x7f117c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.000+0000 7f1180c00640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9f0 tx=0x7f116800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.001+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800ccc0 con 0x7f117c104f00 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.001+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116800ce60 con 0x7f117c104f00 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.001+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c1a1ee0 con 0x7f117c104f00 2024-04-02T17:10:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.001+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168010690 con 0x7f117c104f00 2024-04-02T17:10:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.001+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1a2350 con 0x7f117c104f00 2024-04-02T17:10:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.002+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c10b030 con 0x7f117c104f00 2024-04-02T17:10:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.004+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f11680046c0 con 0x7f117c104f00 2024-04-02T17:10:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.005+0000 7f117a000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.005+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f11680d0710 con 0x7f117c104f00 2024-04-02T17:10:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.006+0000 7f1181600640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.006+0000 7f1181600640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 0x7f11640a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f117c104d60 tx=0x7f1170009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.010+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f116809d1a0 con 0x7f117c104f00 2024-04-02T17:10:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.123+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1168099110 con 0x7f117c104f00 2024-04-02T17:10:44.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.263+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f117c061f40 con 0x7f11640a3bb0 2024-04-02T17:10:44.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.277+0000 7f117a000640 1 -- 172.21.15.59:0/2059705365 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f117c061f40 con 0x7f11640a3bb0 2024-04-02T17:10:44.276 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 2024-04-02T17:10:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 msgr2=0x7f11640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 0x7f11640a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f117c104d60 tx=0x7f1170009260 comp rx=0 tx=0).stop 2024-04-02T17:10:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 msgr2=0x7f117c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9f0 tx=0x7f116800edf0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 shutdown_connections 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f117c104f00 0x7f117c19a510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f117c1058c0 0x7f117c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 --2- 172.21.15.59:0/2059705365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f117c103d10 0x7f117c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.279+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 >> 172.21.15.59:0/2059705365 conn(0x7f117c0ff2c0 msgr2=0x7f117c108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.280+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 shutdown_connections 2024-04-02T17:10:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.280+0000 7f1184000640 1 -- 172.21.15.59:0/2059705365 wait complete. 2024-04-02T17:10:44.299 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:10:44.384 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:44 smithi059 ceph-mon[31664]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.384 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:44 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:active 2024-04-02T17:10:44.384 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:44 smithi059 ceph-mon[31664]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T17:10:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:44 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:10:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:44 smithi059 ceph-mon[31664]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:44 smithi154 ceph-mon[36479]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:44 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:active 2024-04-02T17:10:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:44 smithi154 ceph-mon[36479]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T17:10:44.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:44 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:10:44.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:44 smithi154 ceph-mon[36479]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 -- 172.21.15.59:0/566111787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46001076f0 msgr2=0x7f4600109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 --2- 172.21.15.59:0/566111787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46001076f0 0x7f4600109ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009ca0 tx=0x7f45f802c100 comp rx=0 tx=0).stop 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 -- 172.21.15.59:0/566111787 shutdown_connections 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 --2- 172.21.15.59:0/566111787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46001076f0 0x7f4600109ae0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 --2- 172.21.15.59:0/566111787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 0x7f46001071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 --2- 172.21.15.59:0/566111787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4600102490 0x7f4600104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 -- 172.21.15.59:0/566111787 >> 172.21.15.59:0/566111787 conn(0x7f46000fbda0 msgr2=0x7f46000fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 -- 172.21.15.59:0/566111787 shutdown_connections 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.526+0000 7f4606600640 1 -- 172.21.15.59:0/566111787 wait complete. 2024-04-02T17:10:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.527+0000 7f4606600640 1 Processor -- start 2024-04-02T17:10:44.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.527+0000 7f4606600640 1 -- start start 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 0x7f460019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46001076f0 0x7f46001a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d5e0 con 0x7f46001076f0 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d460 con 0x7f4600102490 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4606600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d760 con 0x7f4600104dc0 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4605200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4605200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56520/0 (socket says 172.21.15.59:56520) 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4605200640 1 -- 172.21.15.59:0/937407564 learned_addr learned my addr 172.21.15.59:0/937407564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f4605c00640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46001076f0 0x7f46001a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.528+0000 7f45f7e00640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 0x7f460019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 -- 172.21.15.59:0/937407564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 msgr2=0x7f460019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 0x7f460019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 -- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46001076f0 msgr2=0x7f46001a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46001076f0 0x7f46001a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 -- 172.21.15.59:0/937407564 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001a1a30 con 0x7f4600102490 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4605200640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d950 tx=0x7f45f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0017070 con 0x7f4600102490 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0009c30 con 0x7f4600102490 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a1d20 con 0x7f4600102490 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000f6b0 con 0x7f4600102490 2024-04-02T17:10:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.529+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a2200 con 0x7f4600102490 2024-04-02T17:10:44.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.531+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46001091e0 con 0x7f4600102490 2024-04-02T17:10:44.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.532+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f45f00109d0 con 0x7f4600102490 2024-04-02T17:10:44.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.534+0000 7f45fe000640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 0x7f45e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:44.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.534+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f45f00d0070 con 0x7f4600102490 2024-04-02T17:10:44.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.534+0000 7f45f7e00640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 0x7f45e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:44.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.535+0000 7f45f7e00640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 0x7f45e40a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f460019b320 tx=0x7f45e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:44.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.538+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f45f0009de0 con 0x7f4600102490 2024-04-02T17:10:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:44 smithi164 ceph-mon[36499]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:44 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] up:active 2024-04-02T17:10:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:44 smithi164 ceph-mon[36499]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T17:10:44.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:44 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:10:44.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:44 smithi164 ceph-mon[36499]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:44.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.788+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f460002d370 con 0x7f4600102490 2024-04-02T17:10:44.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.789+0000 7f45fe000640 1 -- 172.21.15.59:0/937407564 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f460002d370 con 0x7f4600102490 2024-04-02T17:10:44.789 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:44.789 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:10:44.790 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.792+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 msgr2=0x7f45e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.792+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 0x7f45e40a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f460019b320 tx=0x7f45e8005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.792+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 msgr2=0x7f4600199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.792+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d950 tx=0x7f45f000de10 comp rx=0 tx=0).stop 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 shutdown_connections 2024-04-02T17:10:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e40a3cc0 0x7f45e40a6180 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46001076f0 0x7f46001a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4600104dc0 0x7f460019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 --2- 172.21.15.59:0/937407564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4600102490 0x7f4600199e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 >> 172.21.15.59:0/937407564 conn(0x7f46000fbda0 msgr2=0x7f4600101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 shutdown_connections 2024-04-02T17:10:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:44.793+0000 7f4606600640 1 -- 172.21.15.59:0/937407564 wait complete. 2024-04-02T17:10:44.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:10:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 -- 172.21.15.59:0/1994226614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab40764c0 msgr2=0x7f8ab40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/1994226614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab40764c0 0x7f8ab40768e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009900 tx=0x7f8aa402bc30 comp rx=0 tx=0).stop 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 -- 172.21.15.59:0/1994226614 shutdown_connections 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/1994226614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40780a0 0x7f8ab4078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/1994226614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab4077700 0x7f8ab4077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/1994226614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab40764c0 0x7f8ab40768e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 -- 172.21.15.59:0/1994226614 >> 172.21.15.59:0/1994226614 conn(0x7f8ab40ff2f0 msgr2=0x7f8ab4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 -- 172.21.15.59:0/1994226614 shutdown_connections 2024-04-02T17:10:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.033+0000 7f8ab8a00640 1 -- 172.21.15.59:0/1994226614 wait complete. 2024-04-02T17:10:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 Processor -- start 2024-04-02T17:10:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 -- start start 2024-04-02T17:10:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40764c0 0x7f8ab419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab40780a0 0x7f8ab41a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4113e10 con 0x7f8ab4077700 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4113c90 con 0x7f8ab40780a0 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.034+0000 7f8ab8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4113f90 con 0x7f8ab40764c0 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39392/0 (socket says 172.21.15.59:39392) 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 -- 172.21.15.59:0/4106295627 learned_addr learned my addr 172.21.15.59:0/4106295627 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1e00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40764c0 0x7f8ab419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab2800640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab40780a0 0x7f8ab41a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 -- 172.21.15.59:0/4106295627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40764c0 msgr2=0x7f8ab419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40764c0 0x7f8ab419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 -- 172.21.15.59:0/4106295627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab40780a0 msgr2=0x7f8ab41a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab40780a0 0x7f8ab41a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.035+0000 7f8ab1400640 1 -- 172.21.15.59:0/4106295627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab41a6020 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8ab1400640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000e9f0 tx=0x7f8aa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000cd80 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0010910 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab41a6310 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0018990 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.036+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a68b0 con 0x7f8ab4077700 2024-04-02T17:10:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.037+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab41107f0 con 0x7f8ab4077700 2024-04-02T17:10:45.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.039+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8aa0010ac0 con 0x7f8ab4077700 2024-04-02T17:10:45.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.041+0000 7f8aaaa00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 0x7f8a980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.041+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8aa00d0650 con 0x7f8ab4077700 2024-04-02T17:10:45.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.041+0000 7f8ab1e00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 0x7f8a980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.042+0000 7f8ab1e00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 0x7f8a980a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8aa402c240 tx=0x7f8aa402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:45.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.045+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8aa009ca30 con 0x7f8ab4077700 2024-04-02T17:10:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.263+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ab402d5c0 con 0x7f8ab4077700 2024-04-02T17:10:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.266+0000 7f8aaaa00640 1 -- 172.21.15.59:0/4106295627 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22221 (secure 0 0 0) 0x7f8aa00a0d30 con 0x7f8ab4077700 2024-04-02T17:10:45.266 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:45.266 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:10:36.009161+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:10:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 msgr2=0x7f8a980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 0x7f8a980a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8aa402c240 tx=0x7f8aa402df90 comp rx=0 tx=0).stop 2024-04-02T17:10:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 msgr2=0x7f8ab419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000e9f0 tx=0x7f8aa000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 shutdown_connections 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8a980a3cc0 0x7f8a980a6180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4077700 0x7f8ab419e940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ab40780a0 0x7f8ab41a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 --2- 172.21.15.59:0/4106295627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ab40764c0 0x7f8ab419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 >> 172.21.15.59:0/4106295627 conn(0x7f8ab40ff2f0 msgr2=0x7f8ab41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 shutdown_connections 2024-04-02T17:10:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.269+0000 7f8ab8a00640 1 -- 172.21.15.59:0/4106295627 wait complete. 2024-04-02T17:10:45.290 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T17:10:45.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T17:10:45.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:45 smithi059 ceph-mon[31664]: pgmap v314: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s wr, 20 op/s 2024-04-02T17:10:45.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:45 smithi059 ceph-mon[31664]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:45 smithi059 ceph-mon[31664]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T17:10:45.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:45 smithi059 ceph-mon[31664]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:45 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/937407564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:45 smithi154 ceph-mon[36479]: pgmap v314: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s wr, 20 op/s 2024-04-02T17:10:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:45 smithi154 ceph-mon[36479]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:45 smithi154 ceph-mon[36479]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T17:10:45.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:45 smithi154 ceph-mon[36479]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:45 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/937407564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 -- 172.21.15.59:0/1372327614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81058f0 msgr2=0x7f7fb810c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 --2- 172.21.15.59:0/1372327614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81058f0 0x7f7fb810c180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009be0 tx=0x7f7fb002bef0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 -- 172.21.15.59:0/1372327614 shutdown_connections 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 --2- 172.21.15.59:0/1372327614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81058f0 0x7f7fb810c180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 --2- 172.21.15.59:0/1372327614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb8104f30 0x7f7fb81053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 --2- 172.21.15.59:0/1372327614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8103d40 0x7f7fb8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 -- 172.21.15.59:0/1372327614 >> 172.21.15.59:0/1372327614 conn(0x7f7fb80ff2f0 msgr2=0x7f7fb8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 -- 172.21.15.59:0/1372327614 shutdown_connections 2024-04-02T17:10:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.519+0000 7f7fc0200640 1 -- 172.21.15.59:0/1372327614 wait complete. 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.520+0000 7f7fc0200640 1 Processor -- start 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.520+0000 7f7fc0200640 1 -- start start 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 0x7f7fb819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 0x7f7fb81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810f9a0 con 0x7f7fb8103d40 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810f820 con 0x7f7fb81058f0 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fc0200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810fb20 con 0x7f7fb8104f30 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39408/0 (socket says 172.21.15.59:39408) 2024-04-02T17:10:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 0x7f7fb819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 -- 172.21.15.59:0/858948922 learned_addr learned my addr 172.21.15.59:0/858948922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbe200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 0x7f7fb81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 -- 172.21.15.59:0/858948922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 msgr2=0x7f7fb819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 0x7f7fb819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 -- 172.21.15.59:0/858948922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 msgr2=0x7f7fb81a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 0x7f7fb81a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbd800640 1 -- 172.21.15.59:0/858948922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb81a1c00 con 0x7f7fb8103d40 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbce00640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 0x7f7fb819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.521+0000 7f7fbe200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 0x7f7fb81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fbd800640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00e9f0 tx=0x7f7fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00ccc0 con 0x7f7fb8103d40 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fac0035f0 con 0x7f7fb8103d40 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00ce60 con 0x7f7fb8103d40 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb81a1ef0 con 0x7f7fb8103d40 2024-04-02T17:10:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.522+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb81a22a0 con 0x7f7fb8103d40 2024-04-02T17:10:45.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.523+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb810b0e0 con 0x7f7fb8103d40 2024-04-02T17:10:45.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.525+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7fac0037a0 con 0x7f7fb8103d40 2024-04-02T17:10:45.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.528+0000 7f7fb6000640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 0x7f7fa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:45.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.528+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7fac0d06f0 con 0x7f7fb8103d40 2024-04-02T17:10:45.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.528+0000 7f7fbce00640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 0x7f7fa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:45.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.528+0000 7f7fbce00640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 0x7f7fa00a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7fb819b4f0 tx=0x7f7fa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:45.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.532+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7fac09d020 con 0x7f7fb8103d40 2024-04-02T17:10:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:45 smithi164 ceph-mon[36499]: pgmap v314: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s wr, 20 op/s 2024-04-02T17:10:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:45 smithi164 ceph-mon[36499]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:45 smithi164 ceph-mon[36499]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T17:10:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:45 smithi164 ceph-mon[36499]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:10:45.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:45 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/937407564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:10:45.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.732+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7fb8061f40 con 0x7f7fa00a3c40 2024-04-02T17:10:45.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.748+0000 7f7fb6000640 1 -- 172.21.15.59:0/858948922 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258774 (secure 0 0 0) 0x7f7fb8061f40 con 0x7f7fa00a3c40 2024-04-02T17:10:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 msgr2=0x7f7fa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 0x7f7fa00a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7fb819b4f0 tx=0x7f7fa4005f90 comp rx=0 tx=0).stop 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 msgr2=0x7f7fb8199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00e9f0 tx=0x7f7fac00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 shutdown_connections 2024-04-02T17:10:45.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb81058f0 0x7f7fb81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb8104f30 0x7f7fb819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fa00a3c40 0x7f7fa00a6100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 --2- 172.21.15.59:0/858948922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb8103d40 0x7f7fb8199fe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 >> 172.21.15.59:0/858948922 conn(0x7f7fb80ff2f0 msgr2=0x7f7fb81082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 shutdown_connections 2024-04-02T17:10:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:45.753+0000 7f7fc0200640 1 -- 172.21.15.59:0/858948922 wait complete. 2024-04-02T17:10:45.771 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-04-02T17:10:45.492351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618550,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":145,"num_read_kb":122,"num_write":391,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383076,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558748,"statfs":{"total":1151940427776,"available":1151548157952,"internally_reserved":0,"allocated":13438976,"data_stored":7235236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28182,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":210,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.378179+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:10:36.378179+0000","last_peered":"2024-04-02T17:10:36.378179+0000","last_clean":"2024-04-02T17:10:36.378179+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:10:36.378179+0000","last_undegraded":"2024-04-02T17:10:36.378179+0000","last_fullsized":"2024-04-02T17:10:36.378179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018576+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:10:36.018576+0000","last_peered":"2024-04-02T17:10:36.018576+0000","last_clean":"2024-04-02T17:10:36.018576+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:10:36.018576+0000","last_undegraded":"2024-04-02T17:10:36.018576+0000","last_fullsized":"2024-04-02T17:10:36.018576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018479+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:10:36.018479+0000","last_peered":"2024-04-02T17:10:36.018479+0000","last_clean":"2024-04-02T17:10:36.018479+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:10:36.018479+0000","last_undegraded":"2024-04-02T17:10:36.018479+0000","last_fullsized":"2024-04-02T17:10:36.018479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765406+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:10:36.765406+0000","last_peered":"2024-04-02T17:10:36.765406+0000","last_clean":"2024-04-02T17:10:36.765406+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:10:36.765406+0000","last_undegraded":"2024-04-02T17:10:36.765406+0000","last_fullsized":"2024-04-02T17:10:36.765406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017617+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:10:36.017617+0000","last_peered":"2024-04-02T17:10:36.017617+0000","last_clean":"2024-04-02T17:10:36.017617+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:10:36.017617+0000","last_undegraded":"2024-04-02T17:10:36.017617+0000","last_fullsized":"2024-04-02T17:10:36.017617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765748+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:10:36.765748+0000","last_peered":"2024-04-02T17:10:36.765748+0000","last_clean":"2024-04-02T17:10:36.765748+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:10:36.765748+0000","last_undegraded":"2024-04-02T17:10:36.765748+0000","last_fullsized":"2024-04-02T17:10:36.765748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017952+0000","last_change":"2024-04-02T17:10:35.038283+0000","last_active":"2024-04-02T17:10:36.017952+0000","last_peered":"2024-04-02T17:10:36.017952+0000","last_clean":"2024-04-02T17:10:36.017952+0000","last_became_active":"2024-04-02T17:10:35.037917+0000","last_became_peered":"2024-04-02T17:10:35.037917+0000","last_unstale":"2024-04-02T17:10:36.017952+0000","last_undegraded":"2024-04-02T17:10:36.017952+0000","last_fullsized":"2024-04-02T17:10:36.017952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:03:50.879563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766917+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:10:36.766917+0000","last_peered":"2024-04-02T17:10:36.766917+0000","last_clean":"2024-04-02T17:10:36.766917+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:10:36.766917+0000","last_undegraded":"2024-04-02T17:10:36.766917+0000","last_fullsized":"2024-04-02T17:10:36.766917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264680+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:10:36.264680+0000","last_peered":"2024-04-02T17:10:36.264680+0000","last_clean":"2024-04-02T17:10:36.264680+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:10:36.264680+0000","last_undegraded":"2024-04-02T17:10:36.264680+0000","last_fullsized":"2024-04-02T17:10:36.264680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766109+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:10:36.766109+0000","last_peered":"2024-04-02T17:10:36.766109+0000","last_clean":"2024-04-02T17:10:36.766109+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:10:36.766109+0000","last_undegraded":"2024-04-02T17:10:36.766109+0000","last_fullsized":"2024-04-02T17:10:36.766109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018419+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:10:36.018419+0000","last_peered":"2024-04-02T17:10:36.018419+0000","last_clean":"2024-04-02T17:10:36.018419+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:10:36.018419+0000","last_undegraded":"2024-04-02T17:10:36.018419+0000","last_fullsized":"2024-04-02T17:10:36.018419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765363+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:10:36.765363+0000","last_peered":"2024-04-02T17:10:36.765363+0000","last_clean":"2024-04-02T17:10:36.765363+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:10:36.765363+0000","last_undegraded":"2024-04-02T17:10:36.765363+0000","last_fullsized":"2024-04-02T17:10:36.765363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770151+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:10:36.770151+0000","last_peered":"2024-04-02T17:10:36.770151+0000","last_clean":"2024-04-02T17:10:36.770151+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:10:36.770151+0000","last_undegraded":"2024-04-02T17:10:36.770151+0000","last_fullsized":"2024-04-02T17:10:36.770151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018252+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:10:36.018252+0000","last_peered":"2024-04-02T17:10:36.018252+0000","last_clean":"2024-04-02T17:10:36.018252+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:10:36.018252+0000","last_undegraded":"2024-04-02T17:10:36.018252+0000","last_fullsized":"2024-04-02T17:10:36.018252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017961+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:10:36.017961+0000","last_peered":"2024-04-02T17:10:36.017961+0000","last_clean":"2024-04-02T17:10:36.017961+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:10:36.017961+0000","last_undegraded":"2024-04-02T17:10:36.017961+0000","last_fullsized":"2024-04-02T17:10:36.017961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264163+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:10:36.264163+0000","last_peered":"2024-04-02T17:10:36.264163+0000","last_clean":"2024-04-02T17:10:36.264163+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:10:36.264163+0000","last_undegraded":"2024-04-02T17:10:36.264163+0000","last_fullsized":"2024-04-02T17:10:36.264163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.378147+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:10:36.378147+0000","last_peered":"2024-04-02T17:10:36.378147+0000","last_clean":"2024-04-02T17:10:36.378147+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:10:36.378147+0000","last_undegraded":"2024-04-02T17:10:36.378147+0000","last_fullsized":"2024-04-02T17:10:36.378147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.771439+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:10:36.771439+0000","last_peered":"2024-04-02T17:10:36.771439+0000","last_clean":"2024-04-02T17:10:36.771439+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:10:36.771439+0000","last_undegraded":"2024-04-02T17:10:36.771439+0000","last_fullsized":"2024-04-02T17:10:36.771439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765418+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:10:36.765418+0000","last_peered":"2024-04-02T17:10:36.765418+0000","last_clean":"2024-04-02T17:10:36.765418+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:10:36.765418+0000","last_undegraded":"2024-04-02T17:10:36.765418+0000","last_fullsized":"2024-04-02T17:10:36.765418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.090316+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:10:42.090316+0000","last_peered":"2024-04-02T17:10:42.090316+0000","last_clean":"2024-04-02T17:10:42.090316+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:10:42.090316+0000","last_undegraded":"2024-04-02T17:10:42.090316+0000","last_fullsized":"2024-04-02T17:10:42.090316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768969+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:10:36.768969+0000","last_peered":"2024-04-02T17:10:36.768969+0000","last_clean":"2024-04-02T17:10:36.768969+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:10:36.768969+0000","last_undegraded":"2024-04-02T17:10:36.768969+0000","last_fullsized":"2024-04-02T17:10:36.768969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.769394+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:10:36.769394+0000","last_peered":"2024-04-02T17:10:36.769394+0000","last_clean":"2024-04-02T17:10:36.769394+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:10:36.769394+0000","last_undegraded":"2024-04-02T17:10:36.769394+0000","last_fullsized":"2024-04-02T17:10:36.769394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.019034+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:10:36.019034+0000","last_peered":"2024-04-02T17:10:36.019034+0000","last_clean":"2024-04-02T17:10:36.019034+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:10:36.019034+0000","last_undegraded":"2024-04-02T17:10:36.019034+0000","last_fullsized":"2024-04-02T17:10:36.019034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085712+0000","last_change":"2024-04-02T17:10:34.025021+0000","last_active":"2024-04-02T17:10:42.085712+0000","last_peered":"2024-04-02T17:10:42.085712+0000","last_clean":"2024-04-02T17:10:42.085712+0000","last_became_active":"2024-04-02T17:10:34.024466+0000","last_became_peered":"2024-04-02T17:10:34.024466+0000","last_unstale":"2024-04-02T17:10:42.085712+0000","last_undegraded":"2024-04-02T17:10:42.085712+0000","last_fullsized":"2024-04-02T17:10:42.085712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:53.974216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264306+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:10:36.264306+0000","last_peered":"2024-04-02T17:10:36.264306+0000","last_clean":"2024-04-02T17:10:36.264306+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:10:36.264306+0000","last_undegraded":"2024-04-02T17:10:36.264306+0000","last_fullsized":"2024-04-02T17:10:36.264306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085106+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:10:42.085106+0000","last_peered":"2024-04-02T17:10:42.085106+0000","last_clean":"2024-04-02T17:10:42.085106+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:10:42.085106+0000","last_undegraded":"2024-04-02T17:10:42.085106+0000","last_fullsized":"2024-04-02T17:10:42.085106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766165+0000","last_change":"2024-04-02T17:10:35.030906+0000","last_active":"2024-04-02T17:10:36.766165+0000","last_peered":"2024-04-02T17:10:36.766165+0000","last_clean":"2024-04-02T17:10:36.766165+0000","last_became_active":"2024-04-02T17:10:35.030604+0000","last_became_peered":"2024-04-02T17:10:35.030604+0000","last_unstale":"2024-04-02T17:10:36.766165+0000","last_undegraded":"2024-04-02T17:10:36.766165+0000","last_fullsized":"2024-04-02T17:10:36.766165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:05.643498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:44.096439+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:10:44.096439+0000","last_peered":"2024-04-02T17:10:44.096439+0000","last_clean":"2024-04-02T17:10:44.096439+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:10:44.096439+0000","last_undegraded":"2024-04-02T17:10:44.096439+0000","last_fullsized":"2024-04-02T17:10:44.096439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017729+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:10:36.017729+0000","last_peered":"2024-04-02T17:10:36.017729+0000","last_clean":"2024-04-02T17:10:36.017729+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:10:36.017729+0000","last_undegraded":"2024-04-02T17:10:36.017729+0000","last_fullsized":"2024-04-02T17:10:36.017729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071681+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:10:41.071681+0000","last_peered":"2024-04-02T17:10:41.071681+0000","last_clean":"2024-04-02T17:10:41.071681+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:10:41.071681+0000","last_undegraded":"2024-04-02T17:10:41.071681+0000","last_fullsized":"2024-04-02T17:10:41.071681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766243+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:10:36.766243+0000","last_peered":"2024-04-02T17:10:36.766243+0000","last_clean":"2024-04-02T17:10:36.766243+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:10:36.766243+0000","last_undegraded":"2024-04-02T17:10:36.766243+0000","last_fullsized":"2024-04-02T17:10:36.766243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:40.062228+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:10:40.062228+0000","last_peered":"2024-04-02T17:10:40.062228+0000","last_clean":"2024-04-02T17:10:40.062228+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:10:40.062228+0000","last_undegraded":"2024-04-02T17:10:40.062228+0000","last_fullsized":"2024-04-02T17:10:40.062228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766985+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:10:36.766985+0000","last_peered":"2024-04-02T17:10:36.766985+0000","last_clean":"2024-04-02T17:10:36.766985+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:10:36.766985+0000","last_undegraded":"2024-04-02T17:10:36.766985+0000","last_fullsized":"2024-04-02T17:10:36.766985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085438+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:10:42.085438+0000","last_peered":"2024-04-02T17:10:42.085438+0000","last_clean":"2024-04-02T17:10:42.085438+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:10:42.085438+0000","last_undegraded":"2024-04-02T17:10:42.085438+0000","last_fullsized":"2024-04-02T17:10:42.085438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.769278+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:10:36.769278+0000","last_peered":"2024-04-02T17:10:36.769278+0000","last_clean":"2024-04-02T17:10:36.769278+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:10:36.769278+0000","last_undegraded":"2024-04-02T17:10:36.769278+0000","last_fullsized":"2024-04-02T17:10:36.769278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.051006+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:10:39.051006+0000","last_peered":"2024-04-02T17:10:39.051006+0000","last_clean":"2024-04-02T17:10:39.051006+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:10:39.051006+0000","last_undegraded":"2024-04-02T17:10:39.051006+0000","last_fullsized":"2024-04-02T17:10:39.051006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264616+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:10:36.264616+0000","last_peered":"2024-04-02T17:10:36.264616+0000","last_clean":"2024-04-02T17:10:36.264616+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:10:36.264616+0000","last_undegraded":"2024-04-02T17:10:36.264616+0000","last_fullsized":"2024-04-02T17:10:36.264616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.084978+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:10:42.084978+0000","last_peered":"2024-04-02T17:10:42.084978+0000","last_clean":"2024-04-02T17:10:42.084978+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:10:42.084978+0000","last_undegraded":"2024-04-02T17:10:42.084978+0000","last_fullsized":"2024-04-02T17:10:42.084978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.019209+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:10:36.019209+0000","last_peered":"2024-04-02T17:10:36.019209+0000","last_clean":"2024-04-02T17:10:36.019209+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:10:36.019209+0000","last_undegraded":"2024-04-02T17:10:36.019209+0000","last_fullsized":"2024-04-02T17:10:36.019209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.069483+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:10:41.069483+0000","last_peered":"2024-04-02T17:10:41.069483+0000","last_clean":"2024-04-02T17:10:41.069483+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:10:41.069483+0000","last_undegraded":"2024-04-02T17:10:41.069483+0000","last_fullsized":"2024-04-02T17:10:41.069483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264514+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:10:36.264514+0000","last_peered":"2024-04-02T17:10:36.264514+0000","last_clean":"2024-04-02T17:10:36.264514+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:10:36.264514+0000","last_undegraded":"2024-04-02T17:10:36.264514+0000","last_fullsized":"2024-04-02T17:10:36.264514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'17","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.630587+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:10:42.630587+0000","last_peered":"2024-04-02T17:10:42.630587+0000","last_clean":"2024-04-02T17:10:42.630587+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:10:42.630587+0000","last_undegraded":"2024-04-02T17:10:42.630587+0000","last_fullsized":"2024-04-02T17:10:42.630587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017841+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:10:36.017841+0000","last_peered":"2024-04-02T17:10:36.017841+0000","last_clean":"2024-04-02T17:10:36.017841+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:10:36.017841+0000","last_undegraded":"2024-04-02T17:10:36.017841+0000","last_fullsized":"2024-04-02T17:10:36.017841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:43.092370+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:10:43.092370+0000","last_peered":"2024-04-02T17:10:43.092370+0000","last_clean":"2024-04-02T17:10:43.092370+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:10:43.092370+0000","last_undegraded":"2024-04-02T17:10:43.092370+0000","last_fullsized":"2024-04-02T17:10:43.092370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770173+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:10:36.770173+0000","last_peered":"2024-04-02T17:10:36.770173+0000","last_clean":"2024-04-02T17:10:36.770173+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:10:36.770173+0000","last_undegraded":"2024-04-02T17:10:36.770173+0000","last_fullsized":"2024-04-02T17:10:36.770173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.092043+0000","last_change":"2024-04-02T17:10:34.035776+0000","last_active":"2024-04-02T17:10:42.092043+0000","last_peered":"2024-04-02T17:10:42.092043+0000","last_clean":"2024-04-02T17:10:42.092043+0000","last_became_active":"2024-04-02T17:10:34.035430+0000","last_became_peered":"2024-04-02T17:10:34.035430+0000","last_unstale":"2024-04-02T17:10:42.092043+0000","last_undegraded":"2024-04-02T17:10:42.092043+0000","last_fullsized":"2024-04-02T17:10:42.092043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:08:57.891441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018313+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:10:36.018313+0000","last_peered":"2024-04-02T17:10:36.018313+0000","last_clean":"2024-04-02T17:10:36.018313+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:10:36.018313+0000","last_undegraded":"2024-04-02T17:10:36.018313+0000","last_fullsized":"2024-04-02T17:10:36.018313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.064255+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:10:41.064255+0000","last_peered":"2024-04-02T17:10:41.064255+0000","last_clean":"2024-04-02T17:10:41.064255+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:10:41.064255+0000","last_undegraded":"2024-04-02T17:10:41.064255+0000","last_fullsized":"2024-04-02T17:10:41.064255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017919+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:10:36.017919+0000","last_peered":"2024-04-02T17:10:36.017919+0000","last_clean":"2024-04-02T17:10:36.017919+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:10:36.017919+0000","last_undegraded":"2024-04-02T17:10:36.017919+0000","last_fullsized":"2024-04-02T17:10:36.017919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071751+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:10:41.071751+0000","last_peered":"2024-04-02T17:10:41.071751+0000","last_clean":"2024-04-02T17:10:41.071751+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:10:41.071751+0000","last_undegraded":"2024-04-02T17:10:41.071751+0000","last_fullsized":"2024-04-02T17:10:41.071751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765842+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:10:36.765842+0000","last_peered":"2024-04-02T17:10:36.765842+0000","last_clean":"2024-04-02T17:10:36.765842+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:10:36.765842+0000","last_undegraded":"2024-04-02T17:10:36.765842+0000","last_fullsized":"2024-04-02T17:10:36.765842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.490743+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:10:37.490743+0000","last_peered":"2024-04-02T17:10:37.490743+0000","last_clean":"2024-04-02T17:10:37.490743+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:10:37.490743+0000","last_undegraded":"2024-04-02T17:10:37.490743+0000","last_fullsized":"2024-04-02T17:10:37.490743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377899+0000","last_change":"2024-04-02T17:10:35.035099+0000","last_active":"2024-04-02T17:10:36.377899+0000","last_peered":"2024-04-02T17:10:36.377899+0000","last_clean":"2024-04-02T17:10:36.377899+0000","last_became_active":"2024-04-02T17:10:35.034810+0000","last_became_peered":"2024-04-02T17:10:35.034810+0000","last_unstale":"2024-04-02T17:10:36.377899+0000","last_undegraded":"2024-04-02T17:10:36.377899+0000","last_fullsized":"2024-04-02T17:10:36.377899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:11:50.769301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.302961+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:10:37.302961+0000","last_peered":"2024-04-02T17:10:37.302961+0000","last_clean":"2024-04-02T17:10:37.302961+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:10:37.302961+0000","last_undegraded":"2024-04-02T17:10:37.302961+0000","last_fullsized":"2024-04-02T17:10:37.302961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.950000+0000","last_change":"2024-04-02T17:10:35.043369+0000","last_active":"2024-04-02T17:10:37.950000+0000","last_peered":"2024-04-02T17:10:37.950000+0000","last_clean":"2024-04-02T17:10:37.950000+0000","last_became_active":"2024-04-02T17:10:35.042985+0000","last_became_peered":"2024-04-02T17:10:35.042985+0000","last_unstale":"2024-04-02T17:10:37.950000+0000","last_undegraded":"2024-04-02T17:10:37.950000+0000","last_fullsized":"2024-04-02T17:10:37.950000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:55:21.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.095100+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:10:42.095100+0000","last_peered":"2024-04-02T17:10:42.095100+0000","last_clean":"2024-04-02T17:10:42.095100+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:10:42.095100+0000","last_undegraded":"2024-04-02T17:10:42.095100+0000","last_fullsized":"2024-04-02T17:10:42.095100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766022+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:10:36.766022+0000","last_peered":"2024-04-02T17:10:36.766022+0000","last_clean":"2024-04-02T17:10:36.766022+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:10:36.766022+0000","last_undegraded":"2024-04-02T17:10:36.766022+0000","last_fullsized":"2024-04-02T17:10:36.766022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765688+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:10:36.765688+0000","last_peered":"2024-04-02T17:10:36.765688+0000","last_clean":"2024-04-02T17:10:36.765688+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:10:36.765688+0000","last_undegraded":"2024-04-02T17:10:36.765688+0000","last_fullsized":"2024-04-02T17:10:36.765688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:43.094503+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:10:43.094503+0000","last_peered":"2024-04-02T17:10:43.094503+0000","last_clean":"2024-04-02T17:10:43.094503+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:10:43.094503+0000","last_undegraded":"2024-04-02T17:10:43.094503+0000","last_fullsized":"2024-04-02T17:10:43.094503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767423+0000","last_change":"2024-04-02T17:10:35.031970+0000","last_active":"2024-04-02T17:10:36.767423+0000","last_peered":"2024-04-02T17:10:36.767423+0000","last_clean":"2024-04-02T17:10:36.767423+0000","last_became_active":"2024-04-02T17:10:35.031527+0000","last_became_peered":"2024-04-02T17:10:35.031527+0000","last_unstale":"2024-04-02T17:10:36.767423+0000","last_undegraded":"2024-04-02T17:10:36.767423+0000","last_fullsized":"2024-04-02T17:10:36.767423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:30:54.247319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.068113+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:10:41.068113+0000","last_peered":"2024-04-02T17:10:41.068113+0000","last_clean":"2024-04-02T17:10:41.068113+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:10:41.068113+0000","last_undegraded":"2024-04-02T17:10:41.068113+0000","last_fullsized":"2024-04-02T17:10:41.068113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017823+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:10:36.017823+0000","last_peered":"2024-04-02T17:10:36.017823+0000","last_clean":"2024-04-02T17:10:36.017823+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:10:36.017823+0000","last_undegraded":"2024-04-02T17:10:36.017823+0000","last_fullsized":"2024-04-02T17:10:36.017823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768828+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:10:36.768828+0000","last_peered":"2024-04-02T17:10:36.768828+0000","last_clean":"2024-04-02T17:10:36.768828+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:10:36.768828+0000","last_undegraded":"2024-04-02T17:10:36.768828+0000","last_fullsized":"2024-04-02T17:10:36.768828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:40.056155+0000","last_change":"2024-04-02T17:10:34.025757+0000","last_active":"2024-04-02T17:10:40.056155+0000","last_peered":"2024-04-02T17:10:40.056155+0000","last_clean":"2024-04-02T17:10:40.056155+0000","last_became_active":"2024-04-02T17:10:34.025441+0000","last_became_peered":"2024-04-02T17:10:34.025441+0000","last_unstale":"2024-04-02T17:10:40.056155+0000","last_undegraded":"2024-04-02T17:10:40.056155+0000","last_fullsized":"2024-04-02T17:10:40.056155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:37.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.072018+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:10:41.072018+0000","last_peered":"2024-04-02T17:10:41.072018+0000","last_clean":"2024-04-02T17:10:41.072018+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:10:41.072018+0000","last_undegraded":"2024-04-02T17:10:41.072018+0000","last_fullsized":"2024-04-02T17:10:41.072018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770736+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:10:36.770736+0000","last_peered":"2024-04-02T17:10:36.770736+0000","last_clean":"2024-04-02T17:10:36.770736+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:10:36.770736+0000","last_undegraded":"2024-04-02T17:10:36.770736+0000","last_fullsized":"2024-04-02T17:10:36.770736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071087+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:10:41.071087+0000","last_peered":"2024-04-02T17:10:41.071087+0000","last_clean":"2024-04-02T17:10:41.071087+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:10:41.071087+0000","last_undegraded":"2024-04-02T17:10:41.071087+0000","last_fullsized":"2024-04-02T17:10:41.071087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768090+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:10:36.768090+0000","last_peered":"2024-04-02T17:10:36.768090+0000","last_clean":"2024-04-02T17:10:36.768090+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:10:36.768090+0000","last_undegraded":"2024-04-02T17:10:36.768090+0000","last_fullsized":"2024-04-02T17:10:36.768090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.045937+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:10:39.045937+0000","last_peered":"2024-04-02T17:10:39.045937+0000","last_clean":"2024-04-02T17:10:39.045937+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:10:39.045937+0000","last_undegraded":"2024-04-02T17:10:39.045937+0000","last_fullsized":"2024-04-02T17:10:39.045937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018798+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:10:36.018798+0000","last_peered":"2024-04-02T17:10:36.018798+0000","last_clean":"2024-04-02T17:10:36.018798+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:10:36.018798+0000","last_undegraded":"2024-04-02T17:10:36.018798+0000","last_fullsized":"2024-04-02T17:10:36.018798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.052209+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:10:39.052209+0000","last_peered":"2024-04-02T17:10:39.052209+0000","last_clean":"2024-04-02T17:10:39.052209+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:10:39.052209+0000","last_undegraded":"2024-04-02T17:10:39.052209+0000","last_fullsized":"2024-04-02T17:10:39.052209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017587+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:10:36.017587+0000","last_peered":"2024-04-02T17:10:36.017587+0000","last_clean":"2024-04-02T17:10:36.017587+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:10:36.017587+0000","last_undegraded":"2024-04-02T17:10:36.017587+0000","last_fullsized":"2024-04-02T17:10:36.017587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.094229+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:10:42.094229+0000","last_peered":"2024-04-02T17:10:42.094229+0000","last_clean":"2024-04-02T17:10:42.094229+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:10:42.094229+0000","last_undegraded":"2024-04-02T17:10:42.094229+0000","last_fullsized":"2024-04-02T17:10:42.094229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377786+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:10:36.377786+0000","last_peered":"2024-04-02T17:10:36.377786+0000","last_clean":"2024-04-02T17:10:36.377786+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:10:36.377786+0000","last_undegraded":"2024-04-02T17:10:36.377786+0000","last_fullsized":"2024-04-02T17:10:36.377786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.070886+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:10:41.070886+0000","last_peered":"2024-04-02T17:10:41.070886+0000","last_clean":"2024-04-02T17:10:41.070886+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:10:41.070886+0000","last_undegraded":"2024-04-02T17:10:41.070886+0000","last_fullsized":"2024-04-02T17:10:41.070886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767903+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:10:36.767903+0000","last_peered":"2024-04-02T17:10:36.767903+0000","last_clean":"2024-04-02T17:10:36.767903+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:10:36.767903+0000","last_undegraded":"2024-04-02T17:10:36.767903+0000","last_fullsized":"2024-04-02T17:10:36.767903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.045295+0000","last_change":"2024-04-02T17:10:34.023829+0000","last_active":"2024-04-02T17:10:39.045295+0000","last_peered":"2024-04-02T17:10:39.045295+0000","last_clean":"2024-04-02T17:10:39.045295+0000","last_became_active":"2024-04-02T17:10:34.023472+0000","last_became_peered":"2024-04-02T17:10:34.023472+0000","last_unstale":"2024-04-02T17:10:39.045295+0000","last_undegraded":"2024-04-02T17:10:39.045295+0000","last_fullsized":"2024-04-02T17:10:39.045295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:11.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018352+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:10:36.018352+0000","last_peered":"2024-04-02T17:10:36.018352+0000","last_clean":"2024-04-02T17:10:36.018352+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:10:36.018352+0000","last_undegraded":"2024-04-02T17:10:36.018352+0000","last_fullsized":"2024-04-02T17:10:36.018352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:44.096844+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:10:44.096844+0000","last_peered":"2024-04-02T17:10:44.096844+0000","last_clean":"2024-04-02T17:10:44.096844+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:10:44.096844+0000","last_undegraded":"2024-04-02T17:10:44.096844+0000","last_fullsized":"2024-04-02T17:10:44.096844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018364+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:10:36.018364+0000","last_peered":"2024-04-02T17:10:36.018364+0000","last_clean":"2024-04-02T17:10:36.018364+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:10:36.018364+0000","last_undegraded":"2024-04-02T17:10:36.018364+0000","last_fullsized":"2024-04-02T17:10:36.018364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.092046+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:10:42.092046+0000","last_peered":"2024-04-02T17:10:42.092046+0000","last_clean":"2024-04-02T17:10:42.092046+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:10:42.092046+0000","last_undegraded":"2024-04-02T17:10:42.092046+0000","last_fullsized":"2024-04-02T17:10:42.092046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018262+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:10:36.018262+0000","last_peered":"2024-04-02T17:10:36.018262+0000","last_clean":"2024-04-02T17:10:36.018262+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:10:36.018262+0000","last_undegraded":"2024-04-02T17:10:36.018262+0000","last_fullsized":"2024-04-02T17:10:36.018262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.095201+0000","last_change":"2024-04-02T17:10:34.044886+0000","last_active":"2024-04-02T17:10:42.095201+0000","last_peered":"2024-04-02T17:10:42.095201+0000","last_clean":"2024-04-02T17:10:42.095201+0000","last_became_active":"2024-04-02T17:10:34.043574+0000","last_became_peered":"2024-04-02T17:10:34.043574+0000","last_unstale":"2024-04-02T17:10:42.095201+0000","last_undegraded":"2024-04-02T17:10:42.095201+0000","last_fullsized":"2024-04-02T17:10:42.095201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:58:45.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018413+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:10:36.018413+0000","last_peered":"2024-04-02T17:10:36.018413+0000","last_clean":"2024-04-02T17:10:36.018413+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:10:36.018413+0000","last_undegraded":"2024-04-02T17:10:36.018413+0000","last_fullsized":"2024-04-02T17:10:36.018413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377993+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:10:36.377993+0000","last_peered":"2024-04-02T17:10:36.377993+0000","last_clean":"2024-04-02T17:10:36.377993+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:10:36.377993+0000","last_undegraded":"2024-04-02T17:10:36.377993+0000","last_fullsized":"2024-04-02T17:10:36.377993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765100+0000","last_change":"2024-04-02T17:10:35.035008+0000","last_active":"2024-04-02T17:10:36.765100+0000","last_peered":"2024-04-02T17:10:36.765100+0000","last_clean":"2024-04-02T17:10:36.765100+0000","last_became_active":"2024-04-02T17:10:35.034692+0000","last_became_peered":"2024-04-02T17:10:35.034692+0000","last_unstale":"2024-04-02T17:10:36.765100+0000","last_undegraded":"2024-04-02T17:10:36.765100+0000","last_fullsized":"2024-04-02T17:10:36.765100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:18:58.457401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765274+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:10:36.765274+0000","last_peered":"2024-04-02T17:10:36.765274+0000","last_clean":"2024-04-02T17:10:36.765274+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:10:36.765274+0000","last_undegraded":"2024-04-02T17:10:36.765274+0000","last_fullsized":"2024-04-02T17:10:36.765274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018803+0000","last_change":"2024-04-02T17:10:35.049055+0000","last_active":"2024-04-02T17:10:36.018803+0000","last_peered":"2024-04-02T17:10:36.018803+0000","last_clean":"2024-04-02T17:10:36.018803+0000","last_became_active":"2024-04-02T17:10:35.048454+0000","last_became_peered":"2024-04-02T17:10:35.048454+0000","last_unstale":"2024-04-02T17:10:36.018803+0000","last_undegraded":"2024-04-02T17:10:36.018803+0000","last_fullsized":"2024-04-02T17:10:36.018803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:36:14.686282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770688+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:10:36.770688+0000","last_peered":"2024-04-02T17:10:36.770688+0000","last_clean":"2024-04-02T17:10:36.770688+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:10:36.770688+0000","last_undegraded":"2024-04-02T17:10:36.770688+0000","last_fullsized":"2024-04-02T17:10:36.770688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377945+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:10:36.377945+0000","last_peered":"2024-04-02T17:10:36.377945+0000","last_clean":"2024-04-02T17:10:36.377945+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:10:36.377945+0000","last_undegraded":"2024-04-02T17:10:36.377945+0000","last_fullsized":"2024-04-02T17:10:36.377945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765725+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:10:36.765725+0000","last_peered":"2024-04-02T17:10:36.765725+0000","last_clean":"2024-04-02T17:10:36.765725+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:10:36.765725+0000","last_undegraded":"2024-04-02T17:10:36.765725+0000","last_fullsized":"2024-04-02T17:10:36.765725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767839+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:10:36.767839+0000","last_peered":"2024-04-02T17:10:36.767839+0000","last_clean":"2024-04-02T17:10:36.767839+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:10:36.767839+0000","last_undegraded":"2024-04-02T17:10:36.767839+0000","last_fullsized":"2024-04-02T17:10:36.767839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765651+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:10:36.765651+0000","last_peered":"2024-04-02T17:10:36.765651+0000","last_clean":"2024-04-02T17:10:36.765651+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:10:36.765651+0000","last_undegraded":"2024-04-02T17:10:36.765651+0000","last_fullsized":"2024-04-02T17:10:36.765651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767086+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:10:36.767086+0000","last_peered":"2024-04-02T17:10:36.767086+0000","last_clean":"2024-04-02T17:10:36.767086+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:10:36.767086+0000","last_undegraded":"2024-04-02T17:10:36.767086+0000","last_fullsized":"2024-04-02T17:10:36.767086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264436+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:10:36.264436+0000","last_peered":"2024-04-02T17:10:36.264436+0000","last_clean":"2024-04-02T17:10:36.264436+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:10:36.264436+0000","last_undegraded":"2024-04-02T17:10:36.264436+0000","last_fullsized":"2024-04-02T17:10:36.264436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770354+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:10:36.770354+0000","last_peered":"2024-04-02T17:10:36.770354+0000","last_clean":"2024-04-02T17:10:36.770354+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:10:36.770354+0000","last_undegraded":"2024-04-02T17:10:36.770354+0000","last_fullsized":"2024-04-02T17:10:36.770354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377850+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:10:36.377850+0000","last_peered":"2024-04-02T17:10:36.377850+0000","last_clean":"2024-04-02T17:10:36.377850+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:10:36.377850+0000","last_undegraded":"2024-04-02T17:10:36.377850+0000","last_fullsized":"2024-04-02T17:10:36.377850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":28065,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":209,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":87342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":450862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579926,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1050377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":450578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1042893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":450509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":451282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":452239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":492052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":450334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":450336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1043366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:10:45.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T17:10:46.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.037+0000 7fa21f200640 1 -- 172.21.15.59:0/3773647792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 msgr2=0x7fa220077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:46.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.037+0000 7fa21f200640 1 --2- 172.21.15.59:0/3773647792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa220077b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009860 tx=0x7fa20c02bb70 comp rx=0 tx=0).stop 2024-04-02T17:10:46.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 -- 172.21.15.59:0/3773647792 shutdown_connections 2024-04-02T17:10:46.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 --2- 172.21.15.59:0/3773647792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 0x7fa220078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 --2- 172.21.15.59:0/3773647792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa220077b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 --2- 172.21.15.59:0/3773647792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa2200768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 -- 172.21.15.59:0/3773647792 >> 172.21.15.59:0/3773647792 conn(0x7fa2200ff0e0 msgr2=0x7fa220101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 -- 172.21.15.59:0/3773647792 shutdown_connections 2024-04-02T17:10:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.038+0000 7fa21f200640 1 -- 172.21.15.59:0/3773647792 wait complete. 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.039+0000 7fa21f200640 1 Processor -- start 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.039+0000 7fa21f200640 1 -- start start 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.039+0000 7fa21f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa22019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 0x7fa2201a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113ba0 con 0x7fa2200764c0 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113a20 con 0x7fa2200780a0 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113d20 con 0x7fa220077700 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa22019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39422/0 (socket says 172.21.15.59:39422) 2024-04-02T17:10:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21de00640 1 -- 172.21.15.59:0/3082709517 learned_addr learned my addr 172.21.15.59:0/3082709517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21e800640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 0x7fa2201a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 -- 172.21.15.59:0/3082709517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 msgr2=0x7fa2201a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 0x7fa2201a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 -- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 msgr2=0x7fa22019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21d400640 1 -- 172.21.15.59:0/3082709517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2201a5e40 con 0x7fa220077700 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.040+0000 7fa21de00640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:46.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa21d400640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa22019e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa20c02c0b0 tx=0x7fa20c032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c041020 con 0x7fa220077700 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201a60d0 con 0x7fa220077700 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20c02c9f0 con 0x7fa220077700 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c039040 con 0x7fa220077700 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.041+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a65c0 con 0x7fa220077700 2024-04-02T17:10:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.043+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa220110670 con 0x7fa220077700 2024-04-02T17:10:46.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.044+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa20c044020 con 0x7fa220077700 2024-04-02T17:10:46.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.046+0000 7fa216a00640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 0x7fa2040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.047+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa20c0f1fe0 con 0x7fa220077700 2024-04-02T17:10:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.047+0000 7fa21de00640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 0x7fa2040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.047+0000 7fa21de00640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 0x7fa2040a6100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2100096c0 tx=0x7fa210009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.049+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa20c04b020 con 0x7fa220077700 2024-04-02T17:10:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.254+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa220061f40 con 0x7fa2040a3c40 2024-04-02T17:10:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.274+0000 7fa216a00640 1 -- 172.21.15.59:0/3082709517 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258774 (secure 0 0 0) 0x7fa220061f40 con 0x7fa2040a3c40 2024-04-02T17:10:46.277 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:10:46.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 msgr2=0x7fa2040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 0x7fa2040a6100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2100096c0 tx=0x7fa210009260 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 msgr2=0x7fa22019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa22019e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa20c02c0b0 tx=0x7fa20c032430 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 shutdown_connections 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2200780a0 0x7fa2201a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220077700 0x7fa22019e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2040a3c40 0x7fa2040a6100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 --2- 172.21.15.59:0/3082709517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2200764c0 0x7fa22019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 >> 172.21.15.59:0/3082709517 conn(0x7fa2200ff0e0 msgr2=0x7fa220100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 shutdown_connections 2024-04-02T17:10:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:46.280+0000 7fa21f200640 1 -- 172.21.15.59:0/3082709517 wait complete. 2024-04-02T17:10:46.300 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-04-02T17:10:45.492351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618550,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":145,"num_read_kb":122,"num_write":391,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383076,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558748,"statfs":{"total":1151940427776,"available":1151548157952,"internally_reserved":0,"allocated":13438976,"data_stored":7235236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28182,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":210,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.378179+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:10:36.378179+0000","last_peered":"2024-04-02T17:10:36.378179+0000","last_clean":"2024-04-02T17:10:36.378179+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:10:36.378179+0000","last_undegraded":"2024-04-02T17:10:36.378179+0000","last_fullsized":"2024-04-02T17:10:36.378179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018576+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:10:36.018576+0000","last_peered":"2024-04-02T17:10:36.018576+0000","last_clean":"2024-04-02T17:10:36.018576+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:10:36.018576+0000","last_undegraded":"2024-04-02T17:10:36.018576+0000","last_fullsized":"2024-04-02T17:10:36.018576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018479+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:10:36.018479+0000","last_peered":"2024-04-02T17:10:36.018479+0000","last_clean":"2024-04-02T17:10:36.018479+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:10:36.018479+0000","last_undegraded":"2024-04-02T17:10:36.018479+0000","last_fullsized":"2024-04-02T17:10:36.018479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765406+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:10:36.765406+0000","last_peered":"2024-04-02T17:10:36.765406+0000","last_clean":"2024-04-02T17:10:36.765406+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:10:36.765406+0000","last_undegraded":"2024-04-02T17:10:36.765406+0000","last_fullsized":"2024-04-02T17:10:36.765406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017617+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:10:36.017617+0000","last_peered":"2024-04-02T17:10:36.017617+0000","last_clean":"2024-04-02T17:10:36.017617+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:10:36.017617+0000","last_undegraded":"2024-04-02T17:10:36.017617+0000","last_fullsized":"2024-04-02T17:10:36.017617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765748+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:10:36.765748+0000","last_peered":"2024-04-02T17:10:36.765748+0000","last_clean":"2024-04-02T17:10:36.765748+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:10:36.765748+0000","last_undegraded":"2024-04-02T17:10:36.765748+0000","last_fullsized":"2024-04-02T17:10:36.765748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017952+0000","last_change":"2024-04-02T17:10:35.038283+0000","last_active":"2024-04-02T17:10:36.017952+0000","last_peered":"2024-04-02T17:10:36.017952+0000","last_clean":"2024-04-02T17:10:36.017952+0000","last_became_active":"2024-04-02T17:10:35.037917+0000","last_became_peered":"2024-04-02T17:10:35.037917+0000","last_unstale":"2024-04-02T17:10:36.017952+0000","last_undegraded":"2024-04-02T17:10:36.017952+0000","last_fullsized":"2024-04-02T17:10:36.017952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:03:50.879563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766917+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:10:36.766917+0000","last_peered":"2024-04-02T17:10:36.766917+0000","last_clean":"2024-04-02T17:10:36.766917+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:10:36.766917+0000","last_undegraded":"2024-04-02T17:10:36.766917+0000","last_fullsized":"2024-04-02T17:10:36.766917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264680+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:10:36.264680+0000","last_peered":"2024-04-02T17:10:36.264680+0000","last_clean":"2024-04-02T17:10:36.264680+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:10:36.264680+0000","last_undegraded":"2024-04-02T17:10:36.264680+0000","last_fullsized":"2024-04-02T17:10:36.264680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766109+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:10:36.766109+0000","last_peered":"2024-04-02T17:10:36.766109+0000","last_clean":"2024-04-02T17:10:36.766109+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:10:36.766109+0000","last_undegraded":"2024-04-02T17:10:36.766109+0000","last_fullsized":"2024-04-02T17:10:36.766109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018419+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:10:36.018419+0000","last_peered":"2024-04-02T17:10:36.018419+0000","last_clean":"2024-04-02T17:10:36.018419+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:10:36.018419+0000","last_undegraded":"2024-04-02T17:10:36.018419+0000","last_fullsized":"2024-04-02T17:10:36.018419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765363+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:10:36.765363+0000","last_peered":"2024-04-02T17:10:36.765363+0000","last_clean":"2024-04-02T17:10:36.765363+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:10:36.765363+0000","last_undegraded":"2024-04-02T17:10:36.765363+0000","last_fullsized":"2024-04-02T17:10:36.765363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770151+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:10:36.770151+0000","last_peered":"2024-04-02T17:10:36.770151+0000","last_clean":"2024-04-02T17:10:36.770151+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:10:36.770151+0000","last_undegraded":"2024-04-02T17:10:36.770151+0000","last_fullsized":"2024-04-02T17:10:36.770151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018252+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:10:36.018252+0000","last_peered":"2024-04-02T17:10:36.018252+0000","last_clean":"2024-04-02T17:10:36.018252+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:10:36.018252+0000","last_undegraded":"2024-04-02T17:10:36.018252+0000","last_fullsized":"2024-04-02T17:10:36.018252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017961+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:10:36.017961+0000","last_peered":"2024-04-02T17:10:36.017961+0000","last_clean":"2024-04-02T17:10:36.017961+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:10:36.017961+0000","last_undegraded":"2024-04-02T17:10:36.017961+0000","last_fullsized":"2024-04-02T17:10:36.017961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264163+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:10:36.264163+0000","last_peered":"2024-04-02T17:10:36.264163+0000","last_clean":"2024-04-02T17:10:36.264163+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:10:36.264163+0000","last_undegraded":"2024-04-02T17:10:36.264163+0000","last_fullsized":"2024-04-02T17:10:36.264163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.378147+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:10:36.378147+0000","last_peered":"2024-04-02T17:10:36.378147+0000","last_clean":"2024-04-02T17:10:36.378147+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:10:36.378147+0000","last_undegraded":"2024-04-02T17:10:36.378147+0000","last_fullsized":"2024-04-02T17:10:36.378147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.771439+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:10:36.771439+0000","last_peered":"2024-04-02T17:10:36.771439+0000","last_clean":"2024-04-02T17:10:36.771439+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:10:36.771439+0000","last_undegraded":"2024-04-02T17:10:36.771439+0000","last_fullsized":"2024-04-02T17:10:36.771439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765418+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:10:36.765418+0000","last_peered":"2024-04-02T17:10:36.765418+0000","last_clean":"2024-04-02T17:10:36.765418+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:10:36.765418+0000","last_undegraded":"2024-04-02T17:10:36.765418+0000","last_fullsized":"2024-04-02T17:10:36.765418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.090316+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:10:42.090316+0000","last_peered":"2024-04-02T17:10:42.090316+0000","last_clean":"2024-04-02T17:10:42.090316+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:10:42.090316+0000","last_undegraded":"2024-04-02T17:10:42.090316+0000","last_fullsized":"2024-04-02T17:10:42.090316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768969+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:10:36.768969+0000","last_peered":"2024-04-02T17:10:36.768969+0000","last_clean":"2024-04-02T17:10:36.768969+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:10:36.768969+0000","last_undegraded":"2024-04-02T17:10:36.768969+0000","last_fullsized":"2024-04-02T17:10:36.768969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.769394+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:10:36.769394+0000","last_peered":"2024-04-02T17:10:36.769394+0000","last_clean":"2024-04-02T17:10:36.769394+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:10:36.769394+0000","last_undegraded":"2024-04-02T17:10:36.769394+0000","last_fullsized":"2024-04-02T17:10:36.769394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.019034+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:10:36.019034+0000","last_peered":"2024-04-02T17:10:36.019034+0000","last_clean":"2024-04-02T17:10:36.019034+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:10:36.019034+0000","last_undegraded":"2024-04-02T17:10:36.019034+0000","last_fullsized":"2024-04-02T17:10:36.019034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085712+0000","last_change":"2024-04-02T17:10:34.025021+0000","last_active":"2024-04-02T17:10:42.085712+0000","last_peered":"2024-04-02T17:10:42.085712+0000","last_clean":"2024-04-02T17:10:42.085712+0000","last_became_active":"2024-04-02T17:10:34.024466+0000","last_became_peered":"2024-04-02T17:10:34.024466+0000","last_unstale":"2024-04-02T17:10:42.085712+0000","last_undegraded":"2024-04-02T17:10:42.085712+0000","last_fullsized":"2024-04-02T17:10:42.085712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:53.974216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264306+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:10:36.264306+0000","last_peered":"2024-04-02T17:10:36.264306+0000","last_clean":"2024-04-02T17:10:36.264306+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:10:36.264306+0000","last_undegraded":"2024-04-02T17:10:36.264306+0000","last_fullsized":"2024-04-02T17:10:36.264306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085106+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:10:42.085106+0000","last_peered":"2024-04-02T17:10:42.085106+0000","last_clean":"2024-04-02T17:10:42.085106+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:10:42.085106+0000","last_undegraded":"2024-04-02T17:10:42.085106+0000","last_fullsized":"2024-04-02T17:10:42.085106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766165+0000","last_change":"2024-04-02T17:10:35.030906+0000","last_active":"2024-04-02T17:10:36.766165+0000","last_peered":"2024-04-02T17:10:36.766165+0000","last_clean":"2024-04-02T17:10:36.766165+0000","last_became_active":"2024-04-02T17:10:35.030604+0000","last_became_peered":"2024-04-02T17:10:35.030604+0000","last_unstale":"2024-04-02T17:10:36.766165+0000","last_undegraded":"2024-04-02T17:10:36.766165+0000","last_fullsized":"2024-04-02T17:10:36.766165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:05.643498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:44.096439+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:10:44.096439+0000","last_peered":"2024-04-02T17:10:44.096439+0000","last_clean":"2024-04-02T17:10:44.096439+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:10:44.096439+0000","last_undegraded":"2024-04-02T17:10:44.096439+0000","last_fullsized":"2024-04-02T17:10:44.096439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017729+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:10:36.017729+0000","last_peered":"2024-04-02T17:10:36.017729+0000","last_clean":"2024-04-02T17:10:36.017729+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:10:36.017729+0000","last_undegraded":"2024-04-02T17:10:36.017729+0000","last_fullsized":"2024-04-02T17:10:36.017729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071681+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:10:41.071681+0000","last_peered":"2024-04-02T17:10:41.071681+0000","last_clean":"2024-04-02T17:10:41.071681+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:10:41.071681+0000","last_undegraded":"2024-04-02T17:10:41.071681+0000","last_fullsized":"2024-04-02T17:10:41.071681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766243+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:10:36.766243+0000","last_peered":"2024-04-02T17:10:36.766243+0000","last_clean":"2024-04-02T17:10:36.766243+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:10:36.766243+0000","last_undegraded":"2024-04-02T17:10:36.766243+0000","last_fullsized":"2024-04-02T17:10:36.766243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:40.062228+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:10:40.062228+0000","last_peered":"2024-04-02T17:10:40.062228+0000","last_clean":"2024-04-02T17:10:40.062228+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:10:40.062228+0000","last_undegraded":"2024-04-02T17:10:40.062228+0000","last_fullsized":"2024-04-02T17:10:40.062228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766985+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:10:36.766985+0000","last_peered":"2024-04-02T17:10:36.766985+0000","last_clean":"2024-04-02T17:10:36.766985+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:10:36.766985+0000","last_undegraded":"2024-04-02T17:10:36.766985+0000","last_fullsized":"2024-04-02T17:10:36.766985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.085438+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:10:42.085438+0000","last_peered":"2024-04-02T17:10:42.085438+0000","last_clean":"2024-04-02T17:10:42.085438+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:10:42.085438+0000","last_undegraded":"2024-04-02T17:10:42.085438+0000","last_fullsized":"2024-04-02T17:10:42.085438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.769278+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:10:36.769278+0000","last_peered":"2024-04-02T17:10:36.769278+0000","last_clean":"2024-04-02T17:10:36.769278+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:10:36.769278+0000","last_undegraded":"2024-04-02T17:10:36.769278+0000","last_fullsized":"2024-04-02T17:10:36.769278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.051006+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:10:39.051006+0000","last_peered":"2024-04-02T17:10:39.051006+0000","last_clean":"2024-04-02T17:10:39.051006+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:10:39.051006+0000","last_undegraded":"2024-04-02T17:10:39.051006+0000","last_fullsized":"2024-04-02T17:10:39.051006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264616+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:10:36.264616+0000","last_peered":"2024-04-02T17:10:36.264616+0000","last_clean":"2024-04-02T17:10:36.264616+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:10:36.264616+0000","last_undegraded":"2024-04-02T17:10:36.264616+0000","last_fullsized":"2024-04-02T17:10:36.264616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.084978+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:10:42.084978+0000","last_peered":"2024-04-02T17:10:42.084978+0000","last_clean":"2024-04-02T17:10:42.084978+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:10:42.084978+0000","last_undegraded":"2024-04-02T17:10:42.084978+0000","last_fullsized":"2024-04-02T17:10:42.084978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.019209+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:10:36.019209+0000","last_peered":"2024-04-02T17:10:36.019209+0000","last_clean":"2024-04-02T17:10:36.019209+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:10:36.019209+0000","last_undegraded":"2024-04-02T17:10:36.019209+0000","last_fullsized":"2024-04-02T17:10:36.019209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.069483+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:10:41.069483+0000","last_peered":"2024-04-02T17:10:41.069483+0000","last_clean":"2024-04-02T17:10:41.069483+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:10:41.069483+0000","last_undegraded":"2024-04-02T17:10:41.069483+0000","last_fullsized":"2024-04-02T17:10:41.069483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264514+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:10:36.264514+0000","last_peered":"2024-04-02T17:10:36.264514+0000","last_clean":"2024-04-02T17:10:36.264514+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:10:36.264514+0000","last_undegraded":"2024-04-02T17:10:36.264514+0000","last_fullsized":"2024-04-02T17:10:36.264514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'17","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.630587+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:10:42.630587+0000","last_peered":"2024-04-02T17:10:42.630587+0000","last_clean":"2024-04-02T17:10:42.630587+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:10:42.630587+0000","last_undegraded":"2024-04-02T17:10:42.630587+0000","last_fullsized":"2024-04-02T17:10:42.630587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017841+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:10:36.017841+0000","last_peered":"2024-04-02T17:10:36.017841+0000","last_clean":"2024-04-02T17:10:36.017841+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:10:36.017841+0000","last_undegraded":"2024-04-02T17:10:36.017841+0000","last_fullsized":"2024-04-02T17:10:36.017841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:43.092370+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:10:43.092370+0000","last_peered":"2024-04-02T17:10:43.092370+0000","last_clean":"2024-04-02T17:10:43.092370+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:10:43.092370+0000","last_undegraded":"2024-04-02T17:10:43.092370+0000","last_fullsized":"2024-04-02T17:10:43.092370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770173+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:10:36.770173+0000","last_peered":"2024-04-02T17:10:36.770173+0000","last_clean":"2024-04-02T17:10:36.770173+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:10:36.770173+0000","last_undegraded":"2024-04-02T17:10:36.770173+0000","last_fullsized":"2024-04-02T17:10:36.770173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.092043+0000","last_change":"2024-04-02T17:10:34.035776+0000","last_active":"2024-04-02T17:10:42.092043+0000","last_peered":"2024-04-02T17:10:42.092043+0000","last_clean":"2024-04-02T17:10:42.092043+0000","last_became_active":"2024-04-02T17:10:34.035430+0000","last_became_peered":"2024-04-02T17:10:34.035430+0000","last_unstale":"2024-04-02T17:10:42.092043+0000","last_undegraded":"2024-04-02T17:10:42.092043+0000","last_fullsized":"2024-04-02T17:10:42.092043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:08:57.891441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018313+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:10:36.018313+0000","last_peered":"2024-04-02T17:10:36.018313+0000","last_clean":"2024-04-02T17:10:36.018313+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:10:36.018313+0000","last_undegraded":"2024-04-02T17:10:36.018313+0000","last_fullsized":"2024-04-02T17:10:36.018313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.064255+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:10:41.064255+0000","last_peered":"2024-04-02T17:10:41.064255+0000","last_clean":"2024-04-02T17:10:41.064255+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:10:41.064255+0000","last_undegraded":"2024-04-02T17:10:41.064255+0000","last_fullsized":"2024-04-02T17:10:41.064255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017919+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:10:36.017919+0000","last_peered":"2024-04-02T17:10:36.017919+0000","last_clean":"2024-04-02T17:10:36.017919+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:10:36.017919+0000","last_undegraded":"2024-04-02T17:10:36.017919+0000","last_fullsized":"2024-04-02T17:10:36.017919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071751+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:10:41.071751+0000","last_peered":"2024-04-02T17:10:41.071751+0000","last_clean":"2024-04-02T17:10:41.071751+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:10:41.071751+0000","last_undegraded":"2024-04-02T17:10:41.071751+0000","last_fullsized":"2024-04-02T17:10:41.071751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765842+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:10:36.765842+0000","last_peered":"2024-04-02T17:10:36.765842+0000","last_clean":"2024-04-02T17:10:36.765842+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:10:36.765842+0000","last_undegraded":"2024-04-02T17:10:36.765842+0000","last_fullsized":"2024-04-02T17:10:36.765842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.490743+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:10:37.490743+0000","last_peered":"2024-04-02T17:10:37.490743+0000","last_clean":"2024-04-02T17:10:37.490743+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:10:37.490743+0000","last_undegraded":"2024-04-02T17:10:37.490743+0000","last_fullsized":"2024-04-02T17:10:37.490743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377899+0000","last_change":"2024-04-02T17:10:35.035099+0000","last_active":"2024-04-02T17:10:36.377899+0000","last_peered":"2024-04-02T17:10:36.377899+0000","last_clean":"2024-04-02T17:10:36.377899+0000","last_became_active":"2024-04-02T17:10:35.034810+0000","last_became_peered":"2024-04-02T17:10:35.034810+0000","last_unstale":"2024-04-02T17:10:36.377899+0000","last_undegraded":"2024-04-02T17:10:36.377899+0000","last_fullsized":"2024-04-02T17:10:36.377899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:11:50.769301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.302961+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:10:37.302961+0000","last_peered":"2024-04-02T17:10:37.302961+0000","last_clean":"2024-04-02T17:10:37.302961+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:10:37.302961+0000","last_undegraded":"2024-04-02T17:10:37.302961+0000","last_fullsized":"2024-04-02T17:10:37.302961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:37.950000+0000","last_change":"2024-04-02T17:10:35.043369+0000","last_active":"2024-04-02T17:10:37.950000+0000","last_peered":"2024-04-02T17:10:37.950000+0000","last_clean":"2024-04-02T17:10:37.950000+0000","last_became_active":"2024-04-02T17:10:35.042985+0000","last_became_peered":"2024-04-02T17:10:35.042985+0000","last_unstale":"2024-04-02T17:10:37.950000+0000","last_undegraded":"2024-04-02T17:10:37.950000+0000","last_fullsized":"2024-04-02T17:10:37.950000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:55:21.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.095100+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:10:42.095100+0000","last_peered":"2024-04-02T17:10:42.095100+0000","last_clean":"2024-04-02T17:10:42.095100+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:10:42.095100+0000","last_undegraded":"2024-04-02T17:10:42.095100+0000","last_fullsized":"2024-04-02T17:10:42.095100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.766022+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:10:36.766022+0000","last_peered":"2024-04-02T17:10:36.766022+0000","last_clean":"2024-04-02T17:10:36.766022+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:10:36.766022+0000","last_undegraded":"2024-04-02T17:10:36.766022+0000","last_fullsized":"2024-04-02T17:10:36.766022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765688+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:10:36.765688+0000","last_peered":"2024-04-02T17:10:36.765688+0000","last_clean":"2024-04-02T17:10:36.765688+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:10:36.765688+0000","last_undegraded":"2024-04-02T17:10:36.765688+0000","last_fullsized":"2024-04-02T17:10:36.765688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:43.094503+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:10:43.094503+0000","last_peered":"2024-04-02T17:10:43.094503+0000","last_clean":"2024-04-02T17:10:43.094503+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:10:43.094503+0000","last_undegraded":"2024-04-02T17:10:43.094503+0000","last_fullsized":"2024-04-02T17:10:43.094503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767423+0000","last_change":"2024-04-02T17:10:35.031970+0000","last_active":"2024-04-02T17:10:36.767423+0000","last_peered":"2024-04-02T17:10:36.767423+0000","last_clean":"2024-04-02T17:10:36.767423+0000","last_became_active":"2024-04-02T17:10:35.031527+0000","last_became_peered":"2024-04-02T17:10:35.031527+0000","last_unstale":"2024-04-02T17:10:36.767423+0000","last_undegraded":"2024-04-02T17:10:36.767423+0000","last_fullsized":"2024-04-02T17:10:36.767423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:30:54.247319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.068113+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:10:41.068113+0000","last_peered":"2024-04-02T17:10:41.068113+0000","last_clean":"2024-04-02T17:10:41.068113+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:10:41.068113+0000","last_undegraded":"2024-04-02T17:10:41.068113+0000","last_fullsized":"2024-04-02T17:10:41.068113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017823+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:10:36.017823+0000","last_peered":"2024-04-02T17:10:36.017823+0000","last_clean":"2024-04-02T17:10:36.017823+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:10:36.017823+0000","last_undegraded":"2024-04-02T17:10:36.017823+0000","last_fullsized":"2024-04-02T17:10:36.017823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768828+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:10:36.768828+0000","last_peered":"2024-04-02T17:10:36.768828+0000","last_clean":"2024-04-02T17:10:36.768828+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:10:36.768828+0000","last_undegraded":"2024-04-02T17:10:36.768828+0000","last_fullsized":"2024-04-02T17:10:36.768828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:40.056155+0000","last_change":"2024-04-02T17:10:34.025757+0000","last_active":"2024-04-02T17:10:40.056155+0000","last_peered":"2024-04-02T17:10:40.056155+0000","last_clean":"2024-04-02T17:10:40.056155+0000","last_became_active":"2024-04-02T17:10:34.025441+0000","last_became_peered":"2024-04-02T17:10:34.025441+0000","last_unstale":"2024-04-02T17:10:40.056155+0000","last_undegraded":"2024-04-02T17:10:40.056155+0000","last_fullsized":"2024-04-02T17:10:40.056155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:37.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.072018+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:10:41.072018+0000","last_peered":"2024-04-02T17:10:41.072018+0000","last_clean":"2024-04-02T17:10:41.072018+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:10:41.072018+0000","last_undegraded":"2024-04-02T17:10:41.072018+0000","last_fullsized":"2024-04-02T17:10:41.072018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770736+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:10:36.770736+0000","last_peered":"2024-04-02T17:10:36.770736+0000","last_clean":"2024-04-02T17:10:36.770736+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:10:36.770736+0000","last_undegraded":"2024-04-02T17:10:36.770736+0000","last_fullsized":"2024-04-02T17:10:36.770736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.071087+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:10:41.071087+0000","last_peered":"2024-04-02T17:10:41.071087+0000","last_clean":"2024-04-02T17:10:41.071087+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:10:41.071087+0000","last_undegraded":"2024-04-02T17:10:41.071087+0000","last_fullsized":"2024-04-02T17:10:41.071087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.768090+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:10:36.768090+0000","last_peered":"2024-04-02T17:10:36.768090+0000","last_clean":"2024-04-02T17:10:36.768090+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:10:36.768090+0000","last_undegraded":"2024-04-02T17:10:36.768090+0000","last_fullsized":"2024-04-02T17:10:36.768090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.045937+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:10:39.045937+0000","last_peered":"2024-04-02T17:10:39.045937+0000","last_clean":"2024-04-02T17:10:39.045937+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:10:39.045937+0000","last_undegraded":"2024-04-02T17:10:39.045937+0000","last_fullsized":"2024-04-02T17:10:39.045937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018798+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:10:36.018798+0000","last_peered":"2024-04-02T17:10:36.018798+0000","last_clean":"2024-04-02T17:10:36.018798+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:10:36.018798+0000","last_undegraded":"2024-04-02T17:10:36.018798+0000","last_fullsized":"2024-04-02T17:10:36.018798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.052209+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:10:39.052209+0000","last_peered":"2024-04-02T17:10:39.052209+0000","last_clean":"2024-04-02T17:10:39.052209+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:10:39.052209+0000","last_undegraded":"2024-04-02T17:10:39.052209+0000","last_fullsized":"2024-04-02T17:10:39.052209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.017587+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:10:36.017587+0000","last_peered":"2024-04-02T17:10:36.017587+0000","last_clean":"2024-04-02T17:10:36.017587+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:10:36.017587+0000","last_undegraded":"2024-04-02T17:10:36.017587+0000","last_fullsized":"2024-04-02T17:10:36.017587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.094229+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:10:42.094229+0000","last_peered":"2024-04-02T17:10:42.094229+0000","last_clean":"2024-04-02T17:10:42.094229+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:10:42.094229+0000","last_undegraded":"2024-04-02T17:10:42.094229+0000","last_fullsized":"2024-04-02T17:10:42.094229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377786+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:10:36.377786+0000","last_peered":"2024-04-02T17:10:36.377786+0000","last_clean":"2024-04-02T17:10:36.377786+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:10:36.377786+0000","last_undegraded":"2024-04-02T17:10:36.377786+0000","last_fullsized":"2024-04-02T17:10:36.377786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:41.070886+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:10:41.070886+0000","last_peered":"2024-04-02T17:10:41.070886+0000","last_clean":"2024-04-02T17:10:41.070886+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:10:41.070886+0000","last_undegraded":"2024-04-02T17:10:41.070886+0000","last_fullsized":"2024-04-02T17:10:41.070886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767903+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:10:36.767903+0000","last_peered":"2024-04-02T17:10:36.767903+0000","last_clean":"2024-04-02T17:10:36.767903+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:10:36.767903+0000","last_undegraded":"2024-04-02T17:10:36.767903+0000","last_fullsized":"2024-04-02T17:10:36.767903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:39.045295+0000","last_change":"2024-04-02T17:10:34.023829+0000","last_active":"2024-04-02T17:10:39.045295+0000","last_peered":"2024-04-02T17:10:39.045295+0000","last_clean":"2024-04-02T17:10:39.045295+0000","last_became_active":"2024-04-02T17:10:34.023472+0000","last_became_peered":"2024-04-02T17:10:34.023472+0000","last_unstale":"2024-04-02T17:10:39.045295+0000","last_undegraded":"2024-04-02T17:10:39.045295+0000","last_fullsized":"2024-04-02T17:10:39.045295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:11.822608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018352+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:10:36.018352+0000","last_peered":"2024-04-02T17:10:36.018352+0000","last_clean":"2024-04-02T17:10:36.018352+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:10:36.018352+0000","last_undegraded":"2024-04-02T17:10:36.018352+0000","last_fullsized":"2024-04-02T17:10:36.018352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:44.096844+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:10:44.096844+0000","last_peered":"2024-04-02T17:10:44.096844+0000","last_clean":"2024-04-02T17:10:44.096844+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:10:44.096844+0000","last_undegraded":"2024-04-02T17:10:44.096844+0000","last_fullsized":"2024-04-02T17:10:44.096844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018364+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:10:36.018364+0000","last_peered":"2024-04-02T17:10:36.018364+0000","last_clean":"2024-04-02T17:10:36.018364+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:10:36.018364+0000","last_undegraded":"2024-04-02T17:10:36.018364+0000","last_fullsized":"2024-04-02T17:10:36.018364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.092046+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:10:42.092046+0000","last_peered":"2024-04-02T17:10:42.092046+0000","last_clean":"2024-04-02T17:10:42.092046+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:10:42.092046+0000","last_undegraded":"2024-04-02T17:10:42.092046+0000","last_fullsized":"2024-04-02T17:10:42.092046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018262+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:10:36.018262+0000","last_peered":"2024-04-02T17:10:36.018262+0000","last_clean":"2024-04-02T17:10:36.018262+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:10:36.018262+0000","last_undegraded":"2024-04-02T17:10:36.018262+0000","last_fullsized":"2024-04-02T17:10:36.018262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:42.095201+0000","last_change":"2024-04-02T17:10:34.044886+0000","last_active":"2024-04-02T17:10:42.095201+0000","last_peered":"2024-04-02T17:10:42.095201+0000","last_clean":"2024-04-02T17:10:42.095201+0000","last_became_active":"2024-04-02T17:10:34.043574+0000","last_became_peered":"2024-04-02T17:10:34.043574+0000","last_unstale":"2024-04-02T17:10:42.095201+0000","last_undegraded":"2024-04-02T17:10:42.095201+0000","last_fullsized":"2024-04-02T17:10:42.095201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:58:45.361118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018413+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:10:36.018413+0000","last_peered":"2024-04-02T17:10:36.018413+0000","last_clean":"2024-04-02T17:10:36.018413+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:10:36.018413+0000","last_undegraded":"2024-04-02T17:10:36.018413+0000","last_fullsized":"2024-04-02T17:10:36.018413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377993+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:10:36.377993+0000","last_peered":"2024-04-02T17:10:36.377993+0000","last_clean":"2024-04-02T17:10:36.377993+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:10:36.377993+0000","last_undegraded":"2024-04-02T17:10:36.377993+0000","last_fullsized":"2024-04-02T17:10:36.377993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765100+0000","last_change":"2024-04-02T17:10:35.035008+0000","last_active":"2024-04-02T17:10:36.765100+0000","last_peered":"2024-04-02T17:10:36.765100+0000","last_clean":"2024-04-02T17:10:36.765100+0000","last_became_active":"2024-04-02T17:10:35.034692+0000","last_became_peered":"2024-04-02T17:10:35.034692+0000","last_unstale":"2024-04-02T17:10:36.765100+0000","last_undegraded":"2024-04-02T17:10:36.765100+0000","last_fullsized":"2024-04-02T17:10:36.765100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:18:58.457401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765274+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:10:36.765274+0000","last_peered":"2024-04-02T17:10:36.765274+0000","last_clean":"2024-04-02T17:10:36.765274+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:10:36.765274+0000","last_undegraded":"2024-04-02T17:10:36.765274+0000","last_fullsized":"2024-04-02T17:10:36.765274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.018803+0000","last_change":"2024-04-02T17:10:35.049055+0000","last_active":"2024-04-02T17:10:36.018803+0000","last_peered":"2024-04-02T17:10:36.018803+0000","last_clean":"2024-04-02T17:10:36.018803+0000","last_became_active":"2024-04-02T17:10:35.048454+0000","last_became_peered":"2024-04-02T17:10:35.048454+0000","last_unstale":"2024-04-02T17:10:36.018803+0000","last_undegraded":"2024-04-02T17:10:36.018803+0000","last_fullsized":"2024-04-02T17:10:36.018803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:36:14.686282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770688+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:10:36.770688+0000","last_peered":"2024-04-02T17:10:36.770688+0000","last_clean":"2024-04-02T17:10:36.770688+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:10:36.770688+0000","last_undegraded":"2024-04-02T17:10:36.770688+0000","last_fullsized":"2024-04-02T17:10:36.770688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377945+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:10:36.377945+0000","last_peered":"2024-04-02T17:10:36.377945+0000","last_clean":"2024-04-02T17:10:36.377945+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:10:36.377945+0000","last_undegraded":"2024-04-02T17:10:36.377945+0000","last_fullsized":"2024-04-02T17:10:36.377945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765725+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:10:36.765725+0000","last_peered":"2024-04-02T17:10:36.765725+0000","last_clean":"2024-04-02T17:10:36.765725+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:10:36.765725+0000","last_undegraded":"2024-04-02T17:10:36.765725+0000","last_fullsized":"2024-04-02T17:10:36.765725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767839+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:10:36.767839+0000","last_peered":"2024-04-02T17:10:36.767839+0000","last_clean":"2024-04-02T17:10:36.767839+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:10:36.767839+0000","last_undegraded":"2024-04-02T17:10:36.767839+0000","last_fullsized":"2024-04-02T17:10:36.767839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.765651+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:10:36.765651+0000","last_peered":"2024-04-02T17:10:36.765651+0000","last_clean":"2024-04-02T17:10:36.765651+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:10:36.765651+0000","last_undegraded":"2024-04-02T17:10:36.765651+0000","last_fullsized":"2024-04-02T17:10:36.765651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.767086+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:10:36.767086+0000","last_peered":"2024-04-02T17:10:36.767086+0000","last_clean":"2024-04-02T17:10:36.767086+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:10:36.767086+0000","last_undegraded":"2024-04-02T17:10:36.767086+0000","last_fullsized":"2024-04-02T17:10:36.767086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.264436+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:10:36.264436+0000","last_peered":"2024-04-02T17:10:36.264436+0000","last_clean":"2024-04-02T17:10:36.264436+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:10:36.264436+0000","last_undegraded":"2024-04-02T17:10:36.264436+0000","last_fullsized":"2024-04-02T17:10:36.264436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.770354+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:10:36.770354+0000","last_peered":"2024-04-02T17:10:36.770354+0000","last_clean":"2024-04-02T17:10:36.770354+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:10:36.770354+0000","last_undegraded":"2024-04-02T17:10:36.770354+0000","last_fullsized":"2024-04-02T17:10:36.770354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-02T17:10:36.377850+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:10:36.377850+0000","last_peered":"2024-04-02T17:10:36.377850+0000","last_clean":"2024-04-02T17:10:36.377850+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:10:36.377850+0000","last_undegraded":"2024-04-02T17:10:36.377850+0000","last_fullsized":"2024-04-02T17:10:36.377850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":28065,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":209,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":87342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":450862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579926,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1593344,"data_stored":1050377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":450578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1042893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":450509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":451282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":452239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":492052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":450334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":450336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1043366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":23022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:10:46.305 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T17:10:46.306 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T17:10:46.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T17:10:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:46 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4106295627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:46.428 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:46 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4106295627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:46 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4106295627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:10:47.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:47 smithi059 ceph-mon[31664]: pgmap v315: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-02T17:10:47.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:47 smithi059 ceph-mon[31664]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:47.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:47 smithi059 ceph-mon[31664]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:47.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:47 smithi154 ceph-mon[36479]: pgmap v315: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-02T17:10:47.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:47 smithi154 ceph-mon[36479]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:47.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:47 smithi154 ceph-mon[36479]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:47 smithi164 ceph-mon[36499]: pgmap v315: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-02T17:10:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:47 smithi164 ceph-mon[36499]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:47 smithi164 ceph-mon[36499]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:10:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 -- 172.21.15.59:0/2090623962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558108330 msgr2=0x7fc55810a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 --2- 172.21.15.59:0/2090623962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558108330 0x7fc55810a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc55000b0a0 tx=0x7fc55000b3b0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 -- 172.21.15.59:0/2090623962 shutdown_connections 2024-04-02T17:10:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 --2- 172.21.15.59:0/2090623962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558108330 0x7fc55810a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 --2- 172.21.15.59:0/2090623962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5581030d0 0x7fc5581054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 --2- 172.21.15.59:0/2090623962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc558105a00 0x7fc558107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 -- 172.21.15.59:0/2090623962 >> 172.21.15.59:0/2090623962 conn(0x7fc5580fcb00 msgr2=0x7fc5580fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 -- 172.21.15.59:0/2090623962 shutdown_connections 2024-04-02T17:10:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.953+0000 7fc55f800700 1 -- 172.21.15.59:0/2090623962 wait complete. 2024-04-02T17:10:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 Processor -- start 2024-04-02T17:10:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 -- start start 2024-04-02T17:10:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 0x7fc55819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 0x7fc55819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc558108330 0x7fc5581a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc55810e210 con 0x7fc5581030d0 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc55810e090 con 0x7fc558105a00 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.955+0000 7fc55f800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc55810e390 con 0x7fc558108330 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc55ce00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 0x7fc55819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc55ce00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 0x7fc55819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39436/0 (socket says 172.21.15.59:39436) 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 0x7fc55819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc55ce00700 1 -- 172.21.15.59:0/2327890225 learned_addr learned my addr 172.21.15.59:0/2327890225 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc55d800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc558108330 0x7fc5581a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 -- 172.21.15.59:0/2327890225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc558108330 msgr2=0x7fc5581a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc558108330 0x7fc5581a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 -- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 msgr2=0x7fc55819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 0x7fc55819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 -- 172.21.15.59:0/2327890225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5581a2560 con 0x7fc558105a00 2024-04-02T17:10:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.956+0000 7fc557e00700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 0x7fc55819ae40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d750 tx=0x7fc54400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.957+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544014030 con 0x7fc558105a00 2024-04-02T17:10:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.957+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5581a2850 con 0x7fc558105a00 2024-04-02T17:10:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.957+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc544004500 con 0x7fc558105a00 2024-04-02T17:10:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.957+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544005480 con 0x7fc558105a00 2024-04-02T17:10:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.957+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5581a2d90 con 0x7fc558105a00 2024-04-02T17:10:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.958+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc558050b00 con 0x7fc558105a00 2024-04-02T17:10:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.960+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc544004750 con 0x7fc558105a00 2024-04-02T17:10:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.962+0000 7fc555600700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 0x7fc5400a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.962+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc5440d0570 con 0x7fc558105a00 2024-04-02T17:10:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.962+0000 7fc55ce00700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 0x7fc5400a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.963+0000 7fc55ce00700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 0x7fc5400a6240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009d60 tx=0x7fc54c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:48.964+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc54409cea0 con 0x7fc558105a00 2024-04-02T17:10:49.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.290+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc55802ddd0 con 0x7fc558105a00 2024-04-02T17:10:49.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.300+0000 7fc555600700 1 -- 172.21.15.59:0/2327890225 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc54409c7a0 con 0x7fc558105a00 2024-04-02T17:10:49.300 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-02T17:10:49.300 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAZPAxmQmJxERAAZpS5qXB/Qgs7zLwfV6Iuww== 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 msgr2=0x7fc5400a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 0x7fc5400a6240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009d60 tx=0x7fc54c009260 comp rx=0 tx=0).stop 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 msgr2=0x7fc55819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 0x7fc55819ae40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d750 tx=0x7fc54400da60 comp rx=0 tx=0).stop 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 shutdown_connections 2024-04-02T17:10:49.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc558108330 0x7fc5581a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5400a3d80 0x7fc5400a6240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5581030d0 0x7fc55819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 --2- 172.21.15.59:0/2327890225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc558105a00 0x7fc55819ae40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 >> 172.21.15.59:0/2327890225 conn(0x7fc5580fcb00 msgr2=0x7fc5581026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.303+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 shutdown_connections 2024-04-02T17:10:49.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:49.304+0000 7fc55f800700 1 -- 172.21.15.59:0/2327890225 wait complete. 2024-04-02T17:10:49.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:49 smithi154 ceph-mon[36479]: pgmap v316: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-02T17:10:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:49 smithi059 ceph-mon[31664]: pgmap v316: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-02T17:10:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:49 smithi164 ceph-mon[36499]: pgmap v316: 97 pgs: 97 active+clean; 604 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-02T17:10:50.057 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:10:50.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T17:10:50.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T17:10:50.203 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T17:10:50.204 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T17:10:50.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T17:10:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2327890225' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-02T17:10:50.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:50 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2327890225' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]: dispatch 2024-04-02T17:10:50.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-02T17:10:50.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:50 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-02T17:10:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2327890225' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-02T17:10:50.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:50 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-02T17:10:50.715 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: pgmap v317: 97 pgs: 97 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-02T17:10:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-02T17:10:51.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:51 smithi154 ceph-mon[36479]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T17:10:51.777 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: pgmap v317: 97 pgs: 97 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-04-02T17:10:51.777 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-02T17:10:51.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:51 smithi059 ceph-mon[31664]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T17:10:51.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: pgmap v317: 97 pgs: 97 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8, 5, 9]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-02T17:10:51.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:51 smithi164 ceph-mon[36499]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T17:10:52.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:52 smithi059 ceph-mon[31664]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T17:10:52.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:52 smithi154 ceph-mon[36479]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T17:10:52.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:52 smithi164 ceph-mon[36499]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T17:10:52.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.936+0000 7fece4400700 1 -- 172.21.15.59:0/3959821590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 msgr2=0x7fecdc104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:52.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.936+0000 7fece4400700 1 --2- 172.21.15.59:0/3959821590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc104c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009990 tx=0x7fecc8009ca0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 -- 172.21.15.59:0/3959821590 shutdown_connections 2024-04-02T17:10:52.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 --2- 172.21.15.59:0/3959821590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 --2- 172.21.15.59:0/3959821590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 0x7fecdc10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 --2- 172.21.15.59:0/3959821590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc104c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 -- 172.21.15.59:0/3959821590 >> 172.21.15.59:0/3959821590 conn(0x7fecdc1002d0 msgr2=0x7fecdc1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 -- 172.21.15.59:0/3959821590 shutdown_connections 2024-04-02T17:10:52.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.937+0000 7fece4400700 1 -- 172.21.15.59:0/3959821590 wait complete. 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 Processor -- start 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 -- start start 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 0x7fecdc19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc114bc0 con 0x7fecdc1047e0 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc114a40 con 0x7fecdc10a7e0 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.938+0000 7fece4400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc114d40 con 0x7fecdc1051a0 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece2400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 0x7fecdc19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39076/0 (socket says 172.21.15.59:39076) 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 -- 172.21.15.59:0/2767942782 learned_addr learned my addr 172.21.15.59:0/2767942782 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1a00700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 -- 172.21.15.59:0/2767942782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 msgr2=0x7fecdc19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 0x7fecdc19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 -- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 msgr2=0x7fecdc19b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc19b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1000700 1 -- 172.21.15.59:0/2767942782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc1a0020 con 0x7fecdc1051a0 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.939+0000 7fece1a00700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fece1000700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc800bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8027070 con 0x7fecdc1051a0 2024-04-02T17:10:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc8003d70 con 0x7fecdc1051a0 2024-04-02T17:10:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc1a02b0 con 0x7fecdc1051a0 2024-04-02T17:10:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc800f460 con 0x7fecdc1051a0 2024-04-02T17:10:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.940+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc1a7b90 con 0x7fecdc1051a0 2024-04-02T17:10:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.941+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc105f20 con 0x7fecdc1051a0 2024-04-02T17:10:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.943+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fecc802a070 con 0x7fecdc1051a0 2024-04-02T17:10:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.945+0000 7fecda000700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 0x7fecc40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.945+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fecc80d1c70 con 0x7fecdc1051a0 2024-04-02T17:10:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.945+0000 7fece1a00700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 0x7fecc40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.946+0000 7fece1a00700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 0x7fecc40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009bc0 tx=0x7fecd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:52.947+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fecc8031020 con 0x7fecdc1051a0 2024-04-02T17:10:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.244+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fecdc19c370 con 0x7fecdc1051a0 2024-04-02T17:10:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.251+0000 7fecda000700 1 -- 172.21.15.59:0/2767942782 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7fecc8013070 con 0x7fecdc1051a0 2024-04-02T17:10:53.256 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:53.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 msgr2=0x7fecc40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 0x7fecc40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009bc0 tx=0x7fecd0009260 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 msgr2=0x7fecdc19b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc800bb60 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 shutdown_connections 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fecdc10a7e0 0x7fecdc19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fecc40a3cd0 0x7fecc40a6190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecdc1047e0 0x7fecdc19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.258+0000 7fece4400700 1 --2- 172.21.15.59:0/2767942782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1051a0 0x7fecdc19b550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.259+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 >> 172.21.15.59:0/2767942782 conn(0x7fecdc1002d0 msgr2=0x7fecdc100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:53.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.259+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 shutdown_connections 2024-04-02T17:10:53.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:53.259+0000 7fece4400700 1 -- 172.21.15.59:0/2767942782 wait complete. 2024-04-02T17:10:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:53 smithi059 ceph-mon[31664]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 KiB/s wr, 18 op/s 2024-04-02T17:10:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:53 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2767942782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T17:10:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:53 smithi154 ceph-mon[36479]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 KiB/s wr, 18 op/s 2024-04-02T17:10:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:53 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2767942782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T17:10:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:53 smithi164 ceph-mon[36499]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 627 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 KiB/s wr, 18 op/s 2024-04-02T17:10:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:53 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2767942782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T17:10:54.074 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1634390325},{"type":"v1","addr":"172.21.15.59:6801","nonce":1634390325}]},"active_addr":"172.21.15.59:6801/1634390325","active_change":"2024-04-02T17:02:49.350890+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.59:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2942673111}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1842613387}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":863400844}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3076784263}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3841145229}]}]} 2024-04-02T17:10:54.079 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T17:10:54.079 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T17:10:54.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T17:10:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:10:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:54 smithi154 ceph-mon[36479]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:54.737 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:54 smithi059 ceph-mon[31664]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:54.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:54 smithi164 ceph-mon[36499]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:56.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 -- 172.21.15.59:0/3726752557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406b680 msgr2=0x7f06e410f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:56.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 --2- 172.21.15.59:0/3726752557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406b680 0x7f06e410f440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00b0d0 tx=0x7f06dc00b3e0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 -- 172.21.15.59:0/3726752557 shutdown_connections 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 --2- 172.21.15.59:0/3726752557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406b680 0x7f06e410f440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 --2- 172.21.15.59:0/3726752557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e4108620 0x7f06e4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 --2- 172.21.15.59:0/3726752557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406acc0 0x7f06e406b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.595+0000 7f06eac00700 1 -- 172.21.15.59:0/3726752557 >> 172.21.15.59:0/3726752557 conn(0x7f06e4078460 msgr2=0x7f06e4078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.596+0000 7f06eac00700 1 -- 172.21.15.59:0/3726752557 shutdown_connections 2024-04-02T17:10:56.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.596+0000 7f06eac00700 1 -- 172.21.15.59:0/3726752557 wait complete. 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 Processor -- start 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 -- start start 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 0x7f06e419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406b680 0x7f06e419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06e41128f0 con 0x7f06e4108620 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4112770 con 0x7f06e406acc0 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.597+0000 7f06eac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4112a70 con 0x7f06e406b680 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e8c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e8c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41534/0 (socket says 172.21.15.59:41534) 2024-04-02T17:10:56.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e8c00700 1 -- 172.21.15.59:0/3330162262 learned_addr learned my addr 172.21.15.59:0/3330162262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 0x7f06e419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3400700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406b680 0x7f06e419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 -- 172.21.15.59:0/3330162262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406b680 msgr2=0x7f06e419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406b680 0x7f06e419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 -- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 msgr2=0x7f06e419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e3e00700 1 -- 172.21.15.59:0/3330162262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e419ff40 con 0x7f06e406acc0 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.598+0000 7f06e8c00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:10:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06e3e00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 0x7f06e419af30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f06d400e9f0 tx=0x7f06d400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d400c9c0 con 0x7f06e406acc0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d400cb60 con 0x7f06e406acc0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4018680 con 0x7f06e406acc0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e41a0230 con 0x7f06e406acc0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.599+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41a7ab0 con 0x7f06e406acc0 2024-04-02T17:10:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.600+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e4103d10 con 0x7f06e406acc0 2024-04-02T17:10:56.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.602+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f06d4010430 con 0x7f06e406acc0 2024-04-02T17:10:56.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.603+0000 7f06e0c00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 0x7f06c80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:10:56.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.603+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f06d40d0610 con 0x7f06e406acc0 2024-04-02T17:10:56.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.604+0000 7f06e3400700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 0x7f06c80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:10:56.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.604+0000 7f06e3400700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 0x7f06c80a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e4103560 tx=0x7f06d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:10:56.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.608+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f06d409ce20 con 0x7f06e406acc0 2024-04-02T17:10:56.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:56 smithi059 ceph-mon[31664]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:56.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:56 smithi164 ceph-mon[36499]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.840+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06e41a7da0 con 0x7f06e406acc0 2024-04-02T17:10:56.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.842+0000 7f06e0c00700 1 -- 172.21.15.59:0/3330162262 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22717 (secure 0 0 0) 0x7f06d409c720 con 0x7f06e406acc0 2024-04-02T17:10:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:10:56.843 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:10:51.305063+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:10:56.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.845+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 msgr2=0x7f06c80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:56.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.845+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 0x7f06c80a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e4103560 tx=0x7f06d0009260 comp rx=0 tx=0).stop 2024-04-02T17:10:56.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.845+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 msgr2=0x7f06e419af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.845+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 0x7f06e419af30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f06d400e9f0 tx=0x7f06d400ed00 comp rx=0 tx=0).stop 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 shutdown_connections 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f06c80a3d50 0x7f06c80a6210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4108620 0x7f06e419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06e406acc0 0x7f06e419af30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 --2- 172.21.15.59:0/3330162262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06e406b680 0x7f06e419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 >> 172.21.15.59:0/3330162262 conn(0x7f06e4078460 msgr2=0x7f06e410d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 shutdown_connections 2024-04-02T17:10:56.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:10:56.846+0000 7f06eac00700 1 -- 172.21.15.59:0/3330162262 wait complete. 2024-04-02T17:10:56.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:56 smithi154 ceph-mon[36479]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 13 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:57.691 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T17:10:57.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T17:10:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:57 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3330162262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:57.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:57 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3330162262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:58.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:57 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3330162262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:10:58.179 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:10:58.606 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:10:58 smithi059 ceph-mon[31664]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:58.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:10:58 smithi164 ceph-mon[36499]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:10:58.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:10:58 smithi154 ceph-mon[36479]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.286+0000 7f1165200700 1 -- 172.21.15.59:0/4124782314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 msgr2=0x7f1160108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.286+0000 7f1165200700 1 --2- 172.21.15.59:0/4124782314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f1160108a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1150009990 tx=0x7f1150009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 -- 172.21.15.59:0/4124782314 shutdown_connections 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 --2- 172.21.15.59:0/4124782314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006ac60 0x7f116010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 --2- 172.21.15.59:0/4124782314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f1160108a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 --2- 172.21.15.59:0/4124782314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006a2a0 0x7f116006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 -- 172.21.15.59:0/4124782314 >> 172.21.15.59:0/4124782314 conn(0x7f1160078420 msgr2=0x7f1160078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 -- 172.21.15.59:0/4124782314 shutdown_connections 2024-04-02T17:11:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.287+0000 7f1165200700 1 -- 172.21.15.59:0/4124782314 wait complete. 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.288+0000 7f1165200700 1 Processor -- start 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.288+0000 7f1165200700 1 -- start start 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 0x7f116019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f116019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11601128f0 con 0x7f1160108650 2024-04-02T17:11:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1160112770 con 0x7f116006a2a0 2024-04-02T17:11:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f1165200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1160112a70 con 0x7f116006ac60 2024-04-02T17:11:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f116019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115de00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 0x7f116019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39116/0 (socket says 172.21.15.59:39116) 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115de00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 0x7f116019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48702/0 (socket says 172.21.15.59:48702) 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115d400700 1 -- 172.21.15.59:0/2335450212 learned_addr learned my addr 172.21.15.59:0/2335450212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.289+0000 7f115e800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f116019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41540/0 (socket says 172.21.15.59:41540) 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 -- 172.21.15.59:0/2335450212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 msgr2=0x7f116019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 0x7f116019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 -- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 msgr2=0x7f116019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f116019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 -- 172.21.15.59:0/2335450212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116019ff60 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f115d400700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e9a0 tx=0x7f114c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00ca10 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a0250 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a7ad0 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c00cbb0 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.290+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c018680 con 0x7f116006ac60 2024-04-02T17:11:00.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.292+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1140005330 con 0x7f116006ac60 2024-04-02T17:11:00.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.294+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f114c010430 con 0x7f116006ac60 2024-04-02T17:11:00.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.295+0000 7f1156a00700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 0x7f11440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:00.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.295+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f114c0d0630 con 0x7f116006ac60 2024-04-02T17:11:00.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.295+0000 7f115de00700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 0x7f11440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:00.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.296+0000 7f115de00700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 0x7f11440a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f115000b2e0 tx=0x7f1150005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:00.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.299+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f114c09ce40 con 0x7f116006ac60 2024-04-02T17:11:00.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.530+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11400058f0 con 0x7f116006ac60 2024-04-02T17:11:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.532+0000 7f1156a00700 1 -- 172.21.15.59:0/2335450212 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22717 (secure 0 0 0) 0x7f114c09c740 con 0x7f116006ac60 2024-04-02T17:11:00.532 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:11:00.532 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:10:51.305063+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:11:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.535+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 msgr2=0x7f11440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.535+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 0x7f11440a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f115000b2e0 tx=0x7f1150005960 comp rx=0 tx=0).stop 2024-04-02T17:11:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.535+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 msgr2=0x7f116019b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.535+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e9a0 tx=0x7f114c00ecb0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.535+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 shutdown_connections 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f11440a3d50 0x7f11440a6210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1160108650 0x7f116019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f116006a2a0 0x7f116019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 --2- 172.21.15.59:0/2335450212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f116006ac60 0x7f116019b4e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 >> 172.21.15.59:0/2335450212 conn(0x7f1160078420 msgr2=0x7f11601000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 shutdown_connections 2024-04-02T17:11:00.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:00.536+0000 7f1165200700 1 -- 172.21.15.59:0/2335450212 wait complete. 2024-04-02T17:11:00.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:00 smithi059 ceph-mon[31664]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:11:00.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:00 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2335450212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:11:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:00 smithi164 ceph-mon[36499]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:11:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:00 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2335450212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:11:00.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:00 smithi154 ceph-mon[36479]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-04-02T17:11:00.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:00 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2335450212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:11:01.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T17:11:01.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T17:11:01.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T17:11:01.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T17:11:01.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T17:11:01.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T17:11:01.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-02T17:11:01.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-02T17:11:01.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-04-02T17:11:01.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-04-02T17:11:01.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-04-02T17:11:01.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-04-02T17:11:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:02.266 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:02.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:02 smithi154 ceph-mon[36479]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 8 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:02.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:02 smithi059 ceph-mon[31664]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 8 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:02.860 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:02.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:02 smithi164 ceph-mon[36499]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 8 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:04.632 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:04.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:04 smithi059 ceph-mon[31664]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 7 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:04.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:04 smithi154 ceph-mon[36479]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 7 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:05.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:04 smithi164 ceph-mon[36499]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 7 B/s, 0 keys/s, 1 objects/s recovering 2024-04-02T17:11:05.371 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:05.372 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:05.374 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:05.374 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:05.383 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:06.673 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:06 smithi059 ceph-mon[31664]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T17:11:06.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:06 smithi154 ceph-mon[36479]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T17:11:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:06 smithi164 ceph-mon[36499]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T17:11:08.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 -- 172.21.15.59:0/4171176704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608102640 msgr2=0x7f0608102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 --2- 172.21.15.59:0/4171176704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608102640 0x7f0608102ac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009990 tx=0x7f05f4009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 -- 172.21.15.59:0/4171176704 shutdown_connections 2024-04-02T17:11:08.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 --2- 172.21.15.59:0/4171176704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0608103000 0x7f060810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 --2- 172.21.15.59:0/4171176704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608108630 0x7f0608108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 --2- 172.21.15.59:0/4171176704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608102640 0x7f0608102ac0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 -- 172.21.15.59:0/4171176704 >> 172.21.15.59:0/4171176704 conn(0x7f0608079750 msgr2=0x7f0608079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 -- 172.21.15.59:0/4171176704 shutdown_connections 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.308+0000 7f060da00700 1 -- 172.21.15.59:0/4171176704 wait complete. 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.309+0000 7f060da00700 1 Processor -- start 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 -- start start 2024-04-02T17:11:08.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102640 0x7f060819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0608103000 0x7f060819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 0x7f060819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0608112a10 con 0x7f0608108630 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0608112890 con 0x7f0608103000 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f060da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0608112b90 con 0x7f0608102640 2024-04-02T17:11:08.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f0607400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 0x7f060819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f0606000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0608103000 0x7f060819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f0607400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 0x7f060819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36394/0 (socket says 172.21.15.59:36394) 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.310+0000 7f0607400700 1 -- 172.21.15.59:0/2913209624 learned_addr learned my addr 172.21.15.59:0/2913209624 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0606a00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102640 0x7f060819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 -- 172.21.15.59:0/2913209624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102640 msgr2=0x7f060819b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102640 0x7f060819b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 -- 172.21.15.59:0/2913209624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0608103000 msgr2=0x7f060819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0608103000 0x7f060819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06081a00e0 con 0x7f0608108630 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f0607400700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 0x7f060819f9a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f060000efc0 tx=0x7f060000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.311+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06000092e0 con 0x7f0608108630 2024-04-02T17:11:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.312+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0600004bb0 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.312+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600004280 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.312+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06081a0370 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.312+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06081a7c50 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.313+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0608100e90 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.315+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f060001b030 con 0x7f0608108630 2024-04-02T17:11:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.316+0000 7f05ff400700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05ec0a3d50 0x7f05ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:08.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.316+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f06000d0910 con 0x7f0608108630 2024-04-02T17:11:08.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f0606a00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05ec0a3d50 0x7f05ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f05ff400700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] conn(0x7f05ec0ad910 0x7f05ec0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05ec0b0480 con 0x7f05ec0ad910 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f06000d0d00 con 0x7f0608108630 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f0606000700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] conn(0x7f05ec0ad910 0x7f05ec0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.317+0000 7f0606a00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05ec0a3d50 0x7f05ec0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009bd0 tx=0x7f05f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.318+0000 7f0606000700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] conn(0x7f05ec0ad910 0x7f05ec0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.319+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== osd.3 v2:172.21.15.59:6810/3330665578 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05ec0b0480 con 0x7f05ec0ad910 2024-04-02T17:11:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.337+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 --> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0608103d70 con 0x7f05ec0ad910 2024-04-02T17:11:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.338+0000 7f05ff400700 1 -- 172.21.15.59:0/2913209624 <== osd.3 v2:172.21.15.59:6810/3330665578 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0608103d70 con 0x7f05ec0ad910 2024-04-02T17:11:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.338+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] conn(0x7f05ec0ad910 msgr2=0x7f05ec0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.338+0000 7f060da00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] conn(0x7f05ec0ad910 0x7f05ec0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05ec0a3d50 msgr2=0x7f05ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05ec0a3d50 0x7f05ec0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009bd0 tx=0x7f05f8009260 comp rx=0 tx=0).stop 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 msgr2=0x7f060819f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 --2- 172.21.15.59:0/2913209624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0608108630 0x7f060819f9a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f060000efc0 tx=0x7f060000c3a0 comp rx=0 tx=0).stop 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f0607400700 1 -- 172.21.15.59:0/2913209624 reap_dead start 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 shutdown_connections 2024-04-02T17:11:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 >> 172.21.15.59:0/2913209624 conn(0x7f0608079750 msgr2=0x7f06081004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 shutdown_connections 2024-04-02T17:11:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:06.339+0000 7f060da00700 1 -- 172.21.15.59:0/2913209624 wait complete. 2024-04-02T17:11:08.674 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:08 smithi059 ceph-mon[31664]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:08.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:08 smithi154 ceph-mon[36479]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:09.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:08 smithi164 ceph-mon[36499]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:10.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:10 smithi059 ceph-mon[31664]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T17:11:10.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:10 smithi154 ceph-mon[36479]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T17:11:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:10 smithi164 ceph-mon[36499]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T17:11:11.308 INFO:teuthology.orchestra.run.smithi059.stdout:107374182463 2024-04-02T17:11:11.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T17:11:12.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:12 smithi059 ceph-mon[31664]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:12.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:12 smithi154 ceph-mon[36479]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:13.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:12 smithi164 ceph-mon[36499]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T17:11:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:14 smithi154 ceph-mon[36479]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:11:14.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 -- 172.21.15.59:0/2382089247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc1085a0 msgr2=0x7f18cc108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 --2- 172.21.15.59:0/2382089247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc1085a0 0x7f18cc108980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18c00099c0 tx=0x7f18c0009cd0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 -- 172.21.15.59:0/2382089247 shutdown_connections 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 --2- 172.21.15.59:0/2382089247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18cc102eb0 0x7f18cc10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 --2- 172.21.15.59:0/2382089247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc1085a0 0x7f18cc108980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 --2- 172.21.15.59:0/2382089247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 0x7f18cc102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.174+0000 7f18d3000700 1 -- 172.21.15.59:0/2382089247 >> 172.21.15.59:0/2382089247 conn(0x7f18cc0fdfc0 msgr2=0x7f18cc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.175+0000 7f18d3000700 1 -- 172.21.15.59:0/2382089247 shutdown_connections 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.175+0000 7f18d3000700 1 -- 172.21.15.59:0/2382089247 wait complete. 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 Processor -- start 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 -- start start 2024-04-02T17:11:14.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 0x7f18cc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 0x7f18cc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18cc1085a0 0x7f18cc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112830 con 0x7f18cc102eb0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1126b0 con 0x7f18cc1024f0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.176+0000 7f18d3000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1129b0 con 0x7f18cc1085a0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 0x7f18cc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 0x7f18cc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 0x7f18cc19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44640/0 (socket says 172.21.15.59:44640) 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 -- 172.21.15.59:0/704138270 learned_addr learned my addr 172.21.15.59:0/704138270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d2600700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18cc1085a0 0x7f18cc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 -- 172.21.15.59:0/704138270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18cc1085a0 msgr2=0x7f18cc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18cc1085a0 0x7f18cc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 -- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 msgr2=0x7f18cc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 0x7f18cc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc199190 con 0x7f18cc102eb0 2024-04-02T17:11:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1c00700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18cc1024f0 0x7f18cc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.177+0000 7f18d1200700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 0x7f18cc19edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d980 tx=0x7f18b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:14.948 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:14 smithi059 ceph-mon[31664]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.178+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8016070 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.178+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b8004d10 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.178+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8005670 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.178+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc199480 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.178+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a5740 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.179+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18cc0ffff0 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.181+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f18b8004750 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.182+0000 7f18ca000700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f18b40a3d50 0x7f18b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f18b80d0a80 con 0x7f18cc102eb0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18d1c00700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f18b40a3d50 0x7f18b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18ca000700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] conn(0x7f18b40ad910 0x7f18b40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18b8015ec0 con 0x7f18b40ad910 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f18b80d0e30 con 0x7f18cc102eb0 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18d2600700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] conn(0x7f18b40ad910 0x7f18b40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.183+0000 7f18d1c00700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f18b40a3d50 0x7f18b40a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000c00 tx=0x7f18c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.184+0000 7f18d2600700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] conn(0x7f18b40ad910 0x7f18b40afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.185+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== osd.4 v2:172.21.15.154:6808/2249512865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18b8015ec0 con 0x7f18b40ad910 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.203+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 --> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18cc103c10 con 0x7f18b40ad910 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.203+0000 7f18ca000700 1 -- 172.21.15.59:0/704138270 <== osd.4 v2:172.21.15.154:6808/2249512865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18cc103c10 con 0x7f18b40ad910 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] conn(0x7f18b40ad910 msgr2=0x7f18b40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] conn(0x7f18b40ad910 0x7f18b40afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f18b40a3d50 msgr2=0x7f18b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f18b40a3d50 0x7f18b40a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000c00 tx=0x7f18c0005fb0 comp rx=0 tx=0).stop 2024-04-02T17:11:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 msgr2=0x7f18cc19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 --2- 172.21.15.59:0/704138270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc102eb0 0x7f18cc19edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d980 tx=0x7f18b800dc90 comp rx=0 tx=0).stop 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d2600700 1 -- 172.21.15.59:0/704138270 reap_dead start 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 shutdown_connections 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 >> 172.21.15.59:0/704138270 conn(0x7f18cc0fdfc0 msgr2=0x7f18cc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 shutdown_connections 2024-04-02T17:11:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:13.204+0000 7f18d3000700 1 -- 172.21.15.59:0/704138270 wait complete. 2024-04-02T17:11:15.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:14 smithi164 ceph-mon[36499]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:11:16.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 -- 172.21.15.59:0/1161919309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2010a7e0 msgr2=0x7f3a2010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 --2- 172.21.15.59:0/1161919309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2010a7e0 0x7f3a2010abc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 -- 172.21.15.59:0/1161919309 shutdown_connections 2024-04-02T17:11:16.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 --2- 172.21.15.59:0/1161919309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a201051a0 0x7f3a201116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 --2- 172.21.15.59:0/1161919309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2010a7e0 0x7f3a2010abc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 --2- 172.21.15.59:0/1161919309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a201047e0 0x7f3a20104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 -- 172.21.15.59:0/1161919309 >> 172.21.15.59:0/1161919309 conn(0x7f3a201002d0 msgr2=0x7f3a201026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 -- 172.21.15.59:0/1161919309 shutdown_connections 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.040+0000 7f3a26400700 1 -- 172.21.15.59:0/1161919309 wait complete. 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.041+0000 7f3a26400700 1 Processor -- start 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.041+0000 7f3a26400700 1 -- start start 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.041+0000 7f3a26400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 0x7f3a2019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.041+0000 7f3a26400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a201051a0 0x7f3a2019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a26400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a2010a7e0 0x7f3a2019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a26400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20114b80 con 0x7f3a201047e0 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a26400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20114a00 con 0x7f3a201051a0 2024-04-02T17:11:16.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a26400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20114d00 con 0x7f3a2010a7e0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 0x7f3a2019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 0x7f3a2019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44684/0 (socket says 172.21.15.59:44684) 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 -- 172.21.15.59:0/1186137081 learned_addr learned my addr 172.21.15.59:0/1186137081 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1fe00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a2010a7e0 0x7f3a2019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1ea00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a201051a0 0x7f3a2019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 -- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a2010a7e0 msgr2=0x7f3a2019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a2010a7e0 0x7f3a2019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 -- 172.21.15.59:0/1186137081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a201051a0 msgr2=0x7f3a2019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a201051a0 0x7f3a2019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1f400700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019ff80 con 0x7f3a201047e0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.042+0000 7f3a1fe00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a2010a7e0 0x7f3a2019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a1f400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 0x7f3a2019af70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a10000c00 tx=0x7f3a10003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:16.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10027070 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a10003e40 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10022c90 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a201a0210 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.043+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201a7af0 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.044+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a20062d80 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.046+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a1002a070 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a17e00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a040a3cd0 0x7f3a040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f3a100d1e70 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a1ea00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a040a3cd0 0x7f3a040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a17e00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] conn(0x7f3a040ad890 0x7f3a040afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a10027e70 con 0x7f3a040ad890 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f3a100d2220 con 0x7f3a201047e0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.048+0000 7f3a1fe00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] conn(0x7f3a040ad890 0x7f3a040afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:16.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.049+0000 7f3a1ea00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a040a3cd0 0x7f3a040a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a20109da0 tx=0x7f3a0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.049+0000 7f3a1fe00700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] conn(0x7f3a040ad890 0x7f3a040afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.050+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== osd.5 v2:172.21.15.164:6808/3065900784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a10027e70 con 0x7f3a040ad890 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.068+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 --> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a20063200 con 0x7f3a040ad890 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.069+0000 7f3a17e00700 1 -- 172.21.15.59:0/1186137081 <== osd.5 v2:172.21.15.164:6808/3065900784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3a20063200 con 0x7f3a040ad890 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.069+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] conn(0x7f3a040ad890 msgr2=0x7f3a040afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.069+0000 7f3a26400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] conn(0x7f3a040ad890 0x7f3a040afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a040a3cd0 msgr2=0x7f3a040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3a040a3cd0 0x7f3a040a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a20109da0 tx=0x7f3a0c00b410 comp rx=0 tx=0).stop 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 msgr2=0x7f3a2019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 --2- 172.21.15.59:0/1186137081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a201047e0 0x7f3a2019af70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a10000c00 tx=0x7f3a10003890 comp rx=0 tx=0).stop 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a1fe00700 1 -- 172.21.15.59:0/1186137081 reap_dead start 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 shutdown_connections 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 >> 172.21.15.59:0/1186137081 conn(0x7f3a201002d0 msgr2=0x7f3a20100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 shutdown_connections 2024-04-02T17:11:16.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:16.070+0000 7f3a26400700 1 -- 172.21.15.59:0/1186137081 wait complete. 2024-04-02T17:11:16.478 INFO:teuthology.orchestra.run.smithi059.stdout:141733920829 2024-04-02T17:11:16.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T17:11:16.673 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:16 smithi059 ceph-mon[31664]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T17:11:16.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:16 smithi154 ceph-mon[36479]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T17:11:17.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:16 smithi164 ceph-mon[36499]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 -- 172.21.15.59:0/560298381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24102fe0 msgr2=0x7fda2410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 --2- 172.21.15.59:0/560298381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24102fe0 0x7fda2410f490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00b0a0 tx=0x7fda1c00b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 -- 172.21.15.59:0/560298381 shutdown_connections 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 --2- 172.21.15.59:0/560298381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24102fe0 0x7fda2410f490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 --2- 172.21.15.59:0/560298381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda24107520 0x7fda24107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 --2- 172.21.15.59:0/560298381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 0x7fda24102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 -- 172.21.15.59:0/560298381 >> 172.21.15.59:0/560298381 conn(0x7fda24079720 msgr2=0x7fda24079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 -- 172.21.15.59:0/560298381 shutdown_connections 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.526+0000 7fda2c600700 1 -- 172.21.15.59:0/560298381 wait complete. 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.527+0000 7fda2c600700 1 Processor -- start 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.527+0000 7fda2c600700 1 -- start start 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 0x7fda24198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda24102fe0 0x7fda24199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24107520 0x7fda2419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112900 con 0x7fda24107520 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112780 con 0x7fda24102fe0 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2c600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112a80 con 0x7fda24102640 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda29c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 0x7fda24198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24107520 0x7fda2419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda29c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 0x7fda24198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59690/0 (socket says 172.21.15.59:59690) 2024-04-02T17:11:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda29c00700 1 -- 172.21.15.59:0/1933621313 learned_addr learned my addr 172.21.15.59:0/1933621313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda29200700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda24102fe0 0x7fda24199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 -- 172.21.15.59:0/1933621313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 msgr2=0x7fda24198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda24102640 0x7fda24198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 -- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda24102fe0 msgr2=0x7fda24199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda24102fe0 0x7fda24199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.528+0000 7fda2a600700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2419dd20 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda2a600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24107520 0x7fda2419d5e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00b070 tx=0x7fda1c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c021070 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda1c007b80 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c005040 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2419dfb0 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.529+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda241a5890 con 0x7fda24107520 2024-04-02T17:11:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.530+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda24062d80 con 0x7fda24107520 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.532+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda1c024070 con 0x7fda24107520 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda22000700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fda0c0a3d50 0x7fda0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fda1c0d1340 con 0x7fda24107520 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda29c00700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fda0c0a3d50 0x7fda0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda22000700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] conn(0x7fda0c0ad910 0x7fda0c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda1c015ec0 con 0x7fda0c0ad910 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.534+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fda1c0d16f0 con 0x7fda24107520 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.535+0000 7fda29c00700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fda0c0a3d50 0x7fda0c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fda18005950 tx=0x7fda18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.535+0000 7fda29200700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] conn(0x7fda0c0ad910 0x7fda0c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.535+0000 7fda29200700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] conn(0x7fda0c0ad910 0x7fda0c0afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.536+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== osd.10 v2:172.21.15.154:6824/2125299053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda1c015ec0 con 0x7fda0c0ad910 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.555+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 --> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda24063200 con 0x7fda0c0ad910 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.555+0000 7fda22000700 1 -- 172.21.15.59:0/1933621313 <== osd.10 v2:172.21.15.154:6824/2125299053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda24063200 con 0x7fda0c0ad910 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] conn(0x7fda0c0ad910 msgr2=0x7fda0c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] conn(0x7fda0c0ad910 0x7fda0c0afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fda0c0a3d50 msgr2=0x7fda0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fda0c0a3d50 0x7fda0c0a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fda18005950 tx=0x7fda18009260 comp rx=0 tx=0).stop 2024-04-02T17:11:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24107520 msgr2=0x7fda2419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 --2- 172.21.15.59:0/1933621313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda24107520 0x7fda2419d5e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00b070 tx=0x7fda1c004c80 comp rx=0 tx=0).stop 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2a600700 1 -- 172.21.15.59:0/1933621313 reap_dead start 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 shutdown_connections 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.556+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 >> 172.21.15.59:0/1933621313 conn(0x7fda24079720 msgr2=0x7fda241006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.557+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 shutdown_connections 2024-04-02T17:11:17.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:14.557+0000 7fda2c600700 1 -- 172.21.15.59:0/1933621313 wait complete. 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 -- 172.21.15.59:0/4099333188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb800752e0 msgr2=0x7ffb8010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 --2- 172.21.15.59:0/4099333188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb800752e0 0x7ffb8010e970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009990 tx=0x7ffb68009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 -- 172.21.15.59:0/4099333188 shutdown_connections 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 --2- 172.21.15.59:0/4099333188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb8010eeb0 0x7ffb801112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 --2- 172.21.15.59:0/4099333188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb800749c0 0x7ffb80074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 --2- 172.21.15.59:0/4099333188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb800752e0 0x7ffb8010e970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 -- 172.21.15.59:0/4099333188 >> 172.21.15.59:0/4099333188 conn(0x7ffb800fe0c0 msgr2=0x7ffb801004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 -- 172.21.15.59:0/4099333188 shutdown_connections 2024-04-02T17:11:18.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.678+0000 7ffb85800700 1 -- 172.21.15.59:0/4099333188 wait complete. 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.679+0000 7ffb85800700 1 Processor -- start 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 -- start start 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb800749c0 0x7ffb8019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 0x7ffb8019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8010eeb0 0x7ffb8019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80114bb0 con 0x7ffb800752e0 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80114a30 con 0x7ffb8010eeb0 2024-04-02T17:11:18.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb85800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80114d30 con 0x7ffb800749c0 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb77e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 0x7ffb8019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb77e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 0x7ffb8019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44708/0 (socket says 172.21.15.59:44708) 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb77e00700 1 -- 172.21.15.59:0/4277517225 learned_addr learned my addr 172.21.15.59:0/4277517225 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb7f400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8010eeb0 0x7ffb8019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.680+0000 7ffb7ea00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb800749c0 0x7ffb8019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 -- 172.21.15.59:0/4277517225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb800749c0 msgr2=0x7ffb8019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb800749c0 0x7ffb8019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 -- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8010eeb0 msgr2=0x7ffb8019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8010eeb0 0x7ffb8019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801a0010 con 0x7ffb800752e0 2024-04-02T17:11:18.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb7f400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8010eeb0 0x7ffb8019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:18.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77e00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 0x7ffb8019b540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb6800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:18.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.681+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68027070 con 0x7ffb800752e0 2024-04-02T17:11:18.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.682+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801a02a0 con 0x7ffb800752e0 2024-04-02T17:11:18.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.682+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb680056d0 con 0x7ffb800752e0 2024-04-02T17:11:18.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.682+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68022e10 con 0x7ffb800752e0 2024-04-02T17:11:18.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.682+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb801a7b80 con 0x7ffb800752e0 2024-04-02T17:11:18.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.683+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb80062d80 con 0x7ffb800752e0 2024-04-02T17:11:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.686+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb6802a070 con 0x7ffb800752e0 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb77400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb640a3d50 0x7ffb640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7ffb680d1aa0 con 0x7ffb800752e0 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb7ea00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb640a3d50 0x7ffb640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb77400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] conn(0x7ffb640ad910 0x7ffb640afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb640b0480 con 0x7ffb640ad910 2024-04-02T17:11:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.688+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ffb680d1e50 con 0x7ffb800752e0 2024-04-02T17:11:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.689+0000 7ffb7f400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] conn(0x7ffb640ad910 0x7ffb640afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.689+0000 7ffb7ea00700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb640a3d50 0x7ffb640a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009c60 tx=0x7ffb70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.689+0000 7ffb7f400700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] conn(0x7ffb640ad910 0x7ffb640afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.690+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== osd.7 v2:172.21.15.154:6816/3912867880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb640b0480 con 0x7ffb640ad910 2024-04-02T17:11:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.709+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 --> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb80063180 con 0x7ffb640ad910 2024-04-02T17:11:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.709+0000 7ffb77400700 1 -- 172.21.15.59:0/4277517225 <== osd.7 v2:172.21.15.154:6816/3912867880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb80063180 con 0x7ffb640ad910 2024-04-02T17:11:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] conn(0x7ffb640ad910 msgr2=0x7ffb640afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] conn(0x7ffb640ad910 0x7ffb640afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb640a3d50 msgr2=0x7ffb640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb640a3d50 0x7ffb640a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009c60 tx=0x7ffb70009260 comp rx=0 tx=0).stop 2024-04-02T17:11:18.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 msgr2=0x7ffb8019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:18.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb85800700 1 --2- 172.21.15.59:0/4277517225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb800752e0 0x7ffb8019b540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb68000c00 tx=0x7ffb6800bab0 comp rx=0 tx=0).stop 2024-04-02T17:11:18.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.710+0000 7ffb7f400700 1 -- 172.21.15.59:0/4277517225 reap_dead start 2024-04-02T17:11:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.711+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 shutdown_connections 2024-04-02T17:11:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.711+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 >> 172.21.15.59:0/4277517225 conn(0x7ffb800fe0c0 msgr2=0x7ffb800fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.711+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 shutdown_connections 2024-04-02T17:11:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:18.711+0000 7ffb85800700 1 -- 172.21.15.59:0/4277517225 wait complete. 2024-04-02T17:11:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:18 smithi059 ceph-mon[31664]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:18.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:18 smithi154 ceph-mon[36479]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:18.996 INFO:teuthology.orchestra.run.smithi059.stdout:317827579933 2024-04-02T17:11:18.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-04-02T17:11:19.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:18 smithi164 ceph-mon[36499]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:19.990 INFO:teuthology.orchestra.run.smithi059.stdout:167503724600 2024-04-02T17:11:19.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T17:11:20.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:20 smithi059 ceph-mon[31664]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:20.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:20 smithi154 ceph-mon[36479]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:20 smithi164 ceph-mon[36499]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:21.411 INFO:teuthology.orchestra.run.smithi059.stdout:223338299436 2024-04-02T17:11:21.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 -- 172.21.15.59:0/1932590733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5884075140 msgr2=0x7f5884113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 --2- 172.21.15.59:0/1932590733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5884075140 0x7f5884113280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58700098f0 tx=0x7f5870009c00 comp rx=0 tx=0).stop 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 -- 172.21.15.59:0/1932590733 shutdown_connections 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 --2- 172.21.15.59:0/1932590733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58841137c0 0x7f5884115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 --2- 172.21.15.59:0/1932590733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5884074820 0x7f5884074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 --2- 172.21.15.59:0/1932590733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5884075140 0x7f5884113280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 -- 172.21.15.59:0/1932590733 >> 172.21.15.59:0/1932590733 conn(0x7f5884079750 msgr2=0x7f5884079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 -- 172.21.15.59:0/1932590733 shutdown_connections 2024-04-02T17:11:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.789+0000 7f5889200700 1 -- 172.21.15.59:0/1932590733 wait complete. 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 Processor -- start 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 -- start start 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5884074820 0x7f588419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5884075140 0x7f588419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 0x7f58841a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5884119000 con 0x7f5884074820 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5884118e80 con 0x7f58841137c0 2024-04-02T17:11:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.791+0000 7f5889200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5884119180 con 0x7f5884075140 2024-04-02T17:11:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5881400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5884075140 0x7f588419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 0x7f58841a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 0x7f58841a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53262/0 (socket says 172.21.15.59:53262) 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 -- 172.21.15.59:0/2728308029 learned_addr learned my addr 172.21.15.59:0/2728308029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5881e00700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5884074820 0x7f588419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 -- 172.21.15.59:0/2728308029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5884075140 msgr2=0x7f588419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5884075140 0x7f588419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 -- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5884074820 msgr2=0x7f588419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5884074820 0x7f588419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.792+0000 7f5882800700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58841a4500 con 0x7f58841137c0 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f5882800700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 0x7f58841a3dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f587c007f10 tx=0x7f587c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c00ddd0 con 0x7f58841137c0 2024-04-02T17:11:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58841a47f0 con 0x7f58841137c0 2024-04-02T17:11:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c004020 con 0x7f58841137c0 2024-04-02T17:11:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c015660 con 0x7f58841137c0 2024-04-02T17:11:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.793+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58841ac070 con 0x7f58841137c0 2024-04-02T17:11:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.795+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f588410fc40 con 0x7f58841137c0 2024-04-02T17:11:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.796+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f587c0041d0 con 0x7f58841137c0 2024-04-02T17:11:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.798+0000 7f587aa00700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58680a3cd0 0x7f58680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f587c0cff10 con 0x7f58841137c0 2024-04-02T17:11:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f5881e00700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58680a3cd0 0x7f58680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f587aa00700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] conn(0x7f58680ad890 0x7f58680afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58680b0400 con 0x7f58680ad890 2024-04-02T17:11:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f587c0d2530 con 0x7f58841137c0 2024-04-02T17:11:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.799+0000 7f5881400700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] conn(0x7f58680ad890 0x7f58680afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.800+0000 7f5881e00700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58680a3cd0 0x7f58680a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.800+0000 7f5881400700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] conn(0x7f58680ad890 0x7f58680afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.801+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== osd.6 v2:172.21.15.59:6818/2899017867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58680b0400 con 0x7f58680ad890 2024-04-02T17:11:21.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.824+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 --> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5884075f50 con 0x7f58680ad890 2024-04-02T17:11:21.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.824+0000 7f587aa00700 1 -- 172.21.15.59:0/2728308029 <== osd.6 v2:172.21.15.59:6818/2899017867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5884075f50 con 0x7f58680ad890 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] conn(0x7f58680ad890 msgr2=0x7f58680afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] conn(0x7f58680ad890 0x7f58680afd30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58680a3cd0 msgr2=0x7f58680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58680a3cd0 0x7f58680a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874009310 comp rx=0 tx=0).stop 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 msgr2=0x7f58841a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 --2- 172.21.15.59:0/2728308029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58841137c0 0x7f58841a3dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f587c007f10 tx=0x7f587c00d3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5882800700 1 -- 172.21.15.59:0/2728308029 reap_dead start 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 shutdown_connections 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 >> 172.21.15.59:0/2728308029 conn(0x7f5884079750 msgr2=0x7f588410f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 shutdown_connections 2024-04-02T17:11:21.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.825+0000 7f5889200700 1 -- 172.21.15.59:0/2728308029 wait complete. 2024-04-02T17:11:22.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:22 smithi154 ceph-mon[36479]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:22 smithi059 ceph-mon[31664]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.226+0000 7ff26fe00700 1 -- 172.21.15.59:0/4146920089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270101760 msgr2=0x7ff27010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.226+0000 7ff26fe00700 1 --2- 172.21.15.59:0/4146920089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270101760 0x7ff27010f3d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b0a0 tx=0x7ff26800b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 -- 172.21.15.59:0/4146920089 shutdown_connections 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 --2- 172.21.15.59:0/4146920089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270101760 0x7ff27010f3d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 --2- 172.21.15.59:0/4146920089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff270107530 0x7ff270107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 --2- 172.21.15.59:0/4146920089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff270101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 -- 172.21.15.59:0/4146920089 >> 172.21.15.59:0/4146920089 conn(0x7ff270076730 msgr2=0x7ff270076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 -- 172.21.15.59:0/4146920089 shutdown_connections 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.227+0000 7ff26fe00700 1 -- 172.21.15.59:0/4146920089 wait complete. 2024-04-02T17:11:22.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 Processor -- start 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 -- start start 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff27019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff270101760 0x7ff27019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270107530 0x7ff27019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112810 con 0x7ff270101760 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112690 con 0x7ff270100da0 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.228+0000 7ff26fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112990 con 0x7ff270107530 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff27019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff27019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53246/0 (socket says 172.21.15.59:53246) 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 -- 172.21.15.59:0/2979475416 learned_addr learned my addr 172.21.15.59:0/2979475416 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26e000700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff270101760 0x7ff27019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26f400700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270107530 0x7ff27019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 -- 172.21.15.59:0/2979475416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270107530 msgr2=0x7ff27019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff270107530 0x7ff27019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 -- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff270101760 msgr2=0x7ff27019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff270101760 0x7ff27019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.229+0000 7ff26ea00700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27019fe50 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff26ea00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff27019ae40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff26000e970 tx=0x7ff26000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260004220 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2600043c0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260003dc0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2701a0140 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.230+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2701a79c0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.231+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff27010e1f0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.234+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff26000c7e0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff267400700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff2540a3cd0 0x7ff2540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7ff2600d09c0 con 0x7ff270100da0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff26e000700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff2540a3cd0 0x7ff2540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff267400700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] conn(0x7ff2540ad890 0x7ff2540afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff26000be50 con 0x7ff2540ad890 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.235+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff2600d0d70 con 0x7ff270100da0 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.236+0000 7ff26f400700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] conn(0x7ff2540ad890 0x7ff2540afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.236+0000 7ff26e000700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff2540a3cd0 0x7ff2540a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff270103f10 tx=0x7ff25c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.236+0000 7ff26f400700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] conn(0x7ff2540ad890 0x7ff2540afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.237+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== osd.0 v2:172.21.15.59:6802/1605463711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff26000be50 con 0x7ff2540ad890 2024-04-02T17:11:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.256+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 --> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff27006a6c0 con 0x7ff2540ad890 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.256+0000 7ff267400700 1 -- 172.21.15.59:0/2979475416 <== osd.0 v2:172.21.15.59:6802/1605463711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff27006a6c0 con 0x7ff2540ad890 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] conn(0x7ff2540ad890 msgr2=0x7ff2540afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] conn(0x7ff2540ad890 0x7ff2540afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff2540a3cd0 msgr2=0x7ff2540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff2540a3cd0 0x7ff2540a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff270103f10 tx=0x7ff25c009260 comp rx=0 tx=0).stop 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 msgr2=0x7ff27019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 --2- 172.21.15.59:0/2979475416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff270100da0 0x7ff27019ae40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff26000e970 tx=0x7ff26000ec80 comp rx=0 tx=0).stop 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26f400700 1 -- 172.21.15.59:0/2979475416 reap_dead start 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 shutdown_connections 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 >> 172.21.15.59:0/2979475416 conn(0x7ff270076730 msgr2=0x7ff27010d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 shutdown_connections 2024-04-02T17:11:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:21.257+0000 7ff26fe00700 1 -- 172.21.15.59:0/2979475416 wait complete. 2024-04-02T17:11:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:22 smithi164 ceph-mon[36499]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:11:23.806 INFO:teuthology.orchestra.run.smithi059.stdout:201863462963 2024-04-02T17:11:23.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 -- 172.21.15.59:0/2218119252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc06a2a0 msgr2=0x7fb9cc06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 --2- 172.21.15.59:0/2218119252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc06a720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009920 tx=0x7fb9b8009c30 comp rx=0 tx=0).stop 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 -- 172.21.15.59:0/2218119252 shutdown_connections 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 --2- 172.21.15.59:0/2218119252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06ac60 0x7fb9cc10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 --2- 172.21.15.59:0/2218119252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc108650 0x7fb9cc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 --2- 172.21.15.59:0/2218119252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc06a720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 -- 172.21.15.59:0/2218119252 >> 172.21.15.59:0/2218119252 conn(0x7fb9cc078420 msgr2=0x7fb9cc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:23.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 -- 172.21.15.59:0/2218119252 shutdown_connections 2024-04-02T17:11:23.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.843+0000 7fb9d4400700 1 -- 172.21.15.59:0/2218119252 wait complete. 2024-04-02T17:11:23.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.845+0000 7fb9d4400700 1 Processor -- start 2024-04-02T17:11:23.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.845+0000 7fb9d4400700 1 -- start start 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc06ac60 0x7fb9cc101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc108650 0x7fb9cc102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1128f0 con 0x7fb9cc06a2a0 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc112770 con 0x7fb9cc108650 2024-04-02T17:11:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d4400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc112a70 con 0x7fb9cc06ac60 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d1a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d1a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc1015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51838/0 (socket says 172.21.15.59:51838) 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d2400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc108650 0x7fb9cc102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d1a00700 1 -- 172.21.15.59:0/1160683764 learned_addr learned my addr 172.21.15.59:0/1160683764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.846+0000 7fb9d1000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc06ac60 0x7fb9cc101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 -- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc06ac60 msgr2=0x7fb9cc101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc06ac60 0x7fb9cc101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 -- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06a2a0 msgr2=0x7fb9cc1015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9cc06a2a0 0x7fb9cc1015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1a73d0 con 0x7fb9cc108650 2024-04-02T17:11:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d1000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9cc06ac60 0x7fb9cc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d2400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc108650 0x7fb9cc102040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009cd0 tx=0x7fb9c400c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400cd10 con 0x7fb9cc108650 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c4003a00 con 0x7fb9cc108650 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1a7660 con 0x7fb9cc108650 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.847+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40045a0 con 0x7fb9cc108650 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.848+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a7b20 con 0x7fb9cc108650 2024-04-02T17:11:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.849+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9cc062d00 con 0x7fb9cc108650 2024-04-02T17:11:23.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.851+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb9c401a020 con 0x7fb9cc108650 2024-04-02T17:11:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.852+0000 7fb9ca000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b40a3cd0 0x7fb9b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.852+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fb9c40d0980 con 0x7fb9cc108650 2024-04-02T17:11:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.852+0000 7fb9d1a00700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b40a3cd0 0x7fb9b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.853+0000 7fb9ca000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] conn(0x7fb9b40ad890 0x7fb9b40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.853+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9c4008ec0 con 0x7fb9b40ad890 2024-04-02T17:11:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.853+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb9c40d0d30 con 0x7fb9cc108650 2024-04-02T17:11:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.853+0000 7fb9d1000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] conn(0x7fb9b40ad890 0x7fb9b40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.853+0000 7fb9d1a00700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b40a3cd0 0x7fb9b40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009ce0 tx=0x7fb9c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.854+0000 7fb9d1000700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] conn(0x7fb9b40ad890 0x7fb9b40afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.855+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== osd.8 v2:172.21.15.164:6816/1918014378 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9c4008ec0 con 0x7fb9b40ad890 2024-04-02T17:11:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.873+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 --> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9cc063180 con 0x7fb9b40ad890 2024-04-02T17:11:23.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9ca000700 1 -- 172.21.15.59:0/1160683764 <== osd.8 v2:172.21.15.164:6816/1918014378 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9cc063180 con 0x7fb9b40ad890 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] conn(0x7fb9b40ad890 msgr2=0x7fb9b40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] conn(0x7fb9b40ad890 0x7fb9b40afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b40a3cd0 msgr2=0x7fb9b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b40a3cd0 0x7fb9b40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009ce0 tx=0x7fb9c0009260 comp rx=0 tx=0).stop 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc108650 msgr2=0x7fb9cc102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d4400700 1 --2- 172.21.15.59:0/1160683764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9cc108650 0x7fb9cc102040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009cd0 tx=0x7fb9c400c620 comp rx=0 tx=0).stop 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.874+0000 7fb9d2400700 1 -- 172.21.15.59:0/1160683764 reap_dead start 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.875+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 shutdown_connections 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.875+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 >> 172.21.15.59:0/1160683764 conn(0x7fb9cc078420 msgr2=0x7fb9cc100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.875+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 shutdown_connections 2024-04-02T17:11:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:23.875+0000 7fb9d4400700 1 -- 172.21.15.59:0/1160683764 wait complete. 2024-04-02T17:11:24.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:24 smithi059 ceph-mon[31664]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:24 smithi154 ceph-mon[36479]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:24.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 -- 172.21.15.59:0/323240209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406a2a0 msgr2=0x7fe9f406a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 --2- 172.21.15.59:0/323240209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406a2a0 0x7fe9f406a720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009990 tx=0x7fe9e0009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 -- 172.21.15.59:0/323240209 shutdown_connections 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 --2- 172.21.15.59:0/323240209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406ac60 0x7fe9f410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 --2- 172.21.15.59:0/323240209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f4108650 0x7fe9f4108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 --2- 172.21.15.59:0/323240209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406a2a0 0x7fe9f406a720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 -- 172.21.15.59:0/323240209 >> 172.21.15.59:0/323240209 conn(0x7fe9f4078420 msgr2=0x7fe9f4078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 -- 172.21.15.59:0/323240209 shutdown_connections 2024-04-02T17:11:24.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.984+0000 7fe9fc400700 1 -- 172.21.15.59:0/323240209 wait complete. 2024-04-02T17:11:24.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.985+0000 7fe9fc400700 1 Processor -- start 2024-04-02T17:11:24.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.985+0000 7fe9fc400700 1 -- start start 2024-04-02T17:11:24.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406a2a0 0x7fe9f419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406ac60 0x7fe9f419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f4108650 0x7fe9f419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41128f0 con 0x7fe9f406ac60 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112770 con 0x7fe9f406a2a0 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fc400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112a70 con 0x7fe9f4108650 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9f9000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406ac60 0x7fe9f419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9f9a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406a2a0 0x7fe9f419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9f9000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406ac60 0x7fe9f419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51896/0 (socket says 172.21.15.59:51896) 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9f9000700 1 -- 172.21.15.59:0/1003705909 learned_addr learned my addr 172.21.15.59:0/1003705909 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:24.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.986+0000 7fe9fa400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f4108650 0x7fe9f419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:24.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 -- 172.21.15.59:0/1003705909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f4108650 msgr2=0x7fe9f419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:24.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f4108650 0x7fe9f419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 -- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406ac60 msgr2=0x7fe9f419b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:24.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f406ac60 0x7fe9f419b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f41a0070 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f9a00700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406a2a0 0x7fe9f419b060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8004220 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f41a0360 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e8004d10 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8005020 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.987+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f41a7be0 con 0x7fe9f406a2a0 2024-04-02T17:11:24.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.988+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9f410e180 con 0x7fe9f406a2a0 2024-04-02T17:11:24.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.990+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe9e8004750 con 0x7fe9f406a2a0 2024-04-02T17:11:24.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.992+0000 7fe9f2000700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9dc0a3cd0 0x7fe9dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:24.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.992+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe9e80d0690 con 0x7fe9f406a2a0 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.992+0000 7fe9f9000700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9dc0a3cd0 0x7fe9dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.992+0000 7fe9f2000700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] conn(0x7fe9dc0ad890 0x7fe9dc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.993+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9dc0b0400 con 0x7fe9dc0ad890 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.993+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe9e80d0a40 con 0x7fe9f406a2a0 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.993+0000 7fe9fa400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] conn(0x7fe9dc0ad890 0x7fe9dc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.993+0000 7fe9f9000700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9dc0a3cd0 0x7fe9dc0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009750 tx=0x7fe9e000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:24.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.994+0000 7fe9fa400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] conn(0x7fe9dc0ad890 0x7fe9dc0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.995+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== osd.9 v2:172.21.15.59:6826/980178556 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9dc0b0400 con 0x7fe9dc0ad890 2024-04-02T17:11:25.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.013+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 --> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9f4103cf0 con 0x7fe9dc0ad890 2024-04-02T17:11:25.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9f2000700 1 -- 172.21.15.59:0/1003705909 <== osd.9 v2:172.21.15.59:6826/980178556 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9f4103cf0 con 0x7fe9dc0ad890 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] conn(0x7fe9dc0ad890 msgr2=0x7fe9dc0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] conn(0x7fe9dc0ad890 0x7fe9dc0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9dc0a3cd0 msgr2=0x7fe9dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9dc0a3cd0 0x7fe9dc0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009750 tx=0x7fe9e000b280 comp rx=0 tx=0).stop 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406a2a0 msgr2=0x7fe9f419b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 --2- 172.21.15.59:0/1003705909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9f406a2a0 0x7fe9f419b060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800ed00 comp rx=0 tx=0).stop 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fa400700 1 -- 172.21.15.59:0/1003705909 reap_dead start 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 shutdown_connections 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.014+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 >> 172.21.15.59:0/1003705909 conn(0x7fe9f4078420 msgr2=0x7fe9f410d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.015+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 shutdown_connections 2024-04-02T17:11:25.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:25.015+0000 7fe9fc400700 1 -- 172.21.15.59:0/1003705909 wait complete. 2024-04-02T17:11:25.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:24 smithi164 ceph-mon[36499]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T17:11:25.702 INFO:teuthology.orchestra.run.smithi059.stdout:257698037800 2024-04-02T17:11:25.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-04-02T17:11:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 -- 172.21.15.59:0/888996936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d41036a0 msgr2=0x7fe7d4103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/888996936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d41036a0 0x7fe7d4103a80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009990 tx=0x7fe7c4009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 -- 172.21.15.59:0/888996936 shutdown_connections 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/888996936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d410cad0 0x7fe7d410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/888996936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d41036a0 0x7fe7d4103a80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/888996936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d4103fc0 0x7fe7d410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 -- 172.21.15.59:0/888996936 >> 172.21.15.59:0/888996936 conn(0x7fe7d40fceb0 msgr2=0x7fe7d40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 -- 172.21.15.59:0/888996936 shutdown_connections 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.877+0000 7fe7d3e00700 1 -- 172.21.15.59:0/888996936 wait complete. 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.878+0000 7fe7d3e00700 1 Processor -- start 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.878+0000 7fe7d3e00700 1 -- start start 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 0x7fe7d419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d4103fc0 0x7fe7d419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d410cad0 0x7fe7d4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4112810 con 0x7fe7d4103fc0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4112690 con 0x7fe7d41036a0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4112990 con 0x7fe7d410cad0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d2000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d4103fc0 0x7fe7d419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d2a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 0x7fe7d419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d2000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d4103fc0 0x7fe7d419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51876/0 (socket says 172.21.15.59:51876) 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d2a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 0x7fe7d419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53322/0 (socket says 172.21.15.59:53322) 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d2000700 1 -- 172.21.15.59:0/4006555680 learned_addr learned my addr 172.21.15.59:0/4006555680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.879+0000 7fe7d3400700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d410cad0 0x7fe7d4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 -- 172.21.15.59:0/4006555680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d410cad0 msgr2=0x7fe7d4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7d410cad0 0x7fe7d4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 -- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d4103fc0 msgr2=0x7fe7d419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7d4103fc0 0x7fe7d419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4199270 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.880+0000 7fe7d2a00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 0x7fe7d419e900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4006010 tx=0x7fe7c400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.881+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4027070 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.881+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d4199500 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.881+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c40056d0 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.881+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4022d90 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.881+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d41a57d0 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.882+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7d4062d80 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.884+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe7c402a070 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.885+0000 7fe7cb400700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7b80a3d50 0x7fe7b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.885+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe7c40d1f40 con 0x7fe7d41036a0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7d2000700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7b80a3d50 0x7fe7b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7cb400700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] conn(0x7fe7b80ad910 0x7fe7b80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7c4017e50 con 0x7fe7b80ad910 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe7c40d2390 con 0x7fe7d41036a0 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7d3400700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] conn(0x7fe7b80ad910 0x7fe7b80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.886+0000 7fe7d2000700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7b80a3d50 0x7fe7b80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7d410e250 tx=0x7fe7c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.887+0000 7fe7d3400700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] conn(0x7fe7b80ad910 0x7fe7b80afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.888+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== osd.1 v2:172.21.15.154:6800/212515491 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7c4017e50 con 0x7fe7b80ad910 2024-04-02T17:11:25.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.906+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 --> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7d4063180 con 0x7fe7b80ad910 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.907+0000 7fe7cb400700 1 -- 172.21.15.59:0/4006555680 <== osd.1 v2:172.21.15.154:6800/212515491 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7d4063180 con 0x7fe7b80ad910 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] conn(0x7fe7b80ad910 msgr2=0x7fe7b80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] conn(0x7fe7b80ad910 0x7fe7b80afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7b80a3d50 msgr2=0x7fe7b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7b80a3d50 0x7fe7b80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7d410e250 tx=0x7fe7c0009260 comp rx=0 tx=0).stop 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 msgr2=0x7fe7d419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 --2- 172.21.15.59:0/4006555680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d41036a0 0x7fe7d419e900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4006010 tx=0x7fe7c400bab0 comp rx=0 tx=0).stop 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3400700 1 -- 172.21.15.59:0/4006555680 reap_dead start 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 shutdown_connections 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 >> 172.21.15.59:0/4006555680 conn(0x7fe7d40fceb0 msgr2=0x7fe7d40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.908+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 shutdown_connections 2024-04-02T17:11:25.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.909+0000 7fe7d3e00700 1 -- 172.21.15.59:0/4006555680 wait complete. 2024-04-02T17:11:26.663 INFO:teuthology.orchestra.run.smithi059.stdout:34359738450 2024-04-02T17:11:26.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T17:11:26.859 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:26 smithi059 ceph-mon[31664]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-02T17:11:26.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:26 smithi154 ceph-mon[36479]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-02T17:11:27.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:26 smithi164 ceph-mon[36499]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-02T17:11:27.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 -- 172.21.15.59:0/1400711496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41085a0 msgr2=0x7f0ac4108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1400711496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41085a0 0x7f0ac4108980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009990 tx=0x7f0ab4009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 -- 172.21.15.59:0/1400711496 shutdown_connections 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1400711496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ac4102eb0 0x7f0ac410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1400711496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41085a0 0x7f0ac4108980 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1400711496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ac41024f0 0x7f0ac4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 -- 172.21.15.59:0/1400711496 >> 172.21.15.59:0/1400711496 conn(0x7f0ac40fdfc0 msgr2=0x7f0ac41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 -- 172.21.15.59:0/1400711496 shutdown_connections 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.598+0000 7f0ac3200700 1 -- 172.21.15.59:0/1400711496 wait complete. 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.600+0000 7f0ac3200700 1 Processor -- start 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.600+0000 7f0ac3200700 1 -- start start 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 0x7f0ac40748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ac4102eb0 0x7f0ac4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ac41085a0 0x7f0ac4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41127f0 con 0x7f0ac41085a0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4112670 con 0x7f0ac4102eb0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4112970 con 0x7f0ac41024f0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 0x7f0ac40748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 0x7f0ac40748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40914/0 (socket says 172.21.15.59:40914) 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 -- 172.21.15.59:0/1987943764 learned_addr learned my addr 172.21.15.59:0/1987943764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac2800700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ac41085a0 0x7f0ac4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1400700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ac4102eb0 0x7f0ac4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 -- 172.21.15.59:0/1987943764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ac4102eb0 msgr2=0x7f0ac4074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ac4102eb0 0x7f0ac4074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 -- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ac41085a0 msgr2=0x7f0ac4075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.601+0000 7f0ac1e00700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ac41085a0 0x7f0ac4075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0ac1e00700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41a7110 con 0x7f0ac41024f0 2024-04-02T17:11:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0ac1e00700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 0x7f0ac40748f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4000c00 tx=0x7f0ab400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4027070 con 0x7f0ac41024f0 2024-04-02T17:11:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41a73a0 con 0x7f0ac41024f0 2024-04-02T17:11:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab4022470 con 0x7f0ac41024f0 2024-04-02T17:11:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4017650 con 0x7f0ac41024f0 2024-04-02T17:11:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.602+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a7850 con 0x7f0ac41024f0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.603+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ac4062d80 con 0x7f0ac41024f0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.606+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0ab402a070 con 0x7f0ac41024f0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.607+0000 7f0abaa00700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0aa80a3d50 0x7f0aa80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.607+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f0ab40d1a50 con 0x7f0ac41024f0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.607+0000 7f0ac1400700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0aa80a3d50 0x7f0aa80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.607+0000 7f0abaa00700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] conn(0x7f0aa80ad910 0x7f0aa80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.608+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ab402be50 con 0x7f0aa80ad910 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.608+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0ab40d1ea0 con 0x7f0ac41024f0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.608+0000 7f0ac2800700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] conn(0x7f0aa80ad910 0x7f0aa80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.608+0000 7f0ac1400700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0aa80a3d50 0x7f0aa80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4107b50 tx=0x7f0ab0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.608+0000 7f0ac2800700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] conn(0x7f0aa80ad910 0x7f0aa80afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.609+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== osd.2 v2:172.21.15.164:6800/4028786479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ab402be50 con 0x7f0aa80ad910 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.628+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 --> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ac4063200 con 0x7f0aa80ad910 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.628+0000 7f0abaa00700 1 -- 172.21.15.59:0/1987943764 <== osd.2 v2:172.21.15.164:6800/4028786479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ac4063200 con 0x7f0aa80ad910 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] conn(0x7f0aa80ad910 msgr2=0x7f0aa80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] conn(0x7f0aa80ad910 0x7f0aa80afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0aa80a3d50 msgr2=0x7f0aa80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0aa80a3d50 0x7f0aa80a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4107b50 tx=0x7f0ab0009260 comp rx=0 tx=0).stop 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 msgr2=0x7f0ac40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 --2- 172.21.15.59:0/1987943764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac41024f0 0x7f0ac40748f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4000c00 tx=0x7f0ab400b610 comp rx=0 tx=0).stop 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac2800700 1 -- 172.21.15.59:0/1987943764 reap_dead start 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 shutdown_connections 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.629+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 >> 172.21.15.59:0/1987943764 conn(0x7f0ac40fdfc0 msgr2=0x7f0ac40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.630+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 shutdown_connections 2024-04-02T17:11:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:24.630+0000 7f0ac3200700 1 -- 172.21.15.59:0/1987943764 wait complete. 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 -- 172.21.15.59:0/3999845271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 msgr2=0x7fc31c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 --2- 172.21.15.59:0/3999845271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 0x7fc31c108a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc310009990 tx=0x7fc310009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 -- 172.21.15.59:0/3999845271 shutdown_connections 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 --2- 172.21.15.59:0/3999845271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 0x7fc31c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 --2- 172.21.15.59:0/3999845271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 0x7fc31c108a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 --2- 172.21.15.59:0/3999845271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 -- 172.21.15.59:0/3999845271 >> 172.21.15.59:0/3999845271 conn(0x7fc31c0fe0a0 msgr2=0x7fc31c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 -- 172.21.15.59:0/3999845271 shutdown_connections 2024-04-02T17:11:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.743+0000 7fc323800700 1 -- 172.21.15.59:0/3999845271 wait complete. 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.744+0000 7fc323800700 1 Processor -- start 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.744+0000 7fc323800700 1 -- start start 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 0x7fc31c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 0x7fc31c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c112970 con 0x7fc31c108660 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1127f0 con 0x7fc31c1025d0 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc323800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c112af0 con 0x7fc31c102f90 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc320e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc320e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53360/0 (socket says 172.21.15.59:53360) 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc320e00700 1 -- 172.21.15.59:0/837851840 learned_addr learned my addr 172.21.15.59:0/837851840 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc31be00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 0x7fc31c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.745+0000 7fc321800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 0x7fc31c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 -- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 msgr2=0x7fc31c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 0x7fc31c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 -- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 msgr2=0x7fc31c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc31c108660 0x7fc31c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c19ff50 con 0x7fc31c1025d0 2024-04-02T17:11:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc31be00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c102f90 0x7fc31c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc320e00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c19af40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3100047b0 tx=0x7fc31000b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310026070 con 0x7fc31c1025d0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c1a01e0 con 0x7fc31c1025d0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310004cc0 con 0x7fc31c1025d0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.746+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310005470 con 0x7fc31c1025d0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.747+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1a7ac0 con 0x7fc31c1025d0 2024-04-02T17:11:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.748+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc31c0ffec0 con 0x7fc31c1025d0 2024-04-02T17:11:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.749+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc310029070 con 0x7fc31c1025d0 2024-04-02T17:11:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc319600700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc3040a3d50 0x7fc3040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fc3100d1070 con 0x7fc31c1025d0 2024-04-02T17:11:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc31be00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc3040a3d50 0x7fc3040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc319600700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] conn(0x7fc3040ad910 0x7fc3040afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc31000dec0 con 0x7fc3040ad910 2024-04-02T17:11:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.751+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc3100d8050 con 0x7fc31c1025d0 2024-04-02T17:11:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.752+0000 7fc321800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] conn(0x7fc3040ad910 0x7fc3040afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.752+0000 7fc31be00700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc3040a3d50 0x7fc3040a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc31c10f080 tx=0x7fc308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.752+0000 7fc321800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] conn(0x7fc3040ad910 0x7fc3040afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:27.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.753+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== osd.11 v2:172.21.15.164:6824/589108001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc31000dec0 con 0x7fc3040ad910 2024-04-02T17:11:27.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.773+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 --> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc31c103d10 con 0x7fc3040ad910 2024-04-02T17:11:27.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.773+0000 7fc319600700 1 -- 172.21.15.59:0/837851840 <== osd.11 v2:172.21.15.164:6824/589108001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc31c103d10 con 0x7fc3040ad910 2024-04-02T17:11:27.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] conn(0x7fc3040ad910 msgr2=0x7fc3040afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] conn(0x7fc3040ad910 0x7fc3040afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:27.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc3040a3d50 msgr2=0x7fc3040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc3040a3d50 0x7fc3040a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc31c10f080 tx=0x7fc308009260 comp rx=0 tx=0).stop 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 msgr2=0x7fc31c19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 --2- 172.21.15.59:0/837851840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc31c1025d0 0x7fc31c19af40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3100047b0 tx=0x7fc31000b610 comp rx=0 tx=0).stop 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc321800700 1 -- 172.21.15.59:0/837851840 reap_dead start 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 shutdown_connections 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 >> 172.21.15.59:0/837851840 conn(0x7fc31c0fe0a0 msgr2=0x7fc31c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 shutdown_connections 2024-04-02T17:11:27.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:27.774+0000 7fc323800700 1 -- 172.21.15.59:0/837851840 wait complete. 2024-04-02T17:11:28.366 INFO:teuthology.orchestra.run.smithi059.stdout:77309411402 2024-04-02T17:11:28.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T17:11:28.826 INFO:teuthology.orchestra.run.smithi059.stdout:55834574926 2024-04-02T17:11:28.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T17:11:28.890 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:28 smithi059 ceph-mon[31664]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T17:11:28.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:28 smithi154 ceph-mon[36479]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T17:11:29.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:28 smithi164 ceph-mon[36499]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T17:11:29.107 INFO:teuthology.orchestra.run.smithi059.stdout:292057776163 2024-04-02T17:11:29.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-04-02T17:11:29.136 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:29.284 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:29.299 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:29.620 INFO:teuthology.orchestra.run.smithi059.stdout:352187318299 2024-04-02T17:11:29.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-04-02T17:11:30.459 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:30.468 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:30.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:30 smithi059 ceph-mon[31664]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-02T17:11:30.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:30 smithi154 ceph-mon[36479]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-02T17:11:31.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:30 smithi164 ceph-mon[36499]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-02T17:11:32.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:32 smithi154 ceph-mon[36479]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:32.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:32 smithi059 ceph-mon[31664]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:33.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:32 smithi164 ceph-mon[36499]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:34.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:34 smithi059 ceph-mon[31664]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:35.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:34 smithi164 ceph-mon[36499]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:35.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:34 smithi154 ceph-mon[36479]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 -- 172.21.15.59:0/1268190725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 msgr2=0x7fe6bc10f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1268190725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc10f450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400b0a0 tx=0x7fe6b400b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 -- 172.21.15.59:0/1268190725 shutdown_connections 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1268190725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc10f450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1268190725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc108650 0x7fe6bc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1268190725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc06a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.562+0000 7fe6c3400700 1 -- 172.21.15.59:0/1268190725 >> 172.21.15.59:0/1268190725 conn(0x7fe6bc078420 msgr2=0x7fe6bc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.563+0000 7fe6c3400700 1 -- 172.21.15.59:0/1268190725 shutdown_connections 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.563+0000 7fe6c3400700 1 -- 172.21.15.59:0/1268190725 wait complete. 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 Processor -- start 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 -- start start 2024-04-02T17:11:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc108650 0x7fe6bc102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc112940 con 0x7fe6bc06ac60 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc1127c0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.564+0000 7fe6c3400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc112ac0 con 0x7fe6bc108650 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6bbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6bbe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51652/0 (socket says 172.21.15.59:51652) 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6bbe00700 1 -- 172.21.15.59:0/1171470793 learned_addr learned my addr 172.21.15.59:0/1171470793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c1400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc108650 0x7fe6bc102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 -- 172.21.15.59:0/1171470793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc108650 msgr2=0x7fe6bc102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc108650 0x7fe6bc102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 -- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 msgr2=0x7fe6bc101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6c0a00700 1 -- 172.21.15.59:0/1171470793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc1a72a0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.565+0000 7fe6bbe00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6c0a00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc1015c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9f0 tx=0x7fe6b000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000cb70 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc1a7590 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b000ee90 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0003dc0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.566+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc1a7b20 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.567+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6bc0692b0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.569+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6b0003f60 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.571+0000 7fe6b9600700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 0x7fe6a40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.571+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe6b00d07c0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.571+0000 7fe6bbe00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 0x7fe6a40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.572+0000 7fe6bbe00700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 0x7fe6a40a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc078ad0 tx=0x7fe6a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.573+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe6b009cfd0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.816+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe6bc102af0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.817+0000 7fe6b9600700 1 -- 172.21.15.59:0/1171470793 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6b009c8d0 con 0x7fe6bc06a2a0 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stdout:107374182468 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 msgr2=0x7fe6a40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 0x7fe6a40a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc078ad0 tx=0x7fe6a8009260 comp rx=0 tx=0).stop 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 msgr2=0x7fe6bc1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc1015c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9f0 tx=0x7fe6b000ed00 comp rx=0 tx=0).stop 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 shutdown_connections 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6bc108650 0x7fe6bc102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6bc06a2a0 0x7fe6bc1015c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe6a40a3d50 0x7fe6a40a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 --2- 172.21.15.59:0/1171470793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc06ac60 0x7fe6bc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 >> 172.21.15.59:0/1171470793 conn(0x7fe6bc078420 msgr2=0x7fe6bc100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 shutdown_connections 2024-04-02T17:11:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:34.821+0000 7fe6c3400700 1 -- 172.21.15.59:0/1171470793 wait complete. 2024-04-02T17:11:35.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:35 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1171470793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T17:11:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:35 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1171470793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T17:11:36.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:35 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1171470793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T17:11:36.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:36 smithi059 ceph-mon[31664]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T17:11:36.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:36 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:11:37.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:36 smithi164 ceph-mon[36499]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T17:11:37.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:36 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:11:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.124+0000 7fcd7ee00700 1 -- 172.21.15.59:0/1566420543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 msgr2=0x7fcd7810f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.124+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/1566420543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd7810f4a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b0a0 tx=0x7fcd7000b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.124+0000 7fcd7ee00700 1 -- 172.21.15.59:0/1566420543 shutdown_connections 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/1566420543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd7810f4a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/1566420543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd78108660 0x7fcd78108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/1566420543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd781025d0 0x7fcd78102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 -- 172.21.15.59:0/1566420543 >> 172.21.15.59:0/1566420543 conn(0x7fcd780fe0a0 msgr2=0x7fcd781004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 -- 172.21.15.59:0/1566420543 shutdown_connections 2024-04-02T17:11:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.125+0000 7fcd7ee00700 1 -- 172.21.15.59:0/1566420543 wait complete. 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 Processor -- start 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 -- start start 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd781025d0 0x7fcd780748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 0x7fcd78075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78112970 con 0x7fcd78102f90 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781127f0 con 0x7fcd781025d0 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.126+0000 7fcd7ee00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78112af0 con 0x7fcd78108660 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd77400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd77400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51692/0 (socket says 172.21.15.59:51692) 2024-04-02T17:11:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd77400700 1 -- 172.21.15.59:0/545988172 learned_addr learned my addr 172.21.15.59:0/545988172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 0x7fcd78075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd77e00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd781025d0 0x7fcd780748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 -- 172.21.15.59:0/545988172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd781025d0 msgr2=0x7fcd780748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd781025d0 0x7fcd780748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 -- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 msgr2=0x7fcd78074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 -- 172.21.15.59:0/545988172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781a71c0 con 0x7fcd78108660 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd77400700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.127+0000 7fcd7ce00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 0x7fcd78075370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b070 tx=0x7fcd70004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.128+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70021070 con 0x7fcd78108660 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.128+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd70007b80 con 0x7fcd78108660 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.128+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70005040 con 0x7fcd78108660 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.128+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781a7450 con 0x7fcd78108660 2024-04-02T17:11:37.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.128+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a78b0 con 0x7fcd78108660 2024-04-02T17:11:37.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.129+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd780692b0 con 0x7fcd78108660 2024-04-02T17:11:37.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.131+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcd70024070 con 0x7fcd78108660 2024-04-02T17:11:37.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.132+0000 7fcd74c00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 0x7fcd5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:37.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.133+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcd7000f020 con 0x7fcd78108660 2024-04-02T17:11:37.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.133+0000 7fcd77e00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 0x7fcd5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:37.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.133+0000 7fcd77e00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 0x7fcd5c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd78103710 tx=0x7fcd68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:37.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.134+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcd7002b020 con 0x7fcd78108660 2024-04-02T17:11:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:36 smithi154 ceph-mon[36479]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-02T17:11:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:36 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:11:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.377+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fcd781a7c70 con 0x7fcd78108660 2024-04-02T17:11:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.378+0000 7fcd74c00700 1 -- 172.21.15.59:0/545988172 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcd70007d30 con 0x7fcd78108660 2024-04-02T17:11:37.379 INFO:teuthology.orchestra.run.smithi059.stdout:317827579937 2024-04-02T17:11:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.381+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 msgr2=0x7fcd5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.381+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 0x7fcd5c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd78103710 tx=0x7fcd68009260 comp rx=0 tx=0).stop 2024-04-02T17:11:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.381+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 msgr2=0x7fcd78075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.381+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 0x7fcd78075370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b070 tx=0x7fcd70004c80 comp rx=0 tx=0).stop 2024-04-02T17:11:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 shutdown_connections 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd78108660 0x7fcd78075370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd781025d0 0x7fcd780748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd5c0a3cd0 0x7fcd5c0a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 --2- 172.21.15.59:0/545988172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78102f90 0x7fcd78074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 >> 172.21.15.59:0/545988172 conn(0x7fcd780fe0a0 msgr2=0x7fcd780ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 shutdown_connections 2024-04-02T17:11:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.382+0000 7fcd7ee00700 1 -- 172.21.15.59:0/545988172 wait complete. 2024-04-02T17:11:37.469 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182468 for osd.3 2024-04-02T17:11:37.470 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:37.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:37 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3317025666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T17:11:37.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:37 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/545988172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T17:11:38.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3317025666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T17:11:38.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:37 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/545988172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T17:11:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:38.108 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:38.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579933 got 317827579937 for osd.10 2024-04-02T17:11:38.109 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:38.116 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:38.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:37 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3317025666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T17:11:38.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:37 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/545988172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T17:11:38.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 -- 172.21.15.59:0/1540628900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa540102eb0 msgr2=0x7fa54010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:38.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 --2- 172.21.15.59:0/1540628900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa540102eb0 0x7fa54010f370 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa53800b0d0 tx=0x7fa53800b3e0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 -- 172.21.15.59:0/1540628900 shutdown_connections 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 --2- 172.21.15.59:0/1540628900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa540102eb0 0x7fa54010f370 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 --2- 172.21.15.59:0/1540628900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401085a0 0x7fa540108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 --2- 172.21.15.59:0/1540628900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5401024f0 0x7fa540102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 -- 172.21.15.59:0/1540628900 >> 172.21.15.59:0/1540628900 conn(0x7fa5400fdfc0 msgr2=0x7fa5401003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.021+0000 7fa53fe00700 1 -- 172.21.15.59:0/1540628900 shutdown_connections 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.022+0000 7fa53fe00700 1 -- 172.21.15.59:0/1540628900 wait complete. 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 Processor -- start 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 -- start start 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 0x7fa54019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa540112840 con 0x7fa5401024f0 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401126c0 con 0x7fa5401085a0 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401129c0 con 0x7fa540102eb0 2024-04-02T17:11:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53ea00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 0x7fa54019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53f400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.023+0000 7fa53e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53f400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38032/0 (socket says 172.21.15.59:38032) 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53ea00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 0x7fa54019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51672/0 (socket says 172.21.15.59:51672) 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43548/0 (socket says 172.21.15.59:43548) 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53f400700 1 -- 172.21.15.59:0/3317025666 learned_addr learned my addr 172.21.15.59:0/3317025666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 -- 172.21.15.59:0/3317025666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 msgr2=0x7fa54019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 -- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 msgr2=0x7fa54019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 0x7fa54019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 -- 172.21.15.59:0/3317025666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54019fda0 con 0x7fa540102eb0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53f400700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.024+0000 7fa53e000700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e9f0 tx=0x7fa52c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.025+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004220 con 0x7fa540102eb0 2024-04-02T17:11:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.025+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52c004d10 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.025+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401a0090 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.025+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c005020 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.025+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a7910 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.026+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5400692b0 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.030+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa52c004750 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.031+0000 7fa537400700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 0x7fa5240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.031+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fa52c0d0540 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.032+0000 7fa53ea00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 0x7fa5240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.032+0000 7fa53ea00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 0x7fa5240a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa530009c60 tx=0x7fa530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.033+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa52c09cde0 con 0x7fa540102eb0 2024-04-02T17:11:38.193 INFO:teuthology.orchestra.run.smithi059.stdout:34359738453 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.268+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa54019c0f0 con 0x7fa540102eb0 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.268+0000 7fa537400700 1 -- 172.21.15.59:0/3317025666 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa54019c0f0 con 0x7fa540102eb0 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.271+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 msgr2=0x7fa5240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.271+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 0x7fa5240a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa530009c60 tx=0x7fa530009260 comp rx=0 tx=0).stop 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.271+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 msgr2=0x7fa54019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.271+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e9f0 tx=0x7fa52c00ed00 comp rx=0 tx=0).stop 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.271+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 shutdown_connections 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5401085a0 0x7fa54019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa5240a3d50 0x7fa5240a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5401024f0 0x7fa54019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 --2- 172.21.15.59:0/3317025666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa540102eb0 0x7fa54019b2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:38.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 >> 172.21.15.59:0/3317025666 conn(0x7fa5400fdfc0 msgr2=0x7fa5400ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:38.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 shutdown_connections 2024-04-02T17:11:38.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:37.272+0000 7fa53fe00700 1 -- 172.21.15.59:0/3317025666 wait complete. 2024-04-02T17:11:39.061 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:39.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738453 for osd.0 2024-04-02T17:11:39.105 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:39.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:39 smithi059 ceph-mon[31664]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T17:11:39.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:39 smithi164 ceph-mon[36499]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T17:11:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:39 smithi154 ceph-mon[36479]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T17:11:39.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 -- 172.21.15.59:0/327105395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac108630 msgr2=0x7fe8ac108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/327105395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac108630 0x7fe8ac108a10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009990 tx=0x7fe89c009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 -- 172.21.15.59:0/327105395 shutdown_connections 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/327105395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac10f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/327105395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac108630 0x7fe8ac108a10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/327105395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac06afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 -- 172.21.15.59:0/327105395 >> 172.21.15.59:0/327105395 conn(0x7fe8ac077540 msgr2=0x7fe8ac077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 -- 172.21.15.59:0/327105395 shutdown_connections 2024-04-02T17:11:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.918+0000 7fe8b0e00700 1 -- 172.21.15.59:0/327105395 wait complete. 2024-04-02T17:11:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.919+0000 7fe8b0e00700 1 Processor -- start 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 -- start start 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac108630 0x7fe8ac19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112940 con 0x7fe8ac108630 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac1127c0 con 0x7fe8ac06ab30 2024-04-02T17:11:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8b0e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112ac0 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8a9e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8a9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8a9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43612/0 (socket says 172.21.15.59:43612) 2024-04-02T17:11:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8a9400700 1 -- 172.21.15.59:0/1722784621 learned_addr learned my addr 172.21.15.59:0/1722784621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.920+0000 7fe8a9e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38064/0 (socket says 172.21.15.59:38064) 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8aa800700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac108630 0x7fe8ac19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 -- 172.21.15.59:0/1722784621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 msgr2=0x7fe8ac198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 -- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac108630 msgr2=0x7fe8ac19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac108630 0x7fe8ac19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 -- 172.21.15.59:0/1722784621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac19dd80 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a9400700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe89800e9c0 tx=0x7fe89800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.921+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89800cbd0 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.922+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89800cd70 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.922+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac19e070 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.922+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898010430 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.922+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a58f0 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.923+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac10ab30 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.925+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe898004750 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 -- 172.21.15.59:0/2991804303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8181062c0 msgr2=0x7fa818106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 --2- 172.21.15.59:0/2991804303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8181062c0 0x7fa818106740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa804009990 tx=0x7fa804009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 -- 172.21.15.59:0/2991804303 shutdown_connections 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 --2- 172.21.15.59:0/2991804303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 0x7fa818111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 --2- 172.21.15.59:0/2991804303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa818104b70 0x7fa818104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 --2- 172.21.15.59:0/2991804303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8181062c0 0x7fa818106740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 -- 172.21.15.59:0/2991804303 >> 172.21.15.59:0/2991804303 conn(0x7fa818100290 msgr2=0x7fa8181026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 -- 172.21.15.59:0/2991804303 shutdown_connections 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.826+0000 7fa81da00700 1 -- 172.21.15.59:0/2991804303 wait complete. 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.828+0000 7fa81da00700 1 Processor -- start 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.828+0000 7fa81da00700 1 -- start start 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.828+0000 7fa81da00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.828+0000 7fa81da00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8181062c0 0x7fa81819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.828+0000 7fa81da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 0x7fa81819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa81da00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114c00 con 0x7fa818104b70 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa81da00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114a80 con 0x7fa8181062c0 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa81da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114d80 con 0x7fa818106c80 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51710/0 (socket says 172.21.15.59:51710) 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 -- 172.21.15.59:0/1920217291 learned_addr learned my addr 172.21.15.59:0/1920217291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816000700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8181062c0 0x7fa81819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa817400700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 0x7fa81819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 -- 172.21.15.59:0/1920217291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 msgr2=0x7fa81819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 0x7fa81819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 -- 172.21.15.59:0/1920217291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8181062c0 msgr2=0x7fa81819b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8181062c0 0x7fa81819b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.829+0000 7fa816a00700 1 -- 172.21.15.59:0/1920217291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8181a0070 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa816a00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa80800eb30 tx=0x7fa80800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800cbd0 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa808004500 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8181a0360 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808010430 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.830+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8181a7be0 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.832+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8180692b0 con 0x7fa818104b70 2024-04-02T17:11:39.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.835+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa808003f00 con 0x7fa818104b70 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.837+0000 7fa80f400700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 0x7fa7fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.926+0000 7fe8a2a00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 0x7fe8900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.837+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fa8080d0860 con 0x7fa818104b70 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.837+0000 7fa816000700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 0x7fa7fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.838+0000 7fa816000700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 0x7fa7fc0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa818102760 tx=0x7fa804005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.840+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa80809d100 con 0x7fa818104b70 2024-04-02T17:11:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.927+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe8980d0960 con 0x7fe8ac06b4f0 2024-04-02T17:11:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.927+0000 7fe8a9e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 0x7fe8900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.927+0000 7fe8a9e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 0x7fe8900a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b2e0 tx=0x7fe89c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:39.929+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe89809d170 con 0x7fe8ac06b4f0 2024-04-02T17:11:40.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.079+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa81819c3c0 con 0x7fa818104b70 2024-04-02T17:11:40.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.080+0000 7fa80f400700 1 -- 172.21.15.59:0/1920217291 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa80809ca00 con 0x7fa818104b70 2024-04-02T17:11:40.080 INFO:teuthology.orchestra.run.smithi059.stdout:201863462966 2024-04-02T17:11:40.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 msgr2=0x7fa7fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:40.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 0x7fa7fc0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa818102760 tx=0x7fa804005fd0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 msgr2=0x7fa81819b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:40.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa80800eb30 tx=0x7fa80800ee40 comp rx=0 tx=0).stop 2024-04-02T17:11:40.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 shutdown_connections 2024-04-02T17:11:40.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa818106c80 0x7fa81819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7fc0a3d50 0x7fa7fc0a6210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa818104b70 0x7fa81819b0b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 --2- 172.21.15.59:0/1920217291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8181062c0 0x7fa81819b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.083+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 >> 172.21.15.59:0/1920217291 conn(0x7fa818100290 msgr2=0x7fa81810a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.084+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 shutdown_connections 2024-04-02T17:11:40.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.084+0000 7fa81da00700 1 -- 172.21.15.59:0/1920217291 wait complete. 2024-04-02T17:11:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.171+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe8ac10a510 con 0x7fe8ac06b4f0 2024-04-02T17:11:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.171+0000 7fe8a2a00700 1 -- 172.21.15.59:0/1722784621 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe89809ca70 con 0x7fe8ac06b4f0 2024-04-02T17:11:40.172 INFO:teuthology.orchestra.run.smithi059.stdout:167503724604 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 msgr2=0x7fe8900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 0x7fe8900a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b2e0 tx=0x7fe89c005fd0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 msgr2=0x7fe8ac1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe89800e9c0 tx=0x7fe89800ecd0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 shutdown_connections 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe8900a3d50 0x7fe8900a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8ac108630 0x7fe8ac19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ac06ab30 0x7fe8ac198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 --2- 172.21.15.59:0/1722784621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ac06b4f0 0x7fe8ac1992b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 >> 172.21.15.59:0/1722784621 conn(0x7fe8ac077540 msgr2=0x7fe8ac100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.174+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 shutdown_connections 2024-04-02T17:11:40.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:40.175+0000 7fe8b0e00700 1 -- 172.21.15.59:0/1722784621 wait complete. 2024-04-02T17:11:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:40.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1920217291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T17:11:40.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:40 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1920217291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T17:11:40.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:40 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1920217291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T17:11:40.735 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:41.381 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:41 smithi059 ceph-mon[31664]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:41.381 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1722784621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T17:11:41.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:41 smithi164 ceph-mon[36499]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:41.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1722784621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T17:11:41.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:41 smithi154 ceph-mon[36479]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:41.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1722784621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T17:11:42.361 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724600 got 167503724604 for osd.5 2024-04-02T17:11:42.361 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:43.522 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:43 smithi059 ceph-mon[31664]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:43 smithi164 ceph-mon[36499]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:43 smithi154 ceph-mon[36479]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:11:43.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462966 for osd.6 2024-04-02T17:11:43.948 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:45.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:45 smithi059 ceph-mon[31664]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:45 smithi164 ceph-mon[36499]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:45 smithi154 ceph-mon[36479]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:46.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 -- 172.21.15.59:0/1115230997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c075140 msgr2=0x7f926c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:46.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 --2- 172.21.15.59:0/1115230997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c075140 0x7f926c10eab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 -- 172.21.15.59:0/1115230997 shutdown_connections 2024-04-02T17:11:46.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 --2- 172.21.15.59:0/1115230997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c10eff0 0x7f926c1113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 --2- 172.21.15.59:0/1115230997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 --2- 172.21.15.59:0/1115230997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c075140 0x7f926c10eab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 -- 172.21.15.59:0/1115230997 >> 172.21.15.59:0/1115230997 conn(0x7f926c079750 msgr2=0x7f926c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 -- 172.21.15.59:0/1115230997 shutdown_connections 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.259+0000 7f9272e00700 1 -- 172.21.15.59:0/1115230997 wait complete. 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.260+0000 7f9272e00700 1 Processor -- start 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.260+0000 7f9272e00700 1 -- start start 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 0x7f926c19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 0x7f926c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f926c114bf0 con 0x7f926c10eff0 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f926c114a70 con 0x7f926c075140 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9272e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f926c114d70 con 0x7f926c074820 2024-04-02T17:11:46.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9270e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 0x7f926c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40500/0 (socket says 172.21.15.59:40500) 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 -- 172.21.15.59:0/3265970186 learned_addr learned my addr 172.21.15.59:0/3265970186 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926b400700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 0x7f926c19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 -- 172.21.15.59:0/3265970186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 msgr2=0x7f926c19b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 0x7f926c19b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 -- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 msgr2=0x7f926c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 0x7f926c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926be00700 1 -- 172.21.15.59:0/3265970186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1a0000 con 0x7f926c074820 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f9270e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 0x7f926c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.261+0000 7f926b400700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 0x7f926c19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f926be00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9c0 tx=0x7f925c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c00cbd0 con 0x7f926c074820 2024-04-02T17:11:46.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c00ee60 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1a02f0 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c005630 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.262+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1a7b70 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.263+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c0692b0 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.265+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f925c004750 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.266+0000 7f9268c00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 0x7f92500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.267+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f925c0d0800 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.267+0000 7f926b400700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 0x7f92500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.267+0000 7f926b400700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 0x7f92500a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f926c1020d0 tx=0x7f9258005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.271+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f925c09d010 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.511+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f926c19c350 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.511+0000 7f9268c00700 1 -- 172.21.15.59:0/3265970186 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f925c09c910 con 0x7f926c074820 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 msgr2=0x7f92500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:46.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 0x7f92500a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f926c1020d0 tx=0x7f9258005fd0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 msgr2=0x7f926c19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9c0 tx=0x7f925c00ecd0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 shutdown_connections 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92500a3d50 0x7f92500a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f926c10eff0 0x7f926c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f926c074820 0x7f926c19b040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 --2- 172.21.15.59:0/3265970186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f926c075140 0x7f926c19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 >> 172.21.15.59:0/3265970186 conn(0x7f926c079750 msgr2=0x7f926c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 shutdown_connections 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:46.515+0000 7f9272e00700 1 -- 172.21.15.59:0/3265970186 wait complete. 2024-04-02T17:11:46.546 INFO:teuthology.orchestra.run.smithi059.stdout:292057776167 2024-04-02T17:11:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3373460267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 msgr2=0x7f8c7806a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3373460267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c7806a720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c640099c0 tx=0x7f8c64009cd0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3373460267 shutdown_connections 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3373460267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c7810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3373460267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 0x7f8c78108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3373460267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c7806a720 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3373460267 >> 172.21.15.59:0/3373460267 conn(0x7f8c78078420 msgr2=0x7f8c78078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3373460267 shutdown_connections 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.199+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3373460267 wait complete. 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.200+0000 7f8c7ce00700 1 Processor -- start 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.200+0000 7f8c7ce00700 1 -- start start 2024-04-02T17:11:47.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c78199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 0x7f8c7819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781128f0 con 0x7f8c78108650 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112770 con 0x7f8c7806ac60 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c7ce00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112a70 con 0x7f8c7806a2a0 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c76800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 0x7f8c7819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c75e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c75400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c78199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c75e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40546/0 (socket says 172.21.15.59:40546) 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.201+0000 7f8c75e00700 1 -- 172.21.15.59:0/3585096308 learned_addr learned my addr 172.21.15.59:0/3585096308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:47.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 -- 172.21.15.59:0/3585096308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 msgr2=0x7f8c78199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c78199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 -- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 msgr2=0x7f8c7819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 0x7f8c7819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 -- 172.21.15.59:0/3585096308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819dda0 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75400700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c78199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c75e00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e9a0 tx=0x7f8c6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800ca10 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6800cbb0 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018680 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7819e090 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.202+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a5910 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.203+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c780692b0 con 0x7f8c7806a2a0 2024-04-02T17:11:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.206+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8c68010430 con 0x7f8c7806a2a0 2024-04-02T17:11:47.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:47 smithi059 ceph-mon[31664]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:47.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:47 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3265970186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T17:11:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.207+0000 7f8c6ea00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 0x7f8c5c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.207+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8c680d0720 con 0x7f8c7806a2a0 2024-04-02T17:11:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.207+0000 7f8c75400700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 0x7f8c5c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.208+0000 7f8c75400700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 0x7f8c5c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8c78078a90 tx=0x7f8c64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.209+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8c6809cf30 con 0x7f8c7806a2a0 2024-04-02T17:11:47.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.446+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8c7802def0 con 0x7f8c7806a2a0 2024-04-02T17:11:47.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.447+0000 7f8c6ea00700 1 -- 172.21.15.59:0/3585096308 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c6809c830 con 0x7f8c7806a2a0 2024-04-02T17:11:47.446 INFO:teuthology.orchestra.run.smithi059.stdout:77309411407 2024-04-02T17:11:47.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.449+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 msgr2=0x7f8c5c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.449+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 0x7f8c5c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8c78078a90 tx=0x7f8c64005fb0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.449+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 msgr2=0x7f8c78198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.449+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e9a0 tx=0x7f8c6800ecb0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 shutdown_connections 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8c5c0a3d50 0x7f8c5c0a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c78108650 0x7f8c7819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c7806a2a0 0x7f8c78198de0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 --2- 172.21.15.59:0/3585096308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c7806ac60 0x7f8c78199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 >> 172.21.15.59:0/3585096308 conn(0x7f8c78078420 msgr2=0x7f8c78100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 shutdown_connections 2024-04-02T17:11:47.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.450+0000 7f8c7ce00700 1 -- 172.21.15.59:0/3585096308 wait complete. 2024-04-02T17:11:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:47 smithi164 ceph-mon[36499]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:47 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3265970186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T17:11:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:47 smithi154 ceph-mon[36479]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:47.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:47 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3265970186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T17:11:47.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 -- 172.21.15.59:0/664852695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b801051a0 msgr2=0x7f0b801116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 --2- 172.21.15.59:0/664852695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b801051a0 0x7f0b801116b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009f90 tx=0x7f0b7800b330 comp rx=0 tx=0).stop 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 -- 172.21.15.59:0/664852695 shutdown_connections 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 --2- 172.21.15.59:0/664852695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b801051a0 0x7f0b801116b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 --2- 172.21.15.59:0/664852695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b8010a7e0 0x7f0b8010abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 --2- 172.21.15.59:0/664852695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801047e0 0x7f0b80104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 -- 172.21.15.59:0/664852695 >> 172.21.15.59:0/664852695 conn(0x7f0b801002d0 msgr2=0x7f0b801026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.030+0000 7f0b86800700 1 -- 172.21.15.59:0/664852695 shutdown_connections 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.031+0000 7f0b86800700 1 -- 172.21.15.59:0/664852695 wait complete. 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.032+0000 7f0b86800700 1 Processor -- start 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 -- start start 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 0x7f0b8019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b8010a7e0 0x7f0b8019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80114bc0 con 0x7f0b801051a0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80114a40 con 0x7f0b801047e0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b86800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80114d40 con 0x7f0b8010a7e0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b7f400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b77e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 0x7f0b8019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b7f400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33046/0 (socket says 172.21.15.59:33046) 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b7f400700 1 -- 172.21.15.59:0/513756791 learned_addr learned my addr 172.21.15.59:0/513756791 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.033+0000 7f0b7fe00700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b8010a7e0 0x7f0b8019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 -- 172.21.15.59:0/513756791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b8010a7e0 msgr2=0x7f0b8019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b8010a7e0 0x7f0b8019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 -- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 msgr2=0x7f0b8019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 0x7f0b8019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 -- 172.21.15.59:0/513756791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8019ff10 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b77e00700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 0x7f0b8019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7f400700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e9a0 tx=0x7f0b7000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.034+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000c9c0 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.035+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7000cb60 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.035+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70018660 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.035+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b801a0200 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.035+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a7a80 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.036+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b800692b0 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.039+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b7000cd10 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.041+0000 7f0b7cc00700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 0x7f0b640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.041+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f0b700d0660 con 0x7f0b801047e0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.042+0000 7f0b77e00700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 0x7f0b640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.042+0000 7f0b77e00700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 0x7f0b640a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b80109ca0 tx=0x7f0b68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:47.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.045+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0b7009ce70 con 0x7f0b801047e0 2024-04-02T17:11:47.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.291+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0b8002dd30 con 0x7f0b801047e0 2024-04-02T17:11:47.702 INFO:teuthology.orchestra.run.smithi059.stdout:141733920835 2024-04-02T17:11:47.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.291+0000 7f0b7cc00700 1 -- 172.21.15.59:0/513756791 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b7009c770 con 0x7f0b801047e0 2024-04-02T17:11:47.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 msgr2=0x7f0b640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 0x7f0b640a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b80109ca0 tx=0x7f0b68009310 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 msgr2=0x7f0b8019af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e9a0 tx=0x7f0b7000ecb0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 shutdown_connections 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b8010a7e0 0x7f0b8019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b801047e0 0x7f0b8019af50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0b640a3d50 0x7f0b640a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 --2- 172.21.15.59:0/513756791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b801051a0 0x7f0b8019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.294+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 >> 172.21.15.59:0/513756791 conn(0x7f0b801002d0 msgr2=0x7f0b801007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.295+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 shutdown_connections 2024-04-02T17:11:47.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:47.295+0000 7f0b86800700 1 -- 172.21.15.59:0/513756791 wait complete. 2024-04-02T17:11:48.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:48 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/513756791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T17:11:48.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:48 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3585096308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T17:11:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:48 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/513756791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T17:11:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:48 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3585096308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T17:11:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:48 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/513756791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T17:11:48.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:48 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3585096308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T17:11:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:49 smithi059 ceph-mon[31664]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.534+0000 7f5345200700 1 -- 172.21.15.59:0/2628930079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534010f0a0 msgr2=0x7f5340111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.534+0000 7f5345200700 1 --2- 172.21.15.59:0/2628930079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534010f0a0 0x7f5340111490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f533800b0a0 tx=0x7f533800b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.537+0000 7f5345200700 1 -- 172.21.15.59:0/2628930079 shutdown_connections 2024-04-02T17:11:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.537+0000 7f5345200700 1 --2- 172.21.15.59:0/2628930079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534010f0a0 0x7f5340111490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.537+0000 7f5345200700 1 --2- 172.21.15.59:0/2628930079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006a2a0 0x7f534006a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.537+0000 7f5345200700 1 --2- 172.21.15.59:0/2628930079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534006abc0 0x7f534010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.537+0000 7f5345200700 1 -- 172.21.15.59:0/2628930079 >> 172.21.15.59:0/2628930079 conn(0x7f5340078420 msgr2=0x7f5340078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.538+0000 7f5345200700 1 -- 172.21.15.59:0/2628930079 shutdown_connections 2024-04-02T17:11:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.538+0000 7f5345200700 1 -- 172.21.15.59:0/2628930079 wait complete. 2024-04-02T17:11:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.539+0000 7f5345200700 1 Processor -- start 2024-04-02T17:11:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.539+0000 7f5345200700 1 -- start start 2024-04-02T17:11:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.540+0000 7f5345200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.540+0000 7f5345200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 0x7f534010bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f533de00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f533de00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41726/0 (socket says 172.21.15.59:41726) 2024-04-02T17:11:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5337e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 0x7f534010bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5345200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 0x7f534010c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5337e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 0x7f534010bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40560/0 (socket says 172.21.15.59:40560) 2024-04-02T17:11:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5337e00700 1 -- 172.21.15.59:0/3945737553 learned_addr learned my addr 172.21.15.59:0/3945737553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114c00 con 0x7f534006a2a0 2024-04-02T17:11:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114a80 con 0x7f534010f0a0 2024-04-02T17:11:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114d80 con 0x7f534006abc0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.541+0000 7f533e800700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 0x7f534010c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 -- 172.21.15.59:0/3945737553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 msgr2=0x7f534010bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 0x7f534010bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 -- 172.21.15.59:0/3945737553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 msgr2=0x7f534010c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 0x7f534010c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53401ab940 con 0x7f534006a2a0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533e800700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 0x7f534010c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f533de00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5330006fd0 tx=0x7f533000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.542+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330015020 con 0x7f534006a2a0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.543+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5330011d10 con 0x7f534006a2a0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.543+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330010e10 con 0x7f534006a2a0 2024-04-02T17:11:49.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.543+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53401abb10 con 0x7f534006a2a0 2024-04-02T17:11:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.543+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53401abfd0 con 0x7f534006a2a0 2024-04-02T17:11:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.545+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c005330 con 0x7f534006a2a0 2024-04-02T17:11:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.546+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5330018020 con 0x7f534006a2a0 2024-04-02T17:11:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.548+0000 7f5336a00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 0x7f53200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.548+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f53300cfb60 con 0x7f534006a2a0 2024-04-02T17:11:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.548+0000 7f5337e00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 0x7f53200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:49.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.549+0000 7f5337e00700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 0x7f53200a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f534010d2d0 tx=0x7f5328009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:49.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.553+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f533001f020 con 0x7f534006a2a0 2024-04-02T17:11:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:49 smithi164 ceph-mon[36499]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:49 smithi154 ceph-mon[36479]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:11:49.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.794+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f531c0058f0 con 0x7f534006a2a0 2024-04-02T17:11:49.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.795+0000 7f5336a00700 1 -- 172.21.15.59:0/3945737553 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f533009c480 con 0x7f534006a2a0 2024-04-02T17:11:49.794 INFO:teuthology.orchestra.run.smithi059.stdout:257698037805 2024-04-02T17:11:49.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 msgr2=0x7f53200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:49.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 0x7f53200a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f534010d2d0 tx=0x7f5328009290 comp rx=0 tx=0).stop 2024-04-02T17:11:49.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 msgr2=0x7f534010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:49.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5330006fd0 tx=0x7f533000b730 comp rx=0 tx=0).stop 2024-04-02T17:11:49.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 shutdown_connections 2024-04-02T17:11:49.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534010f0a0 0x7f534010c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53200a3d50 0x7f53200a6210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534006a2a0 0x7f534010ea60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 --2- 172.21.15.59:0/3945737553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f534006abc0 0x7f534010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 >> 172.21.15.59:0/3945737553 conn(0x7f5340078420 msgr2=0x7f5340110600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.798+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 shutdown_connections 2024-04-02T17:11:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:49.799+0000 7f5345200700 1 -- 172.21.15.59:0/3945737553 wait complete. 2024-04-02T17:11:50.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:50 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3945737553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T17:11:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:50 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3945737553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T17:11:50.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:50 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3945737553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T17:11:51.036 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920835 for osd.4 2024-04-02T17:11:51.037 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:51.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776163 got 292057776167 for osd.9 2024-04-02T17:11:51.172 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:51.245 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:51 smithi059 ceph-mon[31664]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:51.245 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:51 smithi059 ceph-mon[31664]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T17:11:51.245 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-02T17:11:51.245 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T17:11:51.245 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-02T17:11:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:51 smithi164 ceph-mon[36499]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:51 smithi164 ceph-mon[36499]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T17:11:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-02T17:11:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T17:11:51.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-02T17:11:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:51 smithi154 ceph-mon[36479]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:11:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:51 smithi154 ceph-mon[36479]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T17:11:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-02T17:11:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T17:11:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-02T17:11:52.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-02T17:11:52.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T17:11:52.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:52 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-02T17:11:52.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:52 smithi059 ceph-mon[31664]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T17:11:52.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:52 smithi059 ceph-mon[31664]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T17:11:52.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037805 for osd.8 2024-04-02T17:11:52.500 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:52.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-02T17:11:52.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T17:11:52.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:52 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-02T17:11:52.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:52 smithi154 ceph-mon[36479]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T17:11:52.551 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:52 smithi154 ceph-mon[36479]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T17:11:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-02T17:11:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T17:11:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:52 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-02T17:11:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:52 smithi164 ceph-mon[36499]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T17:11:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:52 smithi164 ceph-mon[36499]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 -- 172.21.15.59:0/592557606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e010a7e0 msgr2=0x7fa1e010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 --2- 172.21.15.59:0/592557606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e010a7e0 0x7fa1e010abc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00098f0 tx=0x7fa1d0009c00 comp rx=0 tx=0).stop 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 -- 172.21.15.59:0/592557606 shutdown_connections 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 --2- 172.21.15.59:0/592557606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e01051a0 0x7fa1e01116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 --2- 172.21.15.59:0/592557606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e010a7e0 0x7fa1e010abc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 --2- 172.21.15.59:0/592557606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 0x7fa1e0104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 -- 172.21.15.59:0/592557606 >> 172.21.15.59:0/592557606 conn(0x7fa1e01002f0 msgr2=0x7fa1e0102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 -- 172.21.15.59:0/592557606 shutdown_connections 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.772+0000 7fa1e5600700 1 -- 172.21.15.59:0/592557606 wait complete. 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.773+0000 7fa1e5600700 1 Processor -- start 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.773+0000 7fa1e5600700 1 -- start start 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 0x7fa1e019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 0x7fa1e019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0114b60 con 0x7fa1e01047e0 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01149e0 con 0x7fa1e010a7e0 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1e5600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0114ce0 con 0x7fa1e01051a0 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1df400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1df400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35618/0 (socket says 172.21.15.59:35618) 2024-04-02T17:11:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1df400700 1 -- 172.21.15.59:0/2721110087 learned_addr learned my addr 172.21.15.59:0/2721110087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1dea00700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 0x7fa1e019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.774+0000 7fa1de000700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 0x7fa1e019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 -- 172.21.15.59:0/2721110087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 msgr2=0x7fa1e019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 0x7fa1e019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 -- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 msgr2=0x7fa1e019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 0x7fa1e019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 -- 172.21.15.59:0/2721110087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e01a0030 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1de000700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 0x7fa1e019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1df400700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800ef30 tx=0x7fa1d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80092e0 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8004bb0 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e01a02c0 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004280 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.775+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e01a7ba0 con 0x7fa1e010a7e0 2024-04-02T17:11:52.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.777+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e010cd70 con 0x7fa1e010a7e0 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.780+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1d801b030 con 0x7fa1e010a7e0 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.782+0000 7fa1d7400700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 0x7fa1c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.782+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa1d80cff90 con 0x7fa1e010a7e0 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.782+0000 7fa1dea00700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 0x7fa1c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.783+0000 7fa1dea00700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 0x7fa1c40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009810 tx=0x7fa1d0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:52.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:52.786+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1d8022020 con 0x7fa1e010a7e0 2024-04-02T17:11:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.027+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa1e019c380 con 0x7fa1e010a7e0 2024-04-02T17:11:53.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.027+0000 7fa1d7400700 1 -- 172.21.15.59:0/2721110087 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1d8010040 con 0x7fa1e010a7e0 2024-04-02T17:11:53.027 INFO:teuthology.orchestra.run.smithi059.stdout:55834574932 2024-04-02T17:11:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.030+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 msgr2=0x7fa1c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.030+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 0x7fa1c40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009810 tx=0x7fa1d0019040 comp rx=0 tx=0).stop 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.030+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 msgr2=0x7fa1e019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.030+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800ef30 tx=0x7fa1d800c3a0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 shutdown_connections 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e010a7e0 0x7fa1e019f8f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1c40a3c40 0x7fa1c40a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e01047e0 0x7fa1e019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 --2- 172.21.15.59:0/2721110087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e01051a0 0x7fa1e019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 >> 172.21.15.59:0/2721110087 conn(0x7fa1e01002f0 msgr2=0x7fa1e0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:53.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 shutdown_connections 2024-04-02T17:11:53.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.031+0000 7fa1e5600700 1 -- 172.21.15.59:0/2721110087 wait complete. 2024-04-02T17:11:53.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411407 for osd.2 2024-04-02T17:11:53.034 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:53 smithi059 ceph-mon[31664]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-02T17:11:53.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:53 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2721110087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T17:11:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:53 smithi164 ceph-mon[36499]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-02T17:11:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:53 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2721110087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 -- 172.21.15.59:0/985768512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90108630 msgr2=0x7f7c90108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 --2- 172.21.15.59:0/985768512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90108630 0x7f7c90108a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009990 tx=0x7f7c80009ca0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 -- 172.21.15.59:0/985768512 shutdown_connections 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 --2- 172.21.15.59:0/985768512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90103000 0x7f7c9010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 --2- 172.21.15.59:0/985768512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90108630 0x7f7c90108a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 --2- 172.21.15.59:0/985768512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c90102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 -- 172.21.15.59:0/985768512 >> 172.21.15.59:0/985768512 conn(0x7f7c90079750 msgr2=0x7f7c90079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:53.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 -- 172.21.15.59:0/985768512 shutdown_connections 2024-04-02T17:11:53.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.610+0000 7f7c94e00700 1 -- 172.21.15.59:0/985768512 wait complete. 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.611+0000 7f7c94e00700 1 Processor -- start 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 -- start start 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 0x7f7c9019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90108630 0x7f7c9019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c901129b0 con 0x7f7c90102640 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90112830 con 0x7f7c90103000 2024-04-02T17:11:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c94e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90112b30 con 0x7f7c90108630 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c8de00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c85400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 0x7f7c9019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c8de00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34824/0 (socket says 172.21.15.59:34824) 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.612+0000 7f7c8de00700 1 -- 172.21.15.59:0/2784447253 learned_addr learned my addr 172.21.15.59:0/2784447253 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c8e800700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90108630 0x7f7c9019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 -- 172.21.15.59:0/2784447253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90108630 msgr2=0x7f7c9019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90108630 0x7f7c9019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 -- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 msgr2=0x7f7c9019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 -- 172.21.15.59:0/2784447253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c901a0010 con 0x7f7c90103000 2024-04-02T17:11:53.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c8de00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:11:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c85400700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 0x7f7c9019b540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.613+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800cb70 con 0x7f7c90103000 2024-04-02T17:11:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.614+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7800ee90 con 0x7f7c90103000 2024-04-02T17:11:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.614+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78003dc0 con 0x7f7c90103000 2024-04-02T17:11:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.614+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c901a0300 con 0x7f7c90103000 2024-04-02T17:11:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.614+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c901a7b80 con 0x7f7c90103000 2024-04-02T17:11:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.615+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c900692b0 con 0x7f7c90103000 2024-04-02T17:11:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.617+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c78003f80 con 0x7f7c90103000 2024-04-02T17:11:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.618+0000 7f7c87400700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 0x7f7c700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.619+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7c780d0610 con 0x7f7c90103000 2024-04-02T17:11:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.619+0000 7f7c8de00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 0x7f7c700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.619+0000 7f7c8de00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 0x7f7c700a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c80006010 tx=0x7f7c8000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.620+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7c7809cfc0 con 0x7f7c90103000 2024-04-02T17:11:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:53 smithi154 ceph-mon[36479]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-02T17:11:53.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:53 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2721110087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T17:11:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.853+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7c901a7d50 con 0x7f7c90103000 2024-04-02T17:11:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.853+0000 7f7c87400700 1 -- 172.21.15.59:0/2784447253 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c7809c8c0 con 0x7f7c90103000 2024-04-02T17:11:53.853 INFO:teuthology.orchestra.run.smithi059.stdout:223338299442 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 msgr2=0x7f7c700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 0x7f7c700a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c80006010 tx=0x7f7c8000b260 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 msgr2=0x7f7c9019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 0x7f7c9019b540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800e9f0 tx=0x7f7c7800ed00 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 shutdown_connections 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c90108630 0x7f7c9019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7c700a3cc0 0x7f7c700a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c90102640 0x7f7c9019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 --2- 172.21.15.59:0/2784447253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c90103000 0x7f7c9019b540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 >> 172.21.15.59:0/2784447253 conn(0x7f7c90079750 msgr2=0x7f7c901004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 shutdown_connections 2024-04-02T17:11:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:53.856+0000 7f7c94e00700 1 -- 172.21.15.59:0/2784447253 wait complete. 2024-04-02T17:11:54.296 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574932 for osd.1 2024-04-02T17:11:54.296 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:54 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2784447253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T17:11:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:54 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2784447253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T17:11:54.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:54 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2784447253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T17:11:55.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299442 for osd.7 2024-04-02T17:11:55.270 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:55.518 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:55 smithi059 ceph-mon[31664]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:55 smithi164 ceph-mon[36499]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:55 smithi154 ceph-mon[36479]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:55.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.843+0000 7f8f58200700 1 -- 172.21.15.59:0/4288635638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5006acc0 msgr2=0x7f8f5006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:55.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.843+0000 7f8f58200700 1 --2- 172.21.15.59:0/4288635638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5006acc0 0x7f8f5006b140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f400099c0 tx=0x7f8f40009cd0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 -- 172.21.15.59:0/4288635638 shutdown_connections 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 --2- 172.21.15.59:0/4288635638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f5006b680 0x7f8f5010f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 --2- 172.21.15.59:0/4288635638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f50108620 0x7f8f50108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 --2- 172.21.15.59:0/4288635638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5006acc0 0x7f8f5006b140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 -- 172.21.15.59:0/4288635638 >> 172.21.15.59:0/4288635638 conn(0x7f8f50078460 msgr2=0x7f8f50078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 -- 172.21.15.59:0/4288635638 shutdown_connections 2024-04-02T17:11:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.844+0000 7f8f58200700 1 -- 172.21.15.59:0/4288635638 wait complete. 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.845+0000 7f8f58200700 1 Processor -- start 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.845+0000 7f8f58200700 1 -- start start 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.845+0000 7f8f58200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 0x7f8f5019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.845+0000 7f8f58200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 0x7f8f5019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.845+0000 7f8f58200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 0x7f8f5019ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f58200700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50112920 con 0x7f8f5019fac0 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f58200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501127a0 con 0x7f8f5006b680 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f58200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50112aa0 con 0x7f8f50108620 2024-04-02T17:11:55.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f56200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 0x7f8f5019ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:55.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f54e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 0x7f8f5019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f56200700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 0x7f8f5019ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34832/0 (socket says 172.21.15.59:34832) 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f56200700 1 -- 172.21.15.59:0/2159661162 learned_addr learned my addr 172.21.15.59:0/2159661162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f54e00700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 0x7f8f5019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43368/0 (socket says 172.21.15.59:43368) 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 0x7f8f5019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 -- 172.21.15.59:0/2159661162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 msgr2=0x7f8f5019b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 0x7f8f5019b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 -- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 msgr2=0x7f8f5019ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 0x7f8f5019ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.846+0000 7f8f55800700 1 -- 172.21.15.59:0/2159661162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f501a05e0 con 0x7f8f5006b680 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f55800700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 0x7f8f5019b1f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e9f0 tx=0x7f8f4400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400cb70 con 0x7f8f5006b680 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f501a08d0 con 0x7f8f5006b680 2024-04-02T17:11:55.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4400ee90 con 0x7f8f5006b680 2024-04-02T17:11:55.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44003dc0 con 0x7f8f5006b680 2024-04-02T17:11:55.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.847+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501a8100 con 0x7f8f5006b680 2024-04-02T17:11:55.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.849+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f50103d10 con 0x7f8f5006b680 2024-04-02T17:11:55.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.851+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f44003fa0 con 0x7f8f5006b680 2024-04-02T17:11:55.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.852+0000 7f8f4e000700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 0x7f8f380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.852+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8f440d0700 con 0x7f8f5006b680 2024-04-02T17:11:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.852+0000 7f8f54e00700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 0x7f8f380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.853+0000 7f8f54e00700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 0x7f8f380a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b580 tx=0x7f8f40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:55.854+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8f4409d020 con 0x7f8f5006b680 2024-04-02T17:11:56.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.099+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8f500692b0 con 0x7f8f5006b680 2024-04-02T17:11:56.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.100+0000 7f8f4e000700 1 -- 172.21.15.59:0/2159661162 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8f4409c920 con 0x7f8f5006b680 2024-04-02T17:11:56.099 INFO:teuthology.orchestra.run.smithi059.stdout:352187318304 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.102+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 msgr2=0x7f8f380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.102+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 0x7f8f380a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b580 tx=0x7f8f40005fb0 comp rx=0 tx=0).stop 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.102+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 msgr2=0x7f8f5019b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.102+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 0x7f8f5019b1f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e9f0 tx=0x7f8f4400ed00 comp rx=0 tx=0).stop 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 shutdown_connections 2024-04-02T17:11:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f380a3c40 0x7f8f380a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f5019fac0 0x7f8f5019ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f5006b680 0x7f8f5019b1f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 --2- 172.21.15.59:0/2159661162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f50108620 0x7f8f5019b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 >> 172.21.15.59:0/2159661162 conn(0x7f8f50078460 msgr2=0x7f8f5010d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 shutdown_connections 2024-04-02T17:11:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:56.103+0000 7f8f58200700 1 -- 172.21.15.59:0/2159661162 wait complete. 2024-04-02T17:11:56.870 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318299 got 352187318304 for osd.11 2024-04-02T17:11:56.870 DEBUG:teuthology.parallel:result is None 2024-04-02T17:11:56.871 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T17:11:56.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T17:11:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:57 smithi059 ceph-mon[31664]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:57 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:57 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2159661162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T17:11:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:57 smithi164 ceph-mon[36499]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:57 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:57 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2159661162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T17:11:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:11:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:57 smithi154 ceph-mon[36479]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:57 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:57 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2159661162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T17:11:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:11:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:11:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:11:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:11:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:11:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:11:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:11:59.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:11:59 smithi059 ceph-mon[31664]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:59.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:11:59 smithi164 ceph-mon[36499]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:59.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:11:59 smithi154 ceph-mon[36479]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 -- 172.21.15.59:0/3972199472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c10df40 msgr2=0x7ffa4c115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 --2- 172.21.15.59:0/3972199472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c10df40 0x7ffa4c115b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b0a0 tx=0x7ffa4400b3b0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 -- 172.21.15.59:0/3972199472 shutdown_connections 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 --2- 172.21.15.59:0/3972199472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c10df40 0x7ffa4c115b10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 --2- 172.21.15.59:0/3972199472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c108e60 0x7ffa4c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 --2- 172.21.15.59:0/3972199472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 0x7ffa4c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 -- 172.21.15.59:0/3972199472 >> 172.21.15.59:0/3972199472 conn(0x7ffa4c075dc0 msgr2=0x7ffa4c0781e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 -- 172.21.15.59:0/3972199472 shutdown_connections 2024-04-02T17:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.489+0000 7ffa52800700 1 -- 172.21.15.59:0/3972199472 wait complete. 2024-04-02T17:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.490+0000 7ffa52800700 1 Processor -- start 2024-04-02T17:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 -- start start 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 0x7ffa4c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c10df40 0x7ffa4c1a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c118f70 con 0x7ffa4c109810 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c118df0 con 0x7ffa4c108e60 2024-04-02T17:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa52800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1190f0 con 0x7ffa4c10df40 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa4b400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa4aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 0x7ffa4c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa4b400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35650/0 (socket says 172.21.15.59:35650) 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa4b400700 1 -- 172.21.15.59:0/2759573098 learned_addr learned my addr 172.21.15.59:0/2759573098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.491+0000 7ffa4be00700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c10df40 0x7ffa4c1a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 -- 172.21.15.59:0/2759573098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c10df40 msgr2=0x7ffa4c1a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c10df40 0x7ffa4c1a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 -- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 msgr2=0x7ffa4c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 0x7ffa4c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 -- 172.21.15.59:0/2759573098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c1a4370 con 0x7ffa4c108e60 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa4b400700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9a0 tx=0x7ffa3c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00ca10 con 0x7ffa4c108e60 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa3c00cbb0 con 0x7ffa4c108e60 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.492+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c1a4660 con 0x7ffa4c108e60 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.493+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c018680 con 0x7ffa4c108e60 2024-04-02T17:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.493+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1abee0 con 0x7ffa4c108e60 2024-04-02T17:11:59.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.494+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c0692b0 con 0x7ffa4c108e60 2024-04-02T17:11:59.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.496+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffa3c010430 con 0x7ffa4c108e60 2024-04-02T17:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.497+0000 7ffa43e00700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 0x7ffa300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.497+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ffa3c0d0aa0 con 0x7ffa4c108e60 2024-04-02T17:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.498+0000 7ffa4aa00700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 0x7ffa300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.498+0000 7ffa4aa00700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 0x7ffa300a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c107e90 tx=0x7ffa380073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.499+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffa3c09d3c0 con 0x7ffa4c108e60 2024-04-02T17:11:59.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.727+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa4c076f50 con 0x7ffa300a3cc0 2024-04-02T17:11:59.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.737+0000 7ffa43e00700 1 -- 172.21.15.59:0/2759573098 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258922 (secure 0 0 0) 0x7ffa4c076f50 con 0x7ffa300a3cc0 2024-04-02T17:11:59.742 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:11:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.742+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 msgr2=0x7ffa300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.742+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 0x7ffa300a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c107e90 tx=0x7ffa380073d0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 msgr2=0x7ffa4c19f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9a0 tx=0x7ffa3c00ecb0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 shutdown_connections 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa4c10df40 0x7ffa4c1a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa4c108e60 0x7ffa4c19f360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffa300a3cc0 0x7ffa300a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 --2- 172.21.15.59:0/2759573098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa4c109810 0x7ffa4c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 >> 172.21.15.59:0/2759573098 conn(0x7ffa4c075dc0 msgr2=0x7ffa4c076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 shutdown_connections 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:11:59.743+0000 7ffa52800700 1 -- 172.21.15.59:0/2759573098 wait complete. 2024-04-02T17:11:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:12:00.543 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-04-02T17:11:59.525103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660765,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":907,"num_read_kb":579,"num_write":434,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1180943,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388268,"kb_used_data":14088,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553556,"statfs":{"total":1151940427776,"available":1151542841344,"internally_reserved":0,"allocated":14426112,"data_stored":7791159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":53,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.308226"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430690+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:11:52.430690+0000","last_peered":"2024-04-02T17:11:52.430690+0000","last_clean":"2024-04-02T17:11:52.430690+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:11:52.430690+0000","last_undegraded":"2024-04-02T17:11:52.430690+0000","last_fullsized":"2024-04-02T17:11:52.430690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818086+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:11:52.818086+0000","last_peered":"2024-04-02T17:11:52.818086+0000","last_clean":"2024-04-02T17:11:52.818086+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:11:52.818086+0000","last_undegraded":"2024-04-02T17:11:52.818086+0000","last_fullsized":"2024-04-02T17:11:52.818086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.817910+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:11:52.817910+0000","last_peered":"2024-04-02T17:11:52.817910+0000","last_clean":"2024-04-02T17:11:52.817910+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:11:52.817910+0000","last_undegraded":"2024-04-02T17:11:52.817910+0000","last_fullsized":"2024-04-02T17:11:52.817910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222576+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:11:52.222576+0000","last_peered":"2024-04-02T17:11:52.222576+0000","last_clean":"2024-04-02T17:11:52.222576+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:11:52.222576+0000","last_undegraded":"2024-04-02T17:11:52.222576+0000","last_fullsized":"2024-04-02T17:11:52.222576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430624+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:11:52.430624+0000","last_peered":"2024-04-02T17:11:52.430624+0000","last_clean":"2024-04-02T17:11:52.430624+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:11:52.430624+0000","last_undegraded":"2024-04-02T17:11:52.430624+0000","last_fullsized":"2024-04-02T17:11:52.430624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225467+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:11:52.225467+0000","last_peered":"2024-04-02T17:11:52.225467+0000","last_clean":"2024-04-02T17:11:52.225467+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:11:52.225467+0000","last_undegraded":"2024-04-02T17:11:52.225467+0000","last_fullsized":"2024-04-02T17:11:52.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.432056+0000","last_change":"2024-04-02T17:11:52.432056+0000","last_active":"2024-04-02T17:11:52.432056+0000","last_peered":"2024-04-02T17:11:52.432056+0000","last_clean":"2024-04-02T17:11:52.432056+0000","last_became_active":"2024-04-02T17:11:52.431692+0000","last_became_peered":"2024-04-02T17:11:52.431692+0000","last_unstale":"2024-04-02T17:11:52.432056+0000","last_undegraded":"2024-04-02T17:11:52.432056+0000","last_fullsized":"2024-04-02T17:11:52.432056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:51.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230343+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:11:52.230343+0000","last_peered":"2024-04-02T17:11:52.230343+0000","last_clean":"2024-04-02T17:11:52.230343+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:11:52.230343+0000","last_undegraded":"2024-04-02T17:11:52.230343+0000","last_fullsized":"2024-04-02T17:11:52.230343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430135+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:11:52.430135+0000","last_peered":"2024-04-02T17:11:52.430135+0000","last_clean":"2024-04-02T17:11:52.430135+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:11:52.430135+0000","last_undegraded":"2024-04-02T17:11:52.430135+0000","last_fullsized":"2024-04-02T17:11:52.430135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225843+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:11:52.225843+0000","last_peered":"2024-04-02T17:11:52.225843+0000","last_clean":"2024-04-02T17:11:52.225843+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:11:52.225843+0000","last_undegraded":"2024-04-02T17:11:52.225843+0000","last_fullsized":"2024-04-02T17:11:52.225843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224298+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:11:52.224298+0000","last_peered":"2024-04-02T17:11:52.224298+0000","last_clean":"2024-04-02T17:11:52.224298+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:11:52.224298+0000","last_undegraded":"2024-04-02T17:11:52.224298+0000","last_fullsized":"2024-04-02T17:11:52.224298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222208+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:11:52.222208+0000","last_peered":"2024-04-02T17:11:52.222208+0000","last_clean":"2024-04-02T17:11:52.222208+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:11:52.222208+0000","last_undegraded":"2024-04-02T17:11:52.222208+0000","last_fullsized":"2024-04-02T17:11:52.222208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236269+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:11:52.236269+0000","last_peered":"2024-04-02T17:11:52.236269+0000","last_clean":"2024-04-02T17:11:52.236269+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:11:52.236269+0000","last_undegraded":"2024-04-02T17:11:52.236269+0000","last_fullsized":"2024-04-02T17:11:52.236269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818503+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:11:52.818503+0000","last_peered":"2024-04-02T17:11:52.818503+0000","last_clean":"2024-04-02T17:11:52.818503+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:11:52.818503+0000","last_undegraded":"2024-04-02T17:11:52.818503+0000","last_fullsized":"2024-04-02T17:11:52.818503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818449+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:11:52.818449+0000","last_peered":"2024-04-02T17:11:52.818449+0000","last_clean":"2024-04-02T17:11:52.818449+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:11:52.818449+0000","last_undegraded":"2024-04-02T17:11:52.818449+0000","last_fullsized":"2024-04-02T17:11:52.818449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429504+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:11:52.429504+0000","last_peered":"2024-04-02T17:11:52.429504+0000","last_clean":"2024-04-02T17:11:52.429504+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:11:52.429504+0000","last_undegraded":"2024-04-02T17:11:52.429504+0000","last_fullsized":"2024-04-02T17:11:52.429504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430735+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:11:52.430735+0000","last_peered":"2024-04-02T17:11:52.430735+0000","last_clean":"2024-04-02T17:11:52.430735+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:11:52.430735+0000","last_undegraded":"2024-04-02T17:11:52.430735+0000","last_fullsized":"2024-04-02T17:11:52.430735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222437+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:11:52.222437+0000","last_peered":"2024-04-02T17:11:52.222437+0000","last_clean":"2024-04-02T17:11:52.222437+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:11:52.222437+0000","last_undegraded":"2024-04-02T17:11:52.222437+0000","last_fullsized":"2024-04-02T17:11:52.222437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219619+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:11:52.219619+0000","last_peered":"2024-04-02T17:11:52.219619+0000","last_clean":"2024-04-02T17:11:52.219619+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:11:52.219619+0000","last_undegraded":"2024-04-02T17:11:52.219619+0000","last_fullsized":"2024-04-02T17:11:52.219619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:56.254332+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:11:56.254332+0000","last_peered":"2024-04-02T17:11:56.254332+0000","last_clean":"2024-04-02T17:11:56.254332+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:11:56.254332+0000","last_undegraded":"2024-04-02T17:11:56.254332+0000","last_fullsized":"2024-04-02T17:11:56.254332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232129+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:11:52.232129+0000","last_peered":"2024-04-02T17:11:52.232129+0000","last_clean":"2024-04-02T17:11:52.232129+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:11:52.232129+0000","last_undegraded":"2024-04-02T17:11:52.232129+0000","last_fullsized":"2024-04-02T17:11:52.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232628+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:11:52.232628+0000","last_peered":"2024-04-02T17:11:52.232628+0000","last_clean":"2024-04-02T17:11:52.232628+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:11:52.232628+0000","last_undegraded":"2024-04-02T17:11:52.232628+0000","last_fullsized":"2024-04-02T17:11:52.232628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818935+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:11:52.818935+0000","last_peered":"2024-04-02T17:11:52.818935+0000","last_clean":"2024-04-02T17:11:52.818935+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:11:52.818935+0000","last_undegraded":"2024-04-02T17:11:52.818935+0000","last_fullsized":"2024-04-02T17:11:52.818935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430291+0000","last_change":"2024-04-02T17:10:51.330798+0000","last_active":"2024-04-02T17:11:52.430291+0000","last_peered":"2024-04-02T17:11:52.430291+0000","last_clean":"2024-04-02T17:11:52.430291+0000","last_became_active":"2024-04-02T17:10:51.318708+0000","last_became_peered":"2024-04-02T17:10:51.318708+0000","last_unstale":"2024-04-02T17:11:52.430291+0000","last_undegraded":"2024-04-02T17:11:52.430291+0000","last_fullsized":"2024-04-02T17:11:52.430291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:08:25.035375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429776+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:11:52.429776+0000","last_peered":"2024-04-02T17:11:52.429776+0000","last_clean":"2024-04-02T17:11:52.429776+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:11:52.429776+0000","last_undegraded":"2024-04-02T17:11:52.429776+0000","last_fullsized":"2024-04-02T17:11:52.429776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430934+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:11:52.430934+0000","last_peered":"2024-04-02T17:11:52.430934+0000","last_clean":"2024-04-02T17:11:52.430934+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:11:52.430934+0000","last_undegraded":"2024-04-02T17:11:52.430934+0000","last_fullsized":"2024-04-02T17:11:52.430934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221978+0000","last_change":"2024-04-02T17:10:51.634458+0000","last_active":"2024-04-02T17:11:52.221978+0000","last_peered":"2024-04-02T17:11:52.221978+0000","last_clean":"2024-04-02T17:11:52.221978+0000","last_became_active":"2024-04-02T17:10:51.634064+0000","last_became_peered":"2024-04-02T17:10:51.634064+0000","last_unstale":"2024-04-02T17:11:52.221978+0000","last_undegraded":"2024-04-02T17:11:52.221978+0000","last_fullsized":"2024-04-02T17:11:52.221978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:10.010493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:54.202863+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:11:54.202863+0000","last_peered":"2024-04-02T17:11:54.202863+0000","last_clean":"2024-04-02T17:11:54.202863+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:11:54.202863+0000","last_undegraded":"2024-04-02T17:11:54.202863+0000","last_fullsized":"2024-04-02T17:11:54.202863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430521+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:11:52.430521+0000","last_peered":"2024-04-02T17:11:52.430521+0000","last_clean":"2024-04-02T17:11:52.430521+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:11:52.430521+0000","last_undegraded":"2024-04-02T17:11:52.430521+0000","last_fullsized":"2024-04-02T17:11:52.430521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225449+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:11:52.225449+0000","last_peered":"2024-04-02T17:11:52.225449+0000","last_clean":"2024-04-02T17:11:52.225449+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:11:52.225449+0000","last_undegraded":"2024-04-02T17:11:52.225449+0000","last_fullsized":"2024-04-02T17:11:52.225449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221833+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:11:52.221833+0000","last_peered":"2024-04-02T17:11:52.221833+0000","last_clean":"2024-04-02T17:11:52.221833+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:11:52.221833+0000","last_undegraded":"2024-04-02T17:11:52.221833+0000","last_fullsized":"2024-04-02T17:11:52.221833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235414+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:11:52.235414+0000","last_peered":"2024-04-02T17:11:52.235414+0000","last_clean":"2024-04-02T17:11:52.235414+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:11:52.235414+0000","last_undegraded":"2024-04-02T17:11:52.235414+0000","last_fullsized":"2024-04-02T17:11:52.235414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230214+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:11:52.230214+0000","last_peered":"2024-04-02T17:11:52.230214+0000","last_clean":"2024-04-02T17:11:52.230214+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:11:52.230214+0000","last_undegraded":"2024-04-02T17:11:52.230214+0000","last_fullsized":"2024-04-02T17:11:52.230214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221584+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:11:52.221584+0000","last_peered":"2024-04-02T17:11:52.221584+0000","last_clean":"2024-04-02T17:11:52.221584+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:11:52.221584+0000","last_undegraded":"2024-04-02T17:11:52.221584+0000","last_fullsized":"2024-04-02T17:11:52.221584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221849+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:11:52.221849+0000","last_peered":"2024-04-02T17:11:52.221849+0000","last_clean":"2024-04-02T17:11:52.221849+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:11:52.221849+0000","last_undegraded":"2024-04-02T17:11:52.221849+0000","last_fullsized":"2024-04-02T17:11:52.221849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":179,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.206937+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:11:57.206937+0000","last_peered":"2024-04-02T17:11:57.206937+0000","last_clean":"2024-04-02T17:11:57.206937+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:11:57.206937+0000","last_undegraded":"2024-04-02T17:11:57.206937+0000","last_fullsized":"2024-04-02T17:11:57.206937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430074+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:11:52.430074+0000","last_peered":"2024-04-02T17:11:52.430074+0000","last_clean":"2024-04-02T17:11:52.430074+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:11:52.430074+0000","last_undegraded":"2024-04-02T17:11:52.430074+0000","last_fullsized":"2024-04-02T17:11:52.430074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224087+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:11:52.224087+0000","last_peered":"2024-04-02T17:11:52.224087+0000","last_clean":"2024-04-02T17:11:52.224087+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:11:52.224087+0000","last_undegraded":"2024-04-02T17:11:52.224087+0000","last_fullsized":"2024-04-02T17:11:52.224087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819045+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:11:52.819045+0000","last_peered":"2024-04-02T17:11:52.819045+0000","last_clean":"2024-04-02T17:11:52.819045+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:11:52.819045+0000","last_undegraded":"2024-04-02T17:11:52.819045+0000","last_fullsized":"2024-04-02T17:11:52.819045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222125+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:11:52.222125+0000","last_peered":"2024-04-02T17:11:52.222125+0000","last_clean":"2024-04-02T17:11:52.222125+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:11:52.222125+0000","last_undegraded":"2024-04-02T17:11:52.222125+0000","last_fullsized":"2024-04-02T17:11:52.222125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429869+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:11:52.429869+0000","last_peered":"2024-04-02T17:11:52.429869+0000","last_clean":"2024-04-02T17:11:52.429869+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:11:52.429869+0000","last_undegraded":"2024-04-02T17:11:52.429869+0000","last_fullsized":"2024-04-02T17:11:52.429869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":214,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.212462+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:11:57.212462+0000","last_peered":"2024-04-02T17:11:57.212462+0000","last_clean":"2024-04-02T17:11:57.212462+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:11:57.212462+0000","last_undegraded":"2024-04-02T17:11:57.212462+0000","last_fullsized":"2024-04-02T17:11:57.212462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62145,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":140,"num_write":35,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430928+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:11:52.430928+0000","last_peered":"2024-04-02T17:11:52.430928+0000","last_clean":"2024-04-02T17:11:52.430928+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:11:52.430928+0000","last_undegraded":"2024-04-02T17:11:52.430928+0000","last_fullsized":"2024-04-02T17:11:52.430928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219482+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:11:52.219482+0000","last_peered":"2024-04-02T17:11:52.219482+0000","last_clean":"2024-04-02T17:11:52.219482+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:11:52.219482+0000","last_undegraded":"2024-04-02T17:11:52.219482+0000","last_fullsized":"2024-04-02T17:11:52.219482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235270+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:11:52.235270+0000","last_peered":"2024-04-02T17:11:52.235270+0000","last_clean":"2024-04-02T17:11:52.235270+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:11:52.235270+0000","last_undegraded":"2024-04-02T17:11:52.235270+0000","last_fullsized":"2024-04-02T17:11:52.235270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430181+0000","last_change":"2024-04-02T17:10:51.336678+0000","last_active":"2024-04-02T17:11:52.430181+0000","last_peered":"2024-04-02T17:11:52.430181+0000","last_clean":"2024-04-02T17:11:52.430181+0000","last_became_active":"2024-04-02T17:10:51.325028+0000","last_became_peered":"2024-04-02T17:10:51.325028+0000","last_unstale":"2024-04-02T17:11:52.430181+0000","last_undegraded":"2024-04-02T17:11:52.430181+0000","last_fullsized":"2024-04-02T17:11:52.430181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:54:29.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224169+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:11:52.224169+0000","last_peered":"2024-04-02T17:11:52.224169+0000","last_clean":"2024-04-02T17:11:52.224169+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:11:52.224169+0000","last_undegraded":"2024-04-02T17:11:52.224169+0000","last_fullsized":"2024-04-02T17:11:52.224169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:54.256699+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:11:54.256699+0000","last_peered":"2024-04-02T17:11:54.256699+0000","last_clean":"2024-04-02T17:11:54.256699+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:11:54.256699+0000","last_undegraded":"2024-04-02T17:11:54.256699+0000","last_fullsized":"2024-04-02T17:11:54.256699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223581+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:11:52.223581+0000","last_peered":"2024-04-02T17:11:52.223581+0000","last_clean":"2024-04-02T17:11:52.223581+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:11:52.223581+0000","last_undegraded":"2024-04-02T17:11:52.223581+0000","last_fullsized":"2024-04-02T17:11:52.223581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430414+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:11:52.430414+0000","last_peered":"2024-04-02T17:11:52.430414+0000","last_clean":"2024-04-02T17:11:52.430414+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:11:52.430414+0000","last_undegraded":"2024-04-02T17:11:52.430414+0000","last_fullsized":"2024-04-02T17:11:52.430414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222143+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:11:52.222143+0000","last_peered":"2024-04-02T17:11:52.222143+0000","last_clean":"2024-04-02T17:11:52.222143+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:11:52.222143+0000","last_undegraded":"2024-04-02T17:11:52.222143+0000","last_fullsized":"2024-04-02T17:11:52.222143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431159+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:11:52.431159+0000","last_peered":"2024-04-02T17:11:52.431159+0000","last_clean":"2024-04-02T17:11:52.431159+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:11:52.431159+0000","last_undegraded":"2024-04-02T17:11:52.431159+0000","last_fullsized":"2024-04-02T17:11:52.431159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430473+0000","last_change":"2024-04-02T17:10:51.321193+0000","last_active":"2024-04-02T17:11:52.430473+0000","last_peered":"2024-04-02T17:11:52.430473+0000","last_clean":"2024-04-02T17:11:52.430473+0000","last_became_active":"2024-04-02T17:10:51.320791+0000","last_became_peered":"2024-04-02T17:10:51.320791+0000","last_unstale":"2024-04-02T17:11:52.430473+0000","last_undegraded":"2024-04-02T17:11:52.430473+0000","last_fullsized":"2024-04-02T17:11:52.430473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:22:18.979764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222424+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:11:52.222424+0000","last_peered":"2024-04-02T17:11:52.222424+0000","last_clean":"2024-04-02T17:11:52.222424+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:11:52.222424+0000","last_undegraded":"2024-04-02T17:11:52.222424+0000","last_fullsized":"2024-04-02T17:11:52.222424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.243152+0000","last_change":"2024-04-02T17:11:52.243152+0000","last_active":"2024-04-02T17:11:52.243152+0000","last_peered":"2024-04-02T17:11:52.243152+0000","last_clean":"2024-04-02T17:11:52.243152+0000","last_became_active":"2024-04-02T17:11:52.232947+0000","last_became_peered":"2024-04-02T17:11:52.232947+0000","last_unstale":"2024-04-02T17:11:52.243152+0000","last_undegraded":"2024-04-02T17:11:52.243152+0000","last_fullsized":"2024-04-02T17:11:52.243152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:49:58.858519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:56.255104+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:11:56.255104+0000","last_peered":"2024-04-02T17:11:56.255104+0000","last_clean":"2024-04-02T17:11:56.255104+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:11:56.255104+0000","last_undegraded":"2024-04-02T17:11:56.255104+0000","last_fullsized":"2024-04-02T17:11:56.255104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225632+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:11:52.225632+0000","last_peered":"2024-04-02T17:11:52.225632+0000","last_clean":"2024-04-02T17:11:52.225632+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:11:52.225632+0000","last_undegraded":"2024-04-02T17:11:52.225632+0000","last_fullsized":"2024-04-02T17:11:52.225632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221726+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:11:52.221726+0000","last_peered":"2024-04-02T17:11:52.221726+0000","last_clean":"2024-04-02T17:11:52.221726+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:11:52.221726+0000","last_undegraded":"2024-04-02T17:11:52.221726+0000","last_fullsized":"2024-04-02T17:11:52.221726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.262223+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:11:57.262223+0000","last_peered":"2024-04-02T17:11:57.262223+0000","last_clean":"2024-04-02T17:11:57.262223+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:11:57.262223+0000","last_undegraded":"2024-04-02T17:11:57.262223+0000","last_fullsized":"2024-04-02T17:11:57.262223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222119+0000","last_change":"2024-04-02T17:10:51.633808+0000","last_active":"2024-04-02T17:11:52.222119+0000","last_peered":"2024-04-02T17:11:52.222119+0000","last_clean":"2024-04-02T17:11:52.222119+0000","last_became_active":"2024-04-02T17:10:51.633095+0000","last_became_peered":"2024-04-02T17:10:51.633095+0000","last_unstale":"2024-04-02T17:11:52.222119+0000","last_undegraded":"2024-04-02T17:11:52.222119+0000","last_fullsized":"2024-04-02T17:11:52.222119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:36:16.308694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222193+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:11:52.222193+0000","last_peered":"2024-04-02T17:11:52.222193+0000","last_clean":"2024-04-02T17:11:52.222193+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:11:52.222193+0000","last_undegraded":"2024-04-02T17:11:52.222193+0000","last_fullsized":"2024-04-02T17:11:52.222193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223736+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:11:52.223736+0000","last_peered":"2024-04-02T17:11:52.223736+0000","last_clean":"2024-04-02T17:11:52.223736+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:11:52.223736+0000","last_undegraded":"2024-04-02T17:11:52.223736+0000","last_fullsized":"2024-04-02T17:11:52.223736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235024+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:11:52.235024+0000","last_peered":"2024-04-02T17:11:52.235024+0000","last_clean":"2024-04-02T17:11:52.235024+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:11:52.235024+0000","last_undegraded":"2024-04-02T17:11:52.235024+0000","last_fullsized":"2024-04-02T17:11:52.235024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223789+0000","last_change":"2024-04-02T17:10:51.637846+0000","last_active":"2024-04-02T17:11:52.223789+0000","last_peered":"2024-04-02T17:11:52.223789+0000","last_clean":"2024-04-02T17:11:52.223789+0000","last_became_active":"2024-04-02T17:10:51.633829+0000","last_became_peered":"2024-04-02T17:10:51.633829+0000","last_unstale":"2024-04-02T17:11:52.223789+0000","last_undegraded":"2024-04-02T17:11:52.223789+0000","last_fullsized":"2024-04-02T17:11:52.223789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:10:45.177246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430966+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:11:52.430966+0000","last_peered":"2024-04-02T17:11:52.430966+0000","last_clean":"2024-04-02T17:11:52.430966+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:11:52.430966+0000","last_undegraded":"2024-04-02T17:11:52.430966+0000","last_fullsized":"2024-04-02T17:11:52.430966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237015+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:11:52.237015+0000","last_peered":"2024-04-02T17:11:52.237015+0000","last_clean":"2024-04-02T17:11:52.237015+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:11:52.237015+0000","last_undegraded":"2024-04-02T17:11:52.237015+0000","last_fullsized":"2024-04-02T17:11:52.237015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222571+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:11:52.222571+0000","last_peered":"2024-04-02T17:11:52.222571+0000","last_clean":"2024-04-02T17:11:52.222571+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:11:52.222571+0000","last_undegraded":"2024-04-02T17:11:52.222571+0000","last_fullsized":"2024-04-02T17:11:52.222571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232696+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:11:52.232696+0000","last_peered":"2024-04-02T17:11:52.232696+0000","last_clean":"2024-04-02T17:11:52.232696+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:11:52.232696+0000","last_undegraded":"2024-04-02T17:11:52.232696+0000","last_fullsized":"2024-04-02T17:11:52.232696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222361+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:11:52.222361+0000","last_peered":"2024-04-02T17:11:52.222361+0000","last_clean":"2024-04-02T17:11:52.222361+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:11:52.222361+0000","last_undegraded":"2024-04-02T17:11:52.222361+0000","last_fullsized":"2024-04-02T17:11:52.222361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818224+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:11:52.818224+0000","last_peered":"2024-04-02T17:11:52.818224+0000","last_clean":"2024-04-02T17:11:52.818224+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:11:52.818224+0000","last_undegraded":"2024-04-02T17:11:52.818224+0000","last_fullsized":"2024-04-02T17:11:52.818224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429578+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:11:52.429578+0000","last_peered":"2024-04-02T17:11:52.429578+0000","last_clean":"2024-04-02T17:11:52.429578+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:11:52.429578+0000","last_undegraded":"2024-04-02T17:11:52.429578+0000","last_fullsized":"2024-04-02T17:11:52.429578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430792+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:11:52.430792+0000","last_peered":"2024-04-02T17:11:52.430792+0000","last_clean":"2024-04-02T17:11:52.430792+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:11:52.430792+0000","last_undegraded":"2024-04-02T17:11:52.430792+0000","last_fullsized":"2024-04-02T17:11:52.430792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230739+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:11:52.230739+0000","last_peered":"2024-04-02T17:11:52.230739+0000","last_clean":"2024-04-02T17:11:52.230739+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:11:52.230739+0000","last_undegraded":"2024-04-02T17:11:52.230739+0000","last_fullsized":"2024-04-02T17:11:52.230739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430013+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:11:52.430013+0000","last_peered":"2024-04-02T17:11:52.430013+0000","last_clean":"2024-04-02T17:11:52.430013+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:11:52.430013+0000","last_undegraded":"2024-04-02T17:11:52.430013+0000","last_fullsized":"2024-04-02T17:11:52.430013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219476+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:11:52.219476+0000","last_peered":"2024-04-02T17:11:52.219476+0000","last_clean":"2024-04-02T17:11:52.219476+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:11:52.219476+0000","last_undegraded":"2024-04-02T17:11:52.219476+0000","last_fullsized":"2024-04-02T17:11:52.219476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232076+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:11:52.232076+0000","last_peered":"2024-04-02T17:11:52.232076+0000","last_clean":"2024-04-02T17:11:52.232076+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:11:52.232076+0000","last_undegraded":"2024-04-02T17:11:52.232076+0000","last_fullsized":"2024-04-02T17:11:52.232076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430689+0000","last_change":"2024-04-02T17:10:51.641235+0000","last_active":"2024-04-02T17:11:52.430689+0000","last_peered":"2024-04-02T17:11:52.430689+0000","last_clean":"2024-04-02T17:11:52.430689+0000","last_became_active":"2024-04-02T17:10:51.634715+0000","last_became_peered":"2024-04-02T17:10:51.634715+0000","last_unstale":"2024-04-02T17:11:52.430689+0000","last_undegraded":"2024-04-02T17:11:52.430689+0000","last_fullsized":"2024-04-02T17:11:52.430689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:56:38.764506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223942+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:11:52.223942+0000","last_peered":"2024-04-02T17:11:52.223942+0000","last_clean":"2024-04-02T17:11:52.223942+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:11:52.223942+0000","last_undegraded":"2024-04-02T17:11:52.223942+0000","last_fullsized":"2024-04-02T17:11:52.223942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:53.201401+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:11:53.201401+0000","last_peered":"2024-04-02T17:11:53.201401+0000","last_clean":"2024-04-02T17:11:53.201401+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:11:53.201401+0000","last_undegraded":"2024-04-02T17:11:53.201401+0000","last_fullsized":"2024-04-02T17:11:53.201401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224447+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:11:52.224447+0000","last_peered":"2024-04-02T17:11:52.224447+0000","last_clean":"2024-04-02T17:11:52.224447+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:11:52.224447+0000","last_undegraded":"2024-04-02T17:11:52.224447+0000","last_fullsized":"2024-04-02T17:11:52.224447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221789+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:11:52.221789+0000","last_peered":"2024-04-02T17:11:52.221789+0000","last_clean":"2024-04-02T17:11:52.221789+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:11:52.221789+0000","last_undegraded":"2024-04-02T17:11:52.221789+0000","last_fullsized":"2024-04-02T17:11:52.221789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224042+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:11:52.224042+0000","last_peered":"2024-04-02T17:11:52.224042+0000","last_clean":"2024-04-02T17:11:52.224042+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:11:52.224042+0000","last_undegraded":"2024-04-02T17:11:52.224042+0000","last_fullsized":"2024-04-02T17:11:52.224042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236326+0000","last_change":"2024-04-02T17:10:51.642752+0000","last_active":"2024-04-02T17:11:52.236326+0000","last_peered":"2024-04-02T17:11:52.236326+0000","last_clean":"2024-04-02T17:11:52.236326+0000","last_became_active":"2024-04-02T17:10:51.635734+0000","last_became_peered":"2024-04-02T17:10:51.635734+0000","last_unstale":"2024-04-02T17:11:52.236326+0000","last_undegraded":"2024-04-02T17:11:52.236326+0000","last_fullsized":"2024-04-02T17:11:52.236326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.358529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224085+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:11:52.224085+0000","last_peered":"2024-04-02T17:11:52.224085+0000","last_clean":"2024-04-02T17:11:52.224085+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:11:52.224085+0000","last_undegraded":"2024-04-02T17:11:52.224085+0000","last_fullsized":"2024-04-02T17:11:52.224085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430091+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:11:52.430091+0000","last_peered":"2024-04-02T17:11:52.430091+0000","last_clean":"2024-04-02T17:11:52.430091+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:11:52.430091+0000","last_undegraded":"2024-04-02T17:11:52.430091+0000","last_fullsized":"2024-04-02T17:11:52.430091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236694+0000","last_change":"2024-04-02T17:11:52.236694+0000","last_active":"2024-04-02T17:11:52.236694+0000","last_peered":"2024-04-02T17:11:52.236694+0000","last_clean":"2024-04-02T17:11:52.236694+0000","last_became_active":"2024-04-02T17:11:52.236128+0000","last_became_peered":"2024-04-02T17:11:52.236128+0000","last_unstale":"2024-04-02T17:11:52.236694+0000","last_undegraded":"2024-04-02T17:11:52.236694+0000","last_fullsized":"2024-04-02T17:11:52.236694+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:40.601150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219328+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:11:52.219328+0000","last_peered":"2024-04-02T17:11:52.219328+0000","last_clean":"2024-04-02T17:11:52.219328+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:11:52.219328+0000","last_undegraded":"2024-04-02T17:11:52.219328+0000","last_fullsized":"2024-04-02T17:11:52.219328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819635+0000","last_change":"2024-04-02T17:10:51.332185+0000","last_active":"2024-04-02T17:11:52.819635+0000","last_peered":"2024-04-02T17:11:52.819635+0000","last_clean":"2024-04-02T17:11:52.819635+0000","last_became_active":"2024-04-02T17:10:51.331547+0000","last_became_peered":"2024-04-02T17:10:51.331547+0000","last_unstale":"2024-04-02T17:11:52.819635+0000","last_undegraded":"2024-04-02T17:11:52.819635+0000","last_fullsized":"2024-04-02T17:11:52.819635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:43.766907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236789+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:11:52.236789+0000","last_peered":"2024-04-02T17:11:52.236789+0000","last_clean":"2024-04-02T17:11:52.236789+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:11:52.236789+0000","last_undegraded":"2024-04-02T17:11:52.236789+0000","last_fullsized":"2024-04-02T17:11:52.236789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430427+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:11:52.430427+0000","last_peered":"2024-04-02T17:11:52.430427+0000","last_clean":"2024-04-02T17:11:52.430427+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:11:52.430427+0000","last_undegraded":"2024-04-02T17:11:52.430427+0000","last_fullsized":"2024-04-02T17:11:52.430427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219956+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:11:52.219956+0000","last_peered":"2024-04-02T17:11:52.219956+0000","last_clean":"2024-04-02T17:11:52.219956+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:11:52.219956+0000","last_undegraded":"2024-04-02T17:11:52.219956+0000","last_fullsized":"2024-04-02T17:11:52.219956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232558+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:11:52.232558+0000","last_peered":"2024-04-02T17:11:52.232558+0000","last_clean":"2024-04-02T17:11:52.232558+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:11:52.232558+0000","last_undegraded":"2024-04-02T17:11:52.232558+0000","last_fullsized":"2024-04-02T17:11:52.232558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221654+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:11:52.221654+0000","last_peered":"2024-04-02T17:11:52.221654+0000","last_clean":"2024-04-02T17:11:52.221654+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:11:52.221654+0000","last_undegraded":"2024-04-02T17:11:52.221654+0000","last_fullsized":"2024-04-02T17:11:52.221654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230530+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:11:52.230530+0000","last_peered":"2024-04-02T17:11:52.230530+0000","last_clean":"2024-04-02T17:11:52.230530+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:11:52.230530+0000","last_undegraded":"2024-04-02T17:11:52.230530+0000","last_fullsized":"2024-04-02T17:11:52.230530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429668+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:11:52.429668+0000","last_peered":"2024-04-02T17:11:52.429668+0000","last_clean":"2024-04-02T17:11:52.429668+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:11:52.429668+0000","last_undegraded":"2024-04-02T17:11:52.429668+0000","last_fullsized":"2024-04-02T17:11:52.429668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236536+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:11:52.236536+0000","last_peered":"2024-04-02T17:11:52.236536+0000","last_clean":"2024-04-02T17:11:52.236536+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:11:52.236536+0000","last_undegraded":"2024-04-02T17:11:52.236536+0000","last_fullsized":"2024-04-02T17:11:52.236536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430124+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:11:52.430124+0000","last_peered":"2024-04-02T17:11:52.430124+0000","last_clean":"2024-04-02T17:11:52.430124+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:11:52.430124+0000","last_undegraded":"2024-04-02T17:11:52.430124+0000","last_fullsized":"2024-04-02T17:11:52.430124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70163,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":781,"num_read_kb":470,"num_write":251,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":485886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1720320,"data_stored":1138970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":485311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":485572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1078763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":484911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":547969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":486782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":549253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":484619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":485304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1642496,"data_stored":1077819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:12:00.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T17:12:01.010 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:01 smithi059 ceph-mon[31664]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 862 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:12:01.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:01 smithi059 ceph-mon[31664]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:01 smithi164 ceph-mon[36499]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 862 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:12:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:01 smithi164 ceph-mon[36499]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:01 smithi154 ceph-mon[36479]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 862 B/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-02T17:12:01.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:01 smithi154 ceph-mon[36479]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 -- 172.21.15.59:0/2687493335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b34108630 msgr2=0x7f7b34108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2687493335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b34108630 0x7f7b34108a10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7b240099c0 tx=0x7f7b24009cd0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 -- 172.21.15.59:0/2687493335 shutdown_connections 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2687493335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b3406b4f0 0x7f7b3410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2687493335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b34108630 0x7f7b34108a10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2687493335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3406afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 -- 172.21.15.59:0/2687493335 >> 172.21.15.59:0/2687493335 conn(0x7f7b34077540 msgr2=0x7f7b34077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:02.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 -- 172.21.15.59:0/2687493335 shutdown_connections 2024-04-02T17:12:02.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.872+0000 7f7b39a00700 1 -- 172.21.15.59:0/2687493335 wait complete. 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.873+0000 7f7b39a00700 1 Processor -- start 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.873+0000 7f7b39a00700 1 -- start start 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 0x7f7b3419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b34108630 0x7f7b3419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341128d0 con 0x7f7b3406b4f0 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34112750 con 0x7f7b34108630 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b39a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34112a50 con 0x7f7b3406ab30 2024-04-02T17:12:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b2a000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 0x7f7b3419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b32a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b32a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36292/0 (socket says 172.21.15.59:36292) 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.874+0000 7f7b32a00700 1 -- 172.21.15.59:0/2100283729 learned_addr learned my addr 172.21.15.59:0/2100283729 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 -- 172.21.15.59:0/2100283729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b34108630 msgr2=0x7f7b3419f840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b34108630 0x7f7b3419f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 -- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 msgr2=0x7f7b3419b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 0x7f7b3419b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 -- 172.21.15.59:0/2100283729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3419ff80 con 0x7f7b3406ab30 2024-04-02T17:12:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b2a000700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 0x7f7b3419b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b32a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b2400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.875+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24003680 con 0x7f7b3406ab30 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.876+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b240050d0 con 0x7f7b3406ab30 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.876+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b341a0210 con 0x7f7b3406ab30 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.876+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24003d60 con 0x7f7b3406ab30 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.876+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341a7af0 con 0x7f7b3406ab30 2024-04-02T17:12:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.877+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b34103d20 con 0x7f7b3406ab30 2024-04-02T17:12:02.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.879+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b24005280 con 0x7f7b3406ab30 2024-04-02T17:12:02.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.881+0000 7f7b2be00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 0x7f7b180a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:02.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.881+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7b240d1360 con 0x7f7b3406ab30 2024-04-02T17:12:02.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.881+0000 7f7b2a000700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 0x7f7b180a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:02.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.882+0000 7f7b2a000700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 0x7f7b180a62a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b341035d0 tx=0x7f7b1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:02.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:02.883+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7b2405df20 con 0x7f7b3406ab30 2024-04-02T17:12:03.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.120+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b3410de90 con 0x7f7b180a3de0 2024-04-02T17:12:03.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.143+0000 7f7b2be00700 1 -- 172.21.15.59:0/2100283729 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258900 (secure 0 0 0) 0x7f7b3410de90 con 0x7f7b180a3de0 2024-04-02T17:12:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 msgr2=0x7f7b180a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 0x7f7b180a62a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b341035d0 tx=0x7f7b1c009260 comp rx=0 tx=0).stop 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 msgr2=0x7f7b3419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b2400f690 comp rx=0 tx=0).stop 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 shutdown_connections 2024-04-02T17:12:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b34108630 0x7f7b3419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b3406ab30 0x7f7b3419af70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7b180a3de0 0x7f7b180a62a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 --2- 172.21.15.59:0/2100283729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b3406b4f0 0x7f7b3419b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.149+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 >> 172.21.15.59:0/2100283729 conn(0x7f7b34077540 msgr2=0x7f7b3410d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.150+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 shutdown_connections 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:03.150+0000 7f7b39a00700 1 -- 172.21.15.59:0/2100283729 wait complete. 2024-04-02T17:12:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:12:03.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:03 smithi059 ceph-mon[31664]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-02T17:12:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:03 smithi164 ceph-mon[36499]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-02T17:12:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:03 smithi154 ceph-mon[36479]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-02T17:12:04.025 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-04-02T17:12:01.525964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660765,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":922,"num_read_kb":579,"num_write":434,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1180943,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388268,"kb_used_data":14088,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553556,"statfs":{"total":1151940427776,"available":1151542841344,"internally_reserved":0,"allocated":14426112,"data_stored":7791159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003244"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430690+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:11:52.430690+0000","last_peered":"2024-04-02T17:11:52.430690+0000","last_clean":"2024-04-02T17:11:52.430690+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:11:52.430690+0000","last_undegraded":"2024-04-02T17:11:52.430690+0000","last_fullsized":"2024-04-02T17:11:52.430690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818086+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:11:52.818086+0000","last_peered":"2024-04-02T17:11:52.818086+0000","last_clean":"2024-04-02T17:11:52.818086+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:11:52.818086+0000","last_undegraded":"2024-04-02T17:11:52.818086+0000","last_fullsized":"2024-04-02T17:11:52.818086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.817910+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:11:52.817910+0000","last_peered":"2024-04-02T17:11:52.817910+0000","last_clean":"2024-04-02T17:11:52.817910+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:11:52.817910+0000","last_undegraded":"2024-04-02T17:11:52.817910+0000","last_fullsized":"2024-04-02T17:11:52.817910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222576+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:11:52.222576+0000","last_peered":"2024-04-02T17:11:52.222576+0000","last_clean":"2024-04-02T17:11:52.222576+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:11:52.222576+0000","last_undegraded":"2024-04-02T17:11:52.222576+0000","last_fullsized":"2024-04-02T17:11:52.222576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430624+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:11:52.430624+0000","last_peered":"2024-04-02T17:11:52.430624+0000","last_clean":"2024-04-02T17:11:52.430624+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:11:52.430624+0000","last_undegraded":"2024-04-02T17:11:52.430624+0000","last_fullsized":"2024-04-02T17:11:52.430624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225467+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:11:52.225467+0000","last_peered":"2024-04-02T17:11:52.225467+0000","last_clean":"2024-04-02T17:11:52.225467+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:11:52.225467+0000","last_undegraded":"2024-04-02T17:11:52.225467+0000","last_fullsized":"2024-04-02T17:11:52.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.432056+0000","last_change":"2024-04-02T17:11:52.432056+0000","last_active":"2024-04-02T17:11:52.432056+0000","last_peered":"2024-04-02T17:11:52.432056+0000","last_clean":"2024-04-02T17:11:52.432056+0000","last_became_active":"2024-04-02T17:11:52.431692+0000","last_became_peered":"2024-04-02T17:11:52.431692+0000","last_unstale":"2024-04-02T17:11:52.432056+0000","last_undegraded":"2024-04-02T17:11:52.432056+0000","last_fullsized":"2024-04-02T17:11:52.432056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:51.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230343+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:11:52.230343+0000","last_peered":"2024-04-02T17:11:52.230343+0000","last_clean":"2024-04-02T17:11:52.230343+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:11:52.230343+0000","last_undegraded":"2024-04-02T17:11:52.230343+0000","last_fullsized":"2024-04-02T17:11:52.230343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430135+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:11:52.430135+0000","last_peered":"2024-04-02T17:11:52.430135+0000","last_clean":"2024-04-02T17:11:52.430135+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:11:52.430135+0000","last_undegraded":"2024-04-02T17:11:52.430135+0000","last_fullsized":"2024-04-02T17:11:52.430135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225843+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:11:52.225843+0000","last_peered":"2024-04-02T17:11:52.225843+0000","last_clean":"2024-04-02T17:11:52.225843+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:11:52.225843+0000","last_undegraded":"2024-04-02T17:11:52.225843+0000","last_fullsized":"2024-04-02T17:11:52.225843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224298+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:11:52.224298+0000","last_peered":"2024-04-02T17:11:52.224298+0000","last_clean":"2024-04-02T17:11:52.224298+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:11:52.224298+0000","last_undegraded":"2024-04-02T17:11:52.224298+0000","last_fullsized":"2024-04-02T17:11:52.224298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222208+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:11:52.222208+0000","last_peered":"2024-04-02T17:11:52.222208+0000","last_clean":"2024-04-02T17:11:52.222208+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:11:52.222208+0000","last_undegraded":"2024-04-02T17:11:52.222208+0000","last_fullsized":"2024-04-02T17:11:52.222208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236269+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:11:52.236269+0000","last_peered":"2024-04-02T17:11:52.236269+0000","last_clean":"2024-04-02T17:11:52.236269+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:11:52.236269+0000","last_undegraded":"2024-04-02T17:11:52.236269+0000","last_fullsized":"2024-04-02T17:11:52.236269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818503+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:11:52.818503+0000","last_peered":"2024-04-02T17:11:52.818503+0000","last_clean":"2024-04-02T17:11:52.818503+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:11:52.818503+0000","last_undegraded":"2024-04-02T17:11:52.818503+0000","last_fullsized":"2024-04-02T17:11:52.818503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818449+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:11:52.818449+0000","last_peered":"2024-04-02T17:11:52.818449+0000","last_clean":"2024-04-02T17:11:52.818449+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:11:52.818449+0000","last_undegraded":"2024-04-02T17:11:52.818449+0000","last_fullsized":"2024-04-02T17:11:52.818449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429504+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:11:52.429504+0000","last_peered":"2024-04-02T17:11:52.429504+0000","last_clean":"2024-04-02T17:11:52.429504+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:11:52.429504+0000","last_undegraded":"2024-04-02T17:11:52.429504+0000","last_fullsized":"2024-04-02T17:11:52.429504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430735+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:11:52.430735+0000","last_peered":"2024-04-02T17:11:52.430735+0000","last_clean":"2024-04-02T17:11:52.430735+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:11:52.430735+0000","last_undegraded":"2024-04-02T17:11:52.430735+0000","last_fullsized":"2024-04-02T17:11:52.430735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222437+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:11:52.222437+0000","last_peered":"2024-04-02T17:11:52.222437+0000","last_clean":"2024-04-02T17:11:52.222437+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:11:52.222437+0000","last_undegraded":"2024-04-02T17:11:52.222437+0000","last_fullsized":"2024-04-02T17:11:52.222437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219619+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:11:52.219619+0000","last_peered":"2024-04-02T17:11:52.219619+0000","last_clean":"2024-04-02T17:11:52.219619+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:11:52.219619+0000","last_undegraded":"2024-04-02T17:11:52.219619+0000","last_fullsized":"2024-04-02T17:11:52.219619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:56.254332+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:11:56.254332+0000","last_peered":"2024-04-02T17:11:56.254332+0000","last_clean":"2024-04-02T17:11:56.254332+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:11:56.254332+0000","last_undegraded":"2024-04-02T17:11:56.254332+0000","last_fullsized":"2024-04-02T17:11:56.254332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232129+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:11:52.232129+0000","last_peered":"2024-04-02T17:11:52.232129+0000","last_clean":"2024-04-02T17:11:52.232129+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:11:52.232129+0000","last_undegraded":"2024-04-02T17:11:52.232129+0000","last_fullsized":"2024-04-02T17:11:52.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232628+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:11:52.232628+0000","last_peered":"2024-04-02T17:11:52.232628+0000","last_clean":"2024-04-02T17:11:52.232628+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:11:52.232628+0000","last_undegraded":"2024-04-02T17:11:52.232628+0000","last_fullsized":"2024-04-02T17:11:52.232628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818935+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:11:52.818935+0000","last_peered":"2024-04-02T17:11:52.818935+0000","last_clean":"2024-04-02T17:11:52.818935+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:11:52.818935+0000","last_undegraded":"2024-04-02T17:11:52.818935+0000","last_fullsized":"2024-04-02T17:11:52.818935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430291+0000","last_change":"2024-04-02T17:10:51.330798+0000","last_active":"2024-04-02T17:11:52.430291+0000","last_peered":"2024-04-02T17:11:52.430291+0000","last_clean":"2024-04-02T17:11:52.430291+0000","last_became_active":"2024-04-02T17:10:51.318708+0000","last_became_peered":"2024-04-02T17:10:51.318708+0000","last_unstale":"2024-04-02T17:11:52.430291+0000","last_undegraded":"2024-04-02T17:11:52.430291+0000","last_fullsized":"2024-04-02T17:11:52.430291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:08:25.035375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429776+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:11:52.429776+0000","last_peered":"2024-04-02T17:11:52.429776+0000","last_clean":"2024-04-02T17:11:52.429776+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:11:52.429776+0000","last_undegraded":"2024-04-02T17:11:52.429776+0000","last_fullsized":"2024-04-02T17:11:52.429776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430934+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:11:52.430934+0000","last_peered":"2024-04-02T17:11:52.430934+0000","last_clean":"2024-04-02T17:11:52.430934+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:11:52.430934+0000","last_undegraded":"2024-04-02T17:11:52.430934+0000","last_fullsized":"2024-04-02T17:11:52.430934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221978+0000","last_change":"2024-04-02T17:10:51.634458+0000","last_active":"2024-04-02T17:11:52.221978+0000","last_peered":"2024-04-02T17:11:52.221978+0000","last_clean":"2024-04-02T17:11:52.221978+0000","last_became_active":"2024-04-02T17:10:51.634064+0000","last_became_peered":"2024-04-02T17:10:51.634064+0000","last_unstale":"2024-04-02T17:11:52.221978+0000","last_undegraded":"2024-04-02T17:11:52.221978+0000","last_fullsized":"2024-04-02T17:11:52.221978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:10.010493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:59.210945+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:11:59.210945+0000","last_peered":"2024-04-02T17:11:59.210945+0000","last_clean":"2024-04-02T17:11:59.210945+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:11:59.210945+0000","last_undegraded":"2024-04-02T17:11:59.210945+0000","last_fullsized":"2024-04-02T17:11:59.210945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430521+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:11:52.430521+0000","last_peered":"2024-04-02T17:11:52.430521+0000","last_clean":"2024-04-02T17:11:52.430521+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:11:52.430521+0000","last_undegraded":"2024-04-02T17:11:52.430521+0000","last_fullsized":"2024-04-02T17:11:52.430521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225449+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:11:52.225449+0000","last_peered":"2024-04-02T17:11:52.225449+0000","last_clean":"2024-04-02T17:11:52.225449+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:11:52.225449+0000","last_undegraded":"2024-04-02T17:11:52.225449+0000","last_fullsized":"2024-04-02T17:11:52.225449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221833+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:11:52.221833+0000","last_peered":"2024-04-02T17:11:52.221833+0000","last_clean":"2024-04-02T17:11:52.221833+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:11:52.221833+0000","last_undegraded":"2024-04-02T17:11:52.221833+0000","last_fullsized":"2024-04-02T17:11:52.221833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235414+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:11:52.235414+0000","last_peered":"2024-04-02T17:11:52.235414+0000","last_clean":"2024-04-02T17:11:52.235414+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:11:52.235414+0000","last_undegraded":"2024-04-02T17:11:52.235414+0000","last_fullsized":"2024-04-02T17:11:52.235414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230214+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:11:52.230214+0000","last_peered":"2024-04-02T17:11:52.230214+0000","last_clean":"2024-04-02T17:11:52.230214+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:11:52.230214+0000","last_undegraded":"2024-04-02T17:11:52.230214+0000","last_fullsized":"2024-04-02T17:11:52.230214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221584+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:11:52.221584+0000","last_peered":"2024-04-02T17:11:52.221584+0000","last_clean":"2024-04-02T17:11:52.221584+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:11:52.221584+0000","last_undegraded":"2024-04-02T17:11:52.221584+0000","last_fullsized":"2024-04-02T17:11:52.221584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221849+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:11:52.221849+0000","last_peered":"2024-04-02T17:11:52.221849+0000","last_clean":"2024-04-02T17:11:52.221849+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:11:52.221849+0000","last_undegraded":"2024-04-02T17:11:52.221849+0000","last_fullsized":"2024-04-02T17:11:52.221849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":179,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.206937+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:11:57.206937+0000","last_peered":"2024-04-02T17:11:57.206937+0000","last_clean":"2024-04-02T17:11:57.206937+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:11:57.206937+0000","last_undegraded":"2024-04-02T17:11:57.206937+0000","last_fullsized":"2024-04-02T17:11:57.206937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430074+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:11:52.430074+0000","last_peered":"2024-04-02T17:11:52.430074+0000","last_clean":"2024-04-02T17:11:52.430074+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:11:52.430074+0000","last_undegraded":"2024-04-02T17:11:52.430074+0000","last_fullsized":"2024-04-02T17:11:52.430074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224087+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:11:52.224087+0000","last_peered":"2024-04-02T17:11:52.224087+0000","last_clean":"2024-04-02T17:11:52.224087+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:11:52.224087+0000","last_undegraded":"2024-04-02T17:11:52.224087+0000","last_fullsized":"2024-04-02T17:11:52.224087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819045+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:11:52.819045+0000","last_peered":"2024-04-02T17:11:52.819045+0000","last_clean":"2024-04-02T17:11:52.819045+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:11:52.819045+0000","last_undegraded":"2024-04-02T17:11:52.819045+0000","last_fullsized":"2024-04-02T17:11:52.819045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222125+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:11:52.222125+0000","last_peered":"2024-04-02T17:11:52.222125+0000","last_clean":"2024-04-02T17:11:52.222125+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:11:52.222125+0000","last_undegraded":"2024-04-02T17:11:52.222125+0000","last_fullsized":"2024-04-02T17:11:52.222125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429869+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:11:52.429869+0000","last_peered":"2024-04-02T17:11:52.429869+0000","last_clean":"2024-04-02T17:11:52.429869+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:11:52.429869+0000","last_undegraded":"2024-04-02T17:11:52.429869+0000","last_fullsized":"2024-04-02T17:11:52.429869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":214,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.212462+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:11:57.212462+0000","last_peered":"2024-04-02T17:11:57.212462+0000","last_clean":"2024-04-02T17:11:57.212462+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:11:57.212462+0000","last_undegraded":"2024-04-02T17:11:57.212462+0000","last_fullsized":"2024-04-02T17:11:57.212462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62145,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":140,"num_write":35,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430928+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:11:52.430928+0000","last_peered":"2024-04-02T17:11:52.430928+0000","last_clean":"2024-04-02T17:11:52.430928+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:11:52.430928+0000","last_undegraded":"2024-04-02T17:11:52.430928+0000","last_fullsized":"2024-04-02T17:11:52.430928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219482+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:11:52.219482+0000","last_peered":"2024-04-02T17:11:52.219482+0000","last_clean":"2024-04-02T17:11:52.219482+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:11:52.219482+0000","last_undegraded":"2024-04-02T17:11:52.219482+0000","last_fullsized":"2024-04-02T17:11:52.219482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235270+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:11:52.235270+0000","last_peered":"2024-04-02T17:11:52.235270+0000","last_clean":"2024-04-02T17:11:52.235270+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:11:52.235270+0000","last_undegraded":"2024-04-02T17:11:52.235270+0000","last_fullsized":"2024-04-02T17:11:52.235270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430181+0000","last_change":"2024-04-02T17:10:51.336678+0000","last_active":"2024-04-02T17:11:52.430181+0000","last_peered":"2024-04-02T17:11:52.430181+0000","last_clean":"2024-04-02T17:11:52.430181+0000","last_became_active":"2024-04-02T17:10:51.325028+0000","last_became_peered":"2024-04-02T17:10:51.325028+0000","last_unstale":"2024-04-02T17:11:52.430181+0000","last_undegraded":"2024-04-02T17:11:52.430181+0000","last_fullsized":"2024-04-02T17:11:52.430181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:54:29.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224169+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:11:52.224169+0000","last_peered":"2024-04-02T17:11:52.224169+0000","last_clean":"2024-04-02T17:11:52.224169+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:11:52.224169+0000","last_undegraded":"2024-04-02T17:11:52.224169+0000","last_fullsized":"2024-04-02T17:11:52.224169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:59.268211+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:11:59.268211+0000","last_peered":"2024-04-02T17:11:59.268211+0000","last_clean":"2024-04-02T17:11:59.268211+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:11:59.268211+0000","last_undegraded":"2024-04-02T17:11:59.268211+0000","last_fullsized":"2024-04-02T17:11:59.268211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223581+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:11:52.223581+0000","last_peered":"2024-04-02T17:11:52.223581+0000","last_clean":"2024-04-02T17:11:52.223581+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:11:52.223581+0000","last_undegraded":"2024-04-02T17:11:52.223581+0000","last_fullsized":"2024-04-02T17:11:52.223581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430414+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:11:52.430414+0000","last_peered":"2024-04-02T17:11:52.430414+0000","last_clean":"2024-04-02T17:11:52.430414+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:11:52.430414+0000","last_undegraded":"2024-04-02T17:11:52.430414+0000","last_fullsized":"2024-04-02T17:11:52.430414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222143+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:11:52.222143+0000","last_peered":"2024-04-02T17:11:52.222143+0000","last_clean":"2024-04-02T17:11:52.222143+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:11:52.222143+0000","last_undegraded":"2024-04-02T17:11:52.222143+0000","last_fullsized":"2024-04-02T17:11:52.222143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431159+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:11:52.431159+0000","last_peered":"2024-04-02T17:11:52.431159+0000","last_clean":"2024-04-02T17:11:52.431159+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:11:52.431159+0000","last_undegraded":"2024-04-02T17:11:52.431159+0000","last_fullsized":"2024-04-02T17:11:52.431159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430473+0000","last_change":"2024-04-02T17:10:51.321193+0000","last_active":"2024-04-02T17:11:52.430473+0000","last_peered":"2024-04-02T17:11:52.430473+0000","last_clean":"2024-04-02T17:11:52.430473+0000","last_became_active":"2024-04-02T17:10:51.320791+0000","last_became_peered":"2024-04-02T17:10:51.320791+0000","last_unstale":"2024-04-02T17:11:52.430473+0000","last_undegraded":"2024-04-02T17:11:52.430473+0000","last_fullsized":"2024-04-02T17:11:52.430473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:22:18.979764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222424+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:11:52.222424+0000","last_peered":"2024-04-02T17:11:52.222424+0000","last_clean":"2024-04-02T17:11:52.222424+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:11:52.222424+0000","last_undegraded":"2024-04-02T17:11:52.222424+0000","last_fullsized":"2024-04-02T17:11:52.222424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.243152+0000","last_change":"2024-04-02T17:11:52.243152+0000","last_active":"2024-04-02T17:11:52.243152+0000","last_peered":"2024-04-02T17:11:52.243152+0000","last_clean":"2024-04-02T17:11:52.243152+0000","last_became_active":"2024-04-02T17:11:52.232947+0000","last_became_peered":"2024-04-02T17:11:52.232947+0000","last_unstale":"2024-04-02T17:11:52.243152+0000","last_undegraded":"2024-04-02T17:11:52.243152+0000","last_fullsized":"2024-04-02T17:11:52.243152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:49:58.858519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:56.255104+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:11:56.255104+0000","last_peered":"2024-04-02T17:11:56.255104+0000","last_clean":"2024-04-02T17:11:56.255104+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:11:56.255104+0000","last_undegraded":"2024-04-02T17:11:56.255104+0000","last_fullsized":"2024-04-02T17:11:56.255104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225632+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:11:52.225632+0000","last_peered":"2024-04-02T17:11:52.225632+0000","last_clean":"2024-04-02T17:11:52.225632+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:11:52.225632+0000","last_undegraded":"2024-04-02T17:11:52.225632+0000","last_fullsized":"2024-04-02T17:11:52.225632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221726+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:11:52.221726+0000","last_peered":"2024-04-02T17:11:52.221726+0000","last_clean":"2024-04-02T17:11:52.221726+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:11:52.221726+0000","last_undegraded":"2024-04-02T17:11:52.221726+0000","last_fullsized":"2024-04-02T17:11:52.221726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:57.262223+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:11:57.262223+0000","last_peered":"2024-04-02T17:11:57.262223+0000","last_clean":"2024-04-02T17:11:57.262223+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:11:57.262223+0000","last_undegraded":"2024-04-02T17:11:57.262223+0000","last_fullsized":"2024-04-02T17:11:57.262223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222119+0000","last_change":"2024-04-02T17:10:51.633808+0000","last_active":"2024-04-02T17:11:52.222119+0000","last_peered":"2024-04-02T17:11:52.222119+0000","last_clean":"2024-04-02T17:11:52.222119+0000","last_became_active":"2024-04-02T17:10:51.633095+0000","last_became_peered":"2024-04-02T17:10:51.633095+0000","last_unstale":"2024-04-02T17:11:52.222119+0000","last_undegraded":"2024-04-02T17:11:52.222119+0000","last_fullsized":"2024-04-02T17:11:52.222119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:36:16.308694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222193+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:11:52.222193+0000","last_peered":"2024-04-02T17:11:52.222193+0000","last_clean":"2024-04-02T17:11:52.222193+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:11:52.222193+0000","last_undegraded":"2024-04-02T17:11:52.222193+0000","last_fullsized":"2024-04-02T17:11:52.222193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223736+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:11:52.223736+0000","last_peered":"2024-04-02T17:11:52.223736+0000","last_clean":"2024-04-02T17:11:52.223736+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:11:52.223736+0000","last_undegraded":"2024-04-02T17:11:52.223736+0000","last_fullsized":"2024-04-02T17:11:52.223736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235024+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:11:52.235024+0000","last_peered":"2024-04-02T17:11:52.235024+0000","last_clean":"2024-04-02T17:11:52.235024+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:11:52.235024+0000","last_undegraded":"2024-04-02T17:11:52.235024+0000","last_fullsized":"2024-04-02T17:11:52.235024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223789+0000","last_change":"2024-04-02T17:10:51.637846+0000","last_active":"2024-04-02T17:11:52.223789+0000","last_peered":"2024-04-02T17:11:52.223789+0000","last_clean":"2024-04-02T17:11:52.223789+0000","last_became_active":"2024-04-02T17:10:51.633829+0000","last_became_peered":"2024-04-02T17:10:51.633829+0000","last_unstale":"2024-04-02T17:11:52.223789+0000","last_undegraded":"2024-04-02T17:11:52.223789+0000","last_fullsized":"2024-04-02T17:11:52.223789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:10:45.177246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430966+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:11:52.430966+0000","last_peered":"2024-04-02T17:11:52.430966+0000","last_clean":"2024-04-02T17:11:52.430966+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:11:52.430966+0000","last_undegraded":"2024-04-02T17:11:52.430966+0000","last_fullsized":"2024-04-02T17:11:52.430966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237015+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:11:52.237015+0000","last_peered":"2024-04-02T17:11:52.237015+0000","last_clean":"2024-04-02T17:11:52.237015+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:11:52.237015+0000","last_undegraded":"2024-04-02T17:11:52.237015+0000","last_fullsized":"2024-04-02T17:11:52.237015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222571+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:11:52.222571+0000","last_peered":"2024-04-02T17:11:52.222571+0000","last_clean":"2024-04-02T17:11:52.222571+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:11:52.222571+0000","last_undegraded":"2024-04-02T17:11:52.222571+0000","last_fullsized":"2024-04-02T17:11:52.222571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232696+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:11:52.232696+0000","last_peered":"2024-04-02T17:11:52.232696+0000","last_clean":"2024-04-02T17:11:52.232696+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:11:52.232696+0000","last_undegraded":"2024-04-02T17:11:52.232696+0000","last_fullsized":"2024-04-02T17:11:52.232696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222361+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:11:52.222361+0000","last_peered":"2024-04-02T17:11:52.222361+0000","last_clean":"2024-04-02T17:11:52.222361+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:11:52.222361+0000","last_undegraded":"2024-04-02T17:11:52.222361+0000","last_fullsized":"2024-04-02T17:11:52.222361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818224+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:11:52.818224+0000","last_peered":"2024-04-02T17:11:52.818224+0000","last_clean":"2024-04-02T17:11:52.818224+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:11:52.818224+0000","last_undegraded":"2024-04-02T17:11:52.818224+0000","last_fullsized":"2024-04-02T17:11:52.818224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429578+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:11:52.429578+0000","last_peered":"2024-04-02T17:11:52.429578+0000","last_clean":"2024-04-02T17:11:52.429578+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:11:52.429578+0000","last_undegraded":"2024-04-02T17:11:52.429578+0000","last_fullsized":"2024-04-02T17:11:52.429578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430792+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:11:52.430792+0000","last_peered":"2024-04-02T17:11:52.430792+0000","last_clean":"2024-04-02T17:11:52.430792+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:11:52.430792+0000","last_undegraded":"2024-04-02T17:11:52.430792+0000","last_fullsized":"2024-04-02T17:11:52.430792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230739+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:11:52.230739+0000","last_peered":"2024-04-02T17:11:52.230739+0000","last_clean":"2024-04-02T17:11:52.230739+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:11:52.230739+0000","last_undegraded":"2024-04-02T17:11:52.230739+0000","last_fullsized":"2024-04-02T17:11:52.230739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430013+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:11:52.430013+0000","last_peered":"2024-04-02T17:11:52.430013+0000","last_clean":"2024-04-02T17:11:52.430013+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:11:52.430013+0000","last_undegraded":"2024-04-02T17:11:52.430013+0000","last_fullsized":"2024-04-02T17:11:52.430013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219476+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:11:52.219476+0000","last_peered":"2024-04-02T17:11:52.219476+0000","last_clean":"2024-04-02T17:11:52.219476+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:11:52.219476+0000","last_undegraded":"2024-04-02T17:11:52.219476+0000","last_fullsized":"2024-04-02T17:11:52.219476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232076+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:11:52.232076+0000","last_peered":"2024-04-02T17:11:52.232076+0000","last_clean":"2024-04-02T17:11:52.232076+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:11:52.232076+0000","last_undegraded":"2024-04-02T17:11:52.232076+0000","last_fullsized":"2024-04-02T17:11:52.232076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430689+0000","last_change":"2024-04-02T17:10:51.641235+0000","last_active":"2024-04-02T17:11:52.430689+0000","last_peered":"2024-04-02T17:11:52.430689+0000","last_clean":"2024-04-02T17:11:52.430689+0000","last_became_active":"2024-04-02T17:10:51.634715+0000","last_became_peered":"2024-04-02T17:10:51.634715+0000","last_unstale":"2024-04-02T17:11:52.430689+0000","last_undegraded":"2024-04-02T17:11:52.430689+0000","last_fullsized":"2024-04-02T17:11:52.430689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:56:38.764506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223942+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:11:52.223942+0000","last_peered":"2024-04-02T17:11:52.223942+0000","last_clean":"2024-04-02T17:11:52.223942+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:11:52.223942+0000","last_undegraded":"2024-04-02T17:11:52.223942+0000","last_fullsized":"2024-04-02T17:11:52.223942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:58.209217+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:11:58.209217+0000","last_peered":"2024-04-02T17:11:58.209217+0000","last_clean":"2024-04-02T17:11:58.209217+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:11:58.209217+0000","last_undegraded":"2024-04-02T17:11:58.209217+0000","last_fullsized":"2024-04-02T17:11:58.209217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224447+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:11:52.224447+0000","last_peered":"2024-04-02T17:11:52.224447+0000","last_clean":"2024-04-02T17:11:52.224447+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:11:52.224447+0000","last_undegraded":"2024-04-02T17:11:52.224447+0000","last_fullsized":"2024-04-02T17:11:52.224447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221789+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:11:52.221789+0000","last_peered":"2024-04-02T17:11:52.221789+0000","last_clean":"2024-04-02T17:11:52.221789+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:11:52.221789+0000","last_undegraded":"2024-04-02T17:11:52.221789+0000","last_fullsized":"2024-04-02T17:11:52.221789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224042+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:11:52.224042+0000","last_peered":"2024-04-02T17:11:52.224042+0000","last_clean":"2024-04-02T17:11:52.224042+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:11:52.224042+0000","last_undegraded":"2024-04-02T17:11:52.224042+0000","last_fullsized":"2024-04-02T17:11:52.224042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236326+0000","last_change":"2024-04-02T17:10:51.642752+0000","last_active":"2024-04-02T17:11:52.236326+0000","last_peered":"2024-04-02T17:11:52.236326+0000","last_clean":"2024-04-02T17:11:52.236326+0000","last_became_active":"2024-04-02T17:10:51.635734+0000","last_became_peered":"2024-04-02T17:10:51.635734+0000","last_unstale":"2024-04-02T17:11:52.236326+0000","last_undegraded":"2024-04-02T17:11:52.236326+0000","last_fullsized":"2024-04-02T17:11:52.236326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.358529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224085+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:11:52.224085+0000","last_peered":"2024-04-02T17:11:52.224085+0000","last_clean":"2024-04-02T17:11:52.224085+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:11:52.224085+0000","last_undegraded":"2024-04-02T17:11:52.224085+0000","last_fullsized":"2024-04-02T17:11:52.224085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430091+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:11:52.430091+0000","last_peered":"2024-04-02T17:11:52.430091+0000","last_clean":"2024-04-02T17:11:52.430091+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:11:52.430091+0000","last_undegraded":"2024-04-02T17:11:52.430091+0000","last_fullsized":"2024-04-02T17:11:52.430091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236694+0000","last_change":"2024-04-02T17:11:52.236694+0000","last_active":"2024-04-02T17:11:52.236694+0000","last_peered":"2024-04-02T17:11:52.236694+0000","last_clean":"2024-04-02T17:11:52.236694+0000","last_became_active":"2024-04-02T17:11:52.236128+0000","last_became_peered":"2024-04-02T17:11:52.236128+0000","last_unstale":"2024-04-02T17:11:52.236694+0000","last_undegraded":"2024-04-02T17:11:52.236694+0000","last_fullsized":"2024-04-02T17:11:52.236694+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:40.601150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219328+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:11:52.219328+0000","last_peered":"2024-04-02T17:11:52.219328+0000","last_clean":"2024-04-02T17:11:52.219328+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:11:52.219328+0000","last_undegraded":"2024-04-02T17:11:52.219328+0000","last_fullsized":"2024-04-02T17:11:52.219328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819635+0000","last_change":"2024-04-02T17:10:51.332185+0000","last_active":"2024-04-02T17:11:52.819635+0000","last_peered":"2024-04-02T17:11:52.819635+0000","last_clean":"2024-04-02T17:11:52.819635+0000","last_became_active":"2024-04-02T17:10:51.331547+0000","last_became_peered":"2024-04-02T17:10:51.331547+0000","last_unstale":"2024-04-02T17:11:52.819635+0000","last_undegraded":"2024-04-02T17:11:52.819635+0000","last_fullsized":"2024-04-02T17:11:52.819635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:43.766907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236789+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:11:52.236789+0000","last_peered":"2024-04-02T17:11:52.236789+0000","last_clean":"2024-04-02T17:11:52.236789+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:11:52.236789+0000","last_undegraded":"2024-04-02T17:11:52.236789+0000","last_fullsized":"2024-04-02T17:11:52.236789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430427+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:11:52.430427+0000","last_peered":"2024-04-02T17:11:52.430427+0000","last_clean":"2024-04-02T17:11:52.430427+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:11:52.430427+0000","last_undegraded":"2024-04-02T17:11:52.430427+0000","last_fullsized":"2024-04-02T17:11:52.430427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219956+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:11:52.219956+0000","last_peered":"2024-04-02T17:11:52.219956+0000","last_clean":"2024-04-02T17:11:52.219956+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:11:52.219956+0000","last_undegraded":"2024-04-02T17:11:52.219956+0000","last_fullsized":"2024-04-02T17:11:52.219956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232558+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:11:52.232558+0000","last_peered":"2024-04-02T17:11:52.232558+0000","last_clean":"2024-04-02T17:11:52.232558+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:11:52.232558+0000","last_undegraded":"2024-04-02T17:11:52.232558+0000","last_fullsized":"2024-04-02T17:11:52.232558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221654+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:11:52.221654+0000","last_peered":"2024-04-02T17:11:52.221654+0000","last_clean":"2024-04-02T17:11:52.221654+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:11:52.221654+0000","last_undegraded":"2024-04-02T17:11:52.221654+0000","last_fullsized":"2024-04-02T17:11:52.221654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230530+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:11:52.230530+0000","last_peered":"2024-04-02T17:11:52.230530+0000","last_clean":"2024-04-02T17:11:52.230530+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:11:52.230530+0000","last_undegraded":"2024-04-02T17:11:52.230530+0000","last_fullsized":"2024-04-02T17:11:52.230530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429668+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:11:52.429668+0000","last_peered":"2024-04-02T17:11:52.429668+0000","last_clean":"2024-04-02T17:11:52.429668+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:11:52.429668+0000","last_undegraded":"2024-04-02T17:11:52.429668+0000","last_fullsized":"2024-04-02T17:11:52.429668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236536+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:11:52.236536+0000","last_peered":"2024-04-02T17:11:52.236536+0000","last_clean":"2024-04-02T17:11:52.236536+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:11:52.236536+0000","last_undegraded":"2024-04-02T17:11:52.236536+0000","last_fullsized":"2024-04-02T17:11:52.236536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430124+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:11:52.430124+0000","last_peered":"2024-04-02T17:11:52.430124+0000","last_clean":"2024-04-02T17:11:52.430124+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:11:52.430124+0000","last_undegraded":"2024-04-02T17:11:52.430124+0000","last_fullsized":"2024-04-02T17:11:52.430124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70163,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":796,"num_read_kb":470,"num_write":251,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":485886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1720320,"data_stored":1138970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":485311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":485572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1078763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":484911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":547969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":486782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":549253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":484619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":485304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1642496,"data_stored":1077819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:12:04.030 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T17:12:04.030 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T17:12:04.030 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T17:12:04.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T17:12:04.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:04 smithi154 ceph-mon[36479]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:04.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:04 smithi059 ceph-mon[31664]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:04.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:04 smithi164 ceph-mon[36499]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:12:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:05 smithi059 ceph-mon[31664]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 543 B/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-04-02T17:12:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:05 smithi164 ceph-mon[36499]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 543 B/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-04-02T17:12:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:05 smithi154 ceph-mon[36479]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 543 B/s wr, 12 op/s; 10 B/s, 0 objects/s recovering 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 -- 172.21.15.59:0/2605555430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 msgr2=0x7fe00c102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 --2- 172.21.15.59:0/2605555430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c102ac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0098f0 tx=0x7fdffc009c00 comp rx=0 tx=0).stop 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 -- 172.21.15.59:0/2605555430 shutdown_connections 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 --2- 172.21.15.59:0/2605555430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c103000 0x7fe00c10f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 --2- 172.21.15.59:0/2605555430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c108630 0x7fe00c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 --2- 172.21.15.59:0/2605555430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c102ac0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 -- 172.21.15.59:0/2605555430 >> 172.21.15.59:0/2605555430 conn(0x7fe00c079750 msgr2=0x7fe00c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 -- 172.21.15.59:0/2605555430 shutdown_connections 2024-04-02T17:12:06.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.608+0000 7fe014800700 1 -- 172.21.15.59:0/2605555430 wait complete. 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.609+0000 7fe014800700 1 Processor -- start 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.609+0000 7fe014800700 1 -- start start 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c103000 0x7fe00c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c108630 0x7fe00c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112a10 con 0x7fe00c108630 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112890 con 0x7fe00c102640 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe014800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112b90 con 0x7fe00c103000 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe011e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe011e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38912/0 (socket says 172.21.15.59:38912) 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe011e00700 1 -- 172.21.15.59:0/3588354347 learned_addr learned my addr 172.21.15.59:0/3588354347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:06.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.610+0000 7fe011400700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c103000 0x7fe00c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:06.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe012800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c108630 0x7fe00c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:06.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 -- 172.21.15.59:0/3588354347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c103000 msgr2=0x7fe00c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c103000 0x7fe00c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 -- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c108630 msgr2=0x7fe00c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c108630 0x7fe00c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 -- 172.21.15.59:0/3588354347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1a73f0 con 0x7fe00c102640 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe011e00700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe00000e9c0 tx=0x7fe00000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.611+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00000cbd0 con 0x7fe00c102640 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.612+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe00000ee60 con 0x7fe00c102640 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.612+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1a76e0 con 0x7fe00c102640 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.612+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000005630 con 0x7fe00c102640 2024-04-02T17:12:06.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.612+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a7bf0 con 0x7fe00c102640 2024-04-02T17:12:06.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.613+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c0692b0 con 0x7fe00c102640 2024-04-02T17:12:06.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.615+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe000004750 con 0x7fe00c102640 2024-04-02T17:12:06.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.616+0000 7fe006a00700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 0x7fdff40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:06.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.617+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe0000d0820 con 0x7fe00c102640 2024-04-02T17:12:06.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.617+0000 7fe011400700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 0x7fdff40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:06.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.617+0000 7fe011400700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 0x7fdff40a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009810 tx=0x7fdffc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:06.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.618+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe000053f20 con 0x7fe00c102640 2024-04-02T17:12:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.899+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe00c075e20 con 0x7fe00c102640 2024-04-02T17:12:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.900+0000 7fe006a00700 1 -- 172.21.15.59:0/3588354347 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe00009d140 con 0x7fe00c102640 2024-04-02T17:12:06.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:06.900 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 msgr2=0x7fdff40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 0x7fdff40a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009810 tx=0x7fdffc019040 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 msgr2=0x7fe00c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe00000e9c0 tx=0x7fe00000ecd0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 shutdown_connections 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdff40a3cc0 0x7fdff40a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00c108630 0x7fe00c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c102640 0x7fe00c0748f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 --2- 172.21.15.59:0/3588354347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe00c103000 0x7fe00c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 >> 172.21.15.59:0/3588354347 conn(0x7fe00c079750 msgr2=0x7fe00c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.903+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 shutdown_connections 2024-04-02T17:12:06.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:06.904+0000 7fe014800700 1 -- 172.21.15.59:0/3588354347 wait complete. 2024-04-02T17:12:07.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:07 smithi059 ceph-mon[31664]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:07.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:07 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3588354347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T17:12:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:07 smithi164 ceph-mon[36499]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:07 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3588354347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T17:12:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:07 smithi154 ceph-mon[36479]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:07 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3588354347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T17:12:07.772 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T17:12:07.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 -- 172.21.15.59:0/3892538017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4103d40 msgr2=0x7f58b4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 --2- 172.21.15.59:0/3892538017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4103d40 0x7f58b4104160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009900 tx=0x7f58a402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 -- 172.21.15.59:0/3892538017 shutdown_connections 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 --2- 172.21.15.59:0/3892538017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b41058f0 0x7f58b410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 --2- 172.21.15.59:0/3892538017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b4104f30 0x7f58b41053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 --2- 172.21.15.59:0/3892538017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4103d40 0x7f58b4104160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 -- 172.21.15.59:0/3892538017 >> 172.21.15.59:0/3892538017 conn(0x7f58b40ff2f0 msgr2=0x7f58b4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:08.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 -- 172.21.15.59:0/3892538017 shutdown_connections 2024-04-02T17:12:08.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.009+0000 7f58b9800640 1 -- 172.21.15.59:0/3892538017 wait complete. 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.010+0000 7f58b9800640 1 Processor -- start 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 -- start start 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 0x7f58b419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b41058f0 0x7f58b41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410fa00 con 0x7f58b4104f30 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410f880 con 0x7f58b4103d40 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410fb80 con 0x7f58b41058f0 2024-04-02T17:12:08.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45096/0 (socket says 172.21.15.59:45096) 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b2000640 1 -- 172.21.15.59:0/911471219 learned_addr learned my addr 172.21.15.59:0/911471219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b2a00640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 0x7f58b419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.011+0000 7f58b3400640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b41058f0 0x7f58b41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 -- 172.21.15.59:0/911471219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b41058f0 msgr2=0x7f58b41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b41058f0 0x7f58b41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 -- 172.21.15.59:0/911471219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 msgr2=0x7f58b419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 0x7f58b419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 -- 172.21.15.59:0/911471219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b41a1c90 con 0x7f58b4104f30 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2a00640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 0x7f58b419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b2000640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9f0 tx=0x7f589c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c00cd80 con 0x7f58b4104f30 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f589c005500 con 0x7f58b4104f30 2024-04-02T17:12:08.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.012+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b41a1f80 con 0x7f58b4104f30 2024-04-02T17:12:08.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.013+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c010690 con 0x7f58b4104f30 2024-04-02T17:12:08.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.013+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a2440 con 0x7f58b4104f30 2024-04-02T17:12:08.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.014+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b410b060 con 0x7f58b4104f30 2024-04-02T17:12:08.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.016+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f589c0046c0 con 0x7f58b4104f30 2024-04-02T17:12:08.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.018+0000 7f58ab400640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 0x7f58980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.018+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f589c0d0810 con 0x7f58b4104f30 2024-04-02T17:12:08.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.018+0000 7f58b2a00640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 0x7f58980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.019+0000 7f58b2a00640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 0x7f58980a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58a402c170 tx=0x7f58a402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.024+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f589c09d130 con 0x7f58b4104f30 2024-04-02T17:12:08.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.234+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58b40627a0 con 0x7f58b4104f30 2024-04-02T17:12:08.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.236+0000 7f58ab400640 1 -- 172.21.15.59:0/911471219 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f589c09ca30 con 0x7f58b4104f30 2024-04-02T17:12:08.236 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:08.236 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:08.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 msgr2=0x7f58980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 0x7f58980a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58a402c170 tx=0x7f58a402df90 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 msgr2=0x7f58b419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9f0 tx=0x7f589c00edf0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 shutdown_connections 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f58980a3c30 0x7f58980a60f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58b4104f30 0x7f58b419a5b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58b41058f0 0x7f58b41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 --2- 172.21.15.59:0/911471219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4103d40 0x7f58b419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.239+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 >> 172.21.15.59:0/911471219 conn(0x7f58b40ff2f0 msgr2=0x7f58b4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:08.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.240+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 shutdown_connections 2024-04-02T17:12:08.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.240+0000 7f58b9800640 1 -- 172.21.15.59:0/911471219 wait complete. 2024-04-02T17:12:08.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:08.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 -- 172.21.15.59:0/3748598274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01023a0 msgr2=0x7fb6d0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3748598274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01023a0 0x7fb6d0108c30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009ca0 tx=0x7fb6c802c100 comp rx=0 tx=0).stop 2024-04-02T17:12:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 -- 172.21.15.59:0/3748598274 shutdown_connections 2024-04-02T17:12:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3748598274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01019e0 0x7fb6d0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3748598274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01023a0 0x7fb6d0108c30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3748598274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01007f0 0x7fb6d0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 -- 172.21.15.59:0/3748598274 >> 172.21.15.59:0/3748598274 conn(0x7fb6d00fbda0 msgr2=0x7fb6d00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 -- 172.21.15.59:0/3748598274 shutdown_connections 2024-04-02T17:12:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.492+0000 7fb6cf200640 1 -- 172.21.15.59:0/3748598274 wait complete. 2024-04-02T17:12:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.493+0000 7fb6cf200640 1 Processor -- start 2024-04-02T17:12:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.493+0000 7fb6cf200640 1 -- start start 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01019e0 0x7fb6d01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01023a0 0x7fb6d019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d00697f0 con 0x7fb6d01023a0 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0069670 con 0x7fb6d01019e0 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0069970 con 0x7fb6d01007f0 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36338/0 (socket says 172.21.15.59:36338) 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 -- 172.21.15.59:0/3590955473 learned_addr learned my addr 172.21.15.59:0/3590955473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6c5400640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01019e0 0x7fb6d01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6ce800640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01023a0 0x7fb6d019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 -- 172.21.15.59:0/3590955473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01019e0 msgr2=0x7fb6d01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01019e0 0x7fb6d01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 -- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01023a0 msgr2=0x7fb6d019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01023a0 0x7fb6d019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.494+0000 7fb6cde00640 1 -- 172.21.15.59:0/3590955473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d019f750 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6cde00640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e9c0 tx=0x7fb6c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000cd80 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c00035f0 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d019fa40 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0010690 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.495+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d019ff90 con 0x7fb6d01007f0 2024-04-02T17:12:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.496+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0107b10 con 0x7fb6d01007f0 2024-04-02T17:12:08.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.498+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6c00037a0 con 0x7fb6d01007f0 2024-04-02T17:12:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.500+0000 7fb6c7400640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 0x7fb6b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.500+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb6c00d07a0 con 0x7fb6d01007f0 2024-04-02T17:12:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.500+0000 7fb6c5400640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 0x7fb6b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.501+0000 7fb6c5400640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 0x7fb6b00a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0199040 tx=0x7fb6b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.504+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb6c009d0c0 con 0x7fb6d01007f0 2024-04-02T17:12:08.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.710+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6d00627a0 con 0x7fb6d01007f0 2024-04-02T17:12:08.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.711+0000 7fb6c7400640 1 -- 172.21.15.59:0/3590955473 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fb6c009c9c0 con 0x7fb6d01007f0 2024-04-02T17:12:08.710 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.713+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 msgr2=0x7fb6b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 0x7fb6b00a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0199040 tx=0x7fb6b8005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 msgr2=0x7fb6d0197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e9c0 tx=0x7fb6c000ee80 comp rx=0 tx=0).stop 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 shutdown_connections 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb6b00a3c30 0x7fb6b00a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d01023a0 0x7fb6d019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d01019e0 0x7fb6d01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 --2- 172.21.15.59:0/3590955473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6d01007f0 0x7fb6d0197b80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 >> 172.21.15.59:0/3590955473 conn(0x7fb6d00fbda0 msgr2=0x7fb6d0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 shutdown_connections 2024-04-02T17:12:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.714+0000 7fb6cf200640 1 -- 172.21.15.59:0/3590955473 wait complete. 2024-04-02T17:12:08.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 -- 172.21.15.59:0/13839641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 msgr2=0x7f9ad4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/13839641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4107180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009ca0 tx=0x7f9ac002c0f0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 -- 172.21.15.59:0/13839641 shutdown_connections 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/13839641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4107180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/13839641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad41076c0 0x7f9ad4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/13839641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad4102460 0x7f9ad4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 -- 172.21.15.59:0/13839641 >> 172.21.15.59:0/13839641 conn(0x7f9ad40fbe70 msgr2=0x7f9ad40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.965+0000 7f9ad9e00640 1 -- 172.21.15.59:0/13839641 shutdown_connections 2024-04-02T17:12:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.966+0000 7f9ad9e00640 1 -- 172.21.15.59:0/13839641 wait complete. 2024-04-02T17:12:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.966+0000 7f9ad9e00640 1 Processor -- start 2024-04-02T17:12:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 -- start start 2024-04-02T17:12:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 0x7f9ad4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad41076c0 0x7f9ad419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410d670 con 0x7f9ad4104d90 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410d4f0 con 0x7f9ad4102460 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad410d7f0 con 0x7f9ad41076c0 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad8a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 0x7f9ad4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45142/0 (socket says 172.21.15.59:45142) 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad3e00640 1 -- 172.21.15.59:0/1805637263 learned_addr learned my addr 172.21.15.59:0/1805637263 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.967+0000 7f9ad9400640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad41076c0 0x7f9ad419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 -- 172.21.15.59:0/1805637263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad41076c0 msgr2=0x7f9ad419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad41076c0 0x7f9ad419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 -- 172.21.15.59:0/1805637263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 msgr2=0x7f9ad4197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 0x7f9ad4197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 -- 172.21.15.59:0/1805637263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad419f950 con 0x7f9ad4104d90 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad8a00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 0x7f9ad4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:08.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.968+0000 7f9ad3e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0000f80 tx=0x7f9ac0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.969+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0041020 con 0x7f9ad4104d90 2024-04-02T17:12:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.969+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0034cd0 con 0x7f9ad4104d90 2024-04-02T17:12:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.969+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad419fbe0 con 0x7f9ad4104d90 2024-04-02T17:12:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.969+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0039040 con 0x7f9ad4104d90 2024-04-02T17:12:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.969+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad41a00f0 con 0x7f9ad4104d90 2024-04-02T17:12:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.970+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad40fd960 con 0x7f9ad4104d90 2024-04-02T17:12:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.973+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ac0044020 con 0x7f9ad4104d90 2024-04-02T17:12:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.974+0000 7f9ad1600640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 0x7f9ab80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.974+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9ac00f2210 con 0x7f9ad4104d90 2024-04-02T17:12:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.974+0000 7f9ad8a00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 0x7f9ab80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:08.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.975+0000 7f9ad8a00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 0x7f9ab80a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4199210 tx=0x7f9ac4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:08.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:08.979+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9ac004b020 con 0x7f9ad4104d90 2024-04-02T17:12:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.184+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ad4062700 con 0x7f9ad4104d90 2024-04-02T17:12:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.185+0000 7f9ad1600640 1 -- 172.21.15.59:0/1805637263 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f9ac00beb30 con 0x7f9ad4104d90 2024-04-02T17:12:09.184 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 msgr2=0x7f9ab80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 0x7f9ab80a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4199210 tx=0x7f9ac4005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 msgr2=0x7f9ad4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0000f80 tx=0x7f9ac0032ec0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 shutdown_connections 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ab80a3c30 0x7f9ab80a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad4104d90 0x7f9ad4198280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad41076c0 0x7f9ad419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 --2- 172.21.15.59:0/1805637263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4102460 0x7f9ad4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 >> 172.21.15.59:0/1805637263 conn(0x7f9ad40fbe70 msgr2=0x7f9ad4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 shutdown_connections 2024-04-02T17:12:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.187+0000 7f9ad9e00640 1 -- 172.21.15.59:0/1805637263 wait complete. 2024-04-02T17:12:09.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:09 smithi059 ceph-mon[31664]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:09 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/911471219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:09 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3590955473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:09 smithi164 ceph-mon[36499]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:09 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/911471219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:09 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3590955473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:09 smithi154 ceph-mon[36479]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:09 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/911471219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:09.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:09 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3590955473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 -- 172.21.15.59:0/1824736535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec077700 msgr2=0x7f9cec077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 --2- 172.21.15.59:0/1824736535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec077700 0x7f9cec077b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009900 tx=0x7f9cd402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 -- 172.21.15.59:0/1824736535 shutdown_connections 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 --2- 172.21.15.59:0/1824736535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec077700 0x7f9cec077b60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 --2- 172.21.15.59:0/1824736535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0780a0 0x7f9cec078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 --2- 172.21.15.59:0/1824736535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec0764c0 0x7f9cec0768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 -- 172.21.15.59:0/1824736535 >> 172.21.15.59:0/1824736535 conn(0x7f9cec0ff2b0 msgr2=0x7f9cec1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 -- 172.21.15.59:0/1824736535 shutdown_connections 2024-04-02T17:12:09.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.425+0000 7f9cf3400640 1 -- 172.21.15.59:0/1824736535 wait complete. 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.426+0000 7f9cf3400640 1 Processor -- start 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.426+0000 7f9cf3400640 1 -- start start 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec077700 0x7f9cec19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec0780a0 0x7f9cec1a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec113dd0 con 0x7f9cec0780a0 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec113c50 con 0x7f9cec0764c0 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf3400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec113f50 con 0x7f9cec077700 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39000/0 (socket says 172.21.15.59:39000) 2024-04-02T17:12:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf0a00640 1 -- 172.21.15.59:0/4162924715 learned_addr learned my addr 172.21.15.59:0/4162924715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9ce3e00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec077700 0x7f9cec19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.427+0000 7f9cf1400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec0780a0 0x7f9cec1a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 -- 172.21.15.59:0/4162924715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec077700 msgr2=0x7f9cec19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec077700 0x7f9cec19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 -- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec0780a0 msgr2=0x7f9cec1a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec0780a0 0x7f9cec1a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 -- 172.21.15.59:0/4162924715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec1a6040 con 0x7f9cec0764c0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf0a00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9f0 tx=0x7f9cdc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00cd80 con 0x7f9cec0764c0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cdc005500 con 0x7f9cec0764c0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec1a6330 con 0x7f9cec0764c0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.428+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc010690 con 0x7f9cec0764c0 2024-04-02T17:12:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.429+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec1a6800 con 0x7f9cec0764c0 2024-04-02T17:12:09.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.430+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cec110840 con 0x7f9cec0764c0 2024-04-02T17:12:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.432+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9cdc0046c0 con 0x7f9cec0764c0 2024-04-02T17:12:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.433+0000 7f9cea000640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 0x7f9ccc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.433+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9cdc0d0810 con 0x7f9cec0764c0 2024-04-02T17:12:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.433+0000 7f9ce3e00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 0x7f9ccc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.434+0000 7f9ce3e00640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 0x7f9ccc0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402c7b0 tx=0x7f9cd4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.435+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9cdc09d190 con 0x7f9cec0764c0 2024-04-02T17:12:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.646+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cec062700 con 0x7f9cec0764c0 2024-04-02T17:12:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.647+0000 7f9cea000640 1 -- 172.21.15.59:0/4162924715 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f9cdc0bd0d0 con 0x7f9cec0764c0 2024-04-02T17:12:09.646 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 msgr2=0x7f9ccc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 0x7f9ccc0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402c7b0 tx=0x7f9cd4005fd0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 msgr2=0x7f9cec19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9f0 tx=0x7f9cdc00edf0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 shutdown_connections 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ccc0a3bb0 0x7f9ccc0a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cec0780a0 0x7f9cec1a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cec077700 0x7f9cec19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 --2- 172.21.15.59:0/4162924715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cec0764c0 0x7f9cec19e420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 >> 172.21.15.59:0/4162924715 conn(0x7f9cec0ff2b0 msgr2=0x7f9cec100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 shutdown_connections 2024-04-02T17:12:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.650+0000 7f9cf3400640 1 -- 172.21.15.59:0/4162924715 wait complete. 2024-04-02T17:12:09.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 -- 172.21.15.59:0/3413449714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b01012a0 msgr2=0x7fa7b0101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 --2- 172.21.15.59:0/3413449714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b01012a0 0x7fa7b0101720 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009900 tx=0x7fa79c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 -- 172.21.15.59:0/3413449714 shutdown_connections 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 --2- 172.21.15.59:0/3413449714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b01012a0 0x7fa7b0101720 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 --2- 172.21.15.59:0/3413449714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b0105fa0 0x7fa7b0108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 --2- 172.21.15.59:0/3413449714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 -- 172.21.15.59:0/3413449714 >> 172.21.15.59:0/3413449714 conn(0x7fa7b00fb0e0 msgr2=0x7fa7b00fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 -- 172.21.15.59:0/3413449714 shutdown_connections 2024-04-02T17:12:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.896+0000 7fa7afe00640 1 -- 172.21.15.59:0/3413449714 wait complete. 2024-04-02T17:12:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.897+0000 7fa7afe00640 1 Processor -- start 2024-04-02T17:12:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.897+0000 7fa7afe00640 1 -- start start 2024-04-02T17:12:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.897+0000 7fa7afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.897+0000 7fa7afe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b01012a0 0x7fa7b019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.897+0000 7fa7afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7afe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010d610 con 0x7fa7b0105fa0 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7afe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010d490 con 0x7fa7b01012a0 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7afe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010d790 con 0x7fa7b0067f80 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45190/0 (socket says 172.21.15.59:45190) 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36412/0 (socket says 172.21.15.59:36412) 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7af400640 1 -- 172.21.15.59:0/1435567484 learned_addr learned my addr 172.21.15.59:0/1435567484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7ae000640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b01012a0 0x7fa7b019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 -- 172.21.15.59:0/1435567484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b01012a0 msgr2=0x7fa7b019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b01012a0 0x7fa7b019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 -- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 msgr2=0x7fa7b01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7aea00640 1 -- 172.21.15.59:0/1435567484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01a1b10 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.898+0000 7fa7af400640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7aea00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000e9c0 tx=0x7fa7a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000cd80 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a0010910 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01a1e00 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0018980 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.899+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a2260 con 0x7fa7b0067f80 2024-04-02T17:12:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.900+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b01a2490 con 0x7fa7b0067f80 2024-04-02T17:12:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.902+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa7a001f020 con 0x7fa7b0067f80 2024-04-02T17:12:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.904+0000 7fa7a7400640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 0x7fa7940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.904+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7a00d0440 con 0x7fa7b0067f80 2024-04-02T17:12:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.904+0000 7fa7ae000640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 0x7fa7940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.905+0000 7fa7ae000640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 0x7fa7940a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa79c02c170 tx=0x7fa79c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:09.908+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa7a009cd60 con 0x7fa7b0067f80 2024-04-02T17:12:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.167+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7b01a2dd0 con 0x7fa7b0067f80 2024-04-02T17:12:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.168+0000 7fa7a7400640 1 -- 172.21.15.59:0/1435567484 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7fa7a009c660 con 0x7fa7b0067f80 2024-04-02T17:12:10.168 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:10.168 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:12:10.168 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 msgr2=0x7fa7940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 0x7fa7940a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa79c02c170 tx=0x7fa79c02df90 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 msgr2=0x7fa7b0199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000e9c0 tx=0x7fa7a000ee80 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 shutdown_connections 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7940a3bb0 0x7fa7940a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7b0105fa0 0x7fa7b01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7b01012a0 0x7fa7b019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 --2- 172.21.15.59:0/1435567484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7b0067f80 0x7fa7b0199ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 >> 172.21.15.59:0/1435567484 conn(0x7fa7b00fb0e0 msgr2=0x7fa7b00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 shutdown_connections 2024-04-02T17:12:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.171+0000 7fa7afe00640 1 -- 172.21.15.59:0/1435567484 wait complete. 2024-04-02T17:12:10.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:10 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1805637263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:10 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4162924715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:10 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1805637263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:10 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4162924715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:10 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1805637263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:10 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4162924715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 -- 172.21.15.59:0/3482678607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 msgr2=0x7f892c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 --2- 172.21.15.59:0/3482678607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c108350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f891c009860 tx=0x7f891c02bb70 comp rx=0 tx=0).stop 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 -- 172.21.15.59:0/3482678607 shutdown_connections 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 --2- 172.21.15.59:0/3482678607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c109aa0 0x7f892c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 --2- 172.21.15.59:0/3482678607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c1090e0 0x7f892c109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 --2- 172.21.15.59:0/3482678607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c108350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 -- 172.21.15.59:0/3482678607 >> 172.21.15.59:0/3482678607 conn(0x7f892c076f10 msgr2=0x7f892c079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 -- 172.21.15.59:0/3482678607 shutdown_connections 2024-04-02T17:12:10.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.419+0000 7f892be00640 1 -- 172.21.15.59:0/3482678607 wait complete. 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 Processor -- start 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 -- start start 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 0x7f892c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113be0 con 0x7f892c1090e0 2024-04-02T17:12:10.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113a60 con 0x7f892c107f10 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113d60 con 0x7f892c109aa0 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.421+0000 7f892a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 0x7f892c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36424/0 (socket says 172.21.15.59:36424) 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39038/0 (socket says 172.21.15.59:39038) 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892b400640 1 -- 172.21.15.59:0/3697056296 learned_addr learned my addr 172.21.15.59:0/3697056296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892a000640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 0x7f892c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45210/0 (socket says 172.21.15.59:45210) 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 -- 172.21.15.59:0/3697056296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 msgr2=0x7f892c1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 -- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 msgr2=0x7f892c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 0x7f892c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 -- 172.21.15.59:0/3697056296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c1a5ea0 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892b400640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.422+0000 7f892aa00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f891c009780 tx=0x7f891c032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.423+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c041020 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.423+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c02c9f0 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.423+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c1a6130 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.423+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c039040 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.423+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1a6640 con 0x7f892c107f10 2024-04-02T17:12:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.424+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c10f2a0 con 0x7f892c107f10 2024-04-02T17:12:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.426+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f891c044020 con 0x7f892c107f10 2024-04-02T17:12:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.427+0000 7f8930a00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 0x7f89100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.427+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f891c0f1fd0 con 0x7f892c107f10 2024-04-02T17:12:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.428+0000 7f892a000640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 0x7f89100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.428+0000 7f892a000640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 0x7f89100a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f892c19f790 tx=0x7f8918005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.430+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f891c04b020 con 0x7f892c107f10 2024-04-02T17:12:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.637+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f892c062700 con 0x7f892c107f10 2024-04-02T17:12:10.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.639+0000 7f8930a00640 1 -- 172.21.15.59:0/3697056296 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f891c0be970 con 0x7f892c107f10 2024-04-02T17:12:10.638 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:10.639 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:10.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.641+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 msgr2=0x7f89100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.641+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 0x7f89100a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f892c19f790 tx=0x7f8918005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 msgr2=0x7f892c19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f891c009780 tx=0x7f891c032430 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 shutdown_connections 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f89100a3c30 0x7f89100a60f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c1090e0 0x7f892c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f892c109aa0 0x7f892c1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 --2- 172.21.15.59:0/3697056296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f892c107f10 0x7f892c19e280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 >> 172.21.15.59:0/3697056296 conn(0x7f892c076f10 msgr2=0x7f892c10c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 shutdown_connections 2024-04-02T17:12:10.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.642+0000 7f892be00640 1 -- 172.21.15.59:0/3697056296 wait complete. 2024-04-02T17:12:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:10.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 -- 172.21.15.59:0/1943320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 msgr2=0x7fdf80104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/1943320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80104760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009900 tx=0x7fdf7002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 -- 172.21.15.59:0/1943320937 shutdown_connections 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/1943320937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf80109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/1943320937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf80107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/1943320937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80104760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 -- 172.21.15.59:0/1943320937 >> 172.21.15.59:0/1943320937 conn(0x7fdf800fbda0 msgr2=0x7fdf800fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 -- 172.21.15.59:0/1943320937 shutdown_connections 2024-04-02T17:12:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.885+0000 7fdf7fe00640 1 -- 172.21.15.59:0/1943320937 wait complete. 2024-04-02T17:12:10.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.886+0000 7fdf7fe00640 1 Processor -- start 2024-04-02T17:12:10.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.886+0000 7fdf7fe00640 1 -- start start 2024-04-02T17:12:10.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf8019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf801a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8010d570 con 0x7fdf801075d0 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8010d3f0 con 0x7fdf80104ca0 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8010d6f0 con 0x7fdf80102370 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43702/0 (socket says 172.21.15.59:43702) 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 -- 172.21.15.59:0/422210190 learned_addr learned my addr 172.21.15.59:0/422210190 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7f400640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf801a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7e000640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf8019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 -- 172.21.15.59:0/422210190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 msgr2=0x7fdf8019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf8019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 -- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 msgr2=0x7fdf801a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf801a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7ea00640 1 -- 172.21.15.59:0/422210190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf801a1a20 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.887+0000 7fdf7f400640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf801a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf7e000640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf8019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf7ea00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009230 tx=0x7fdf70005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70041070 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf700044d0 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70034440 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf801a1cb0 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.888+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf801a2150 con 0x7fdf80102370 2024-04-02T17:12:10.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.889+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf80109120 con 0x7fdf80102370 2024-04-02T17:12:10.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.892+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf70044070 con 0x7fdf80102370 2024-04-02T17:12:10.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.893+0000 7fdf77400640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 0x7fdf640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:10.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.893+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdf700f24c0 con 0x7fdf80102370 2024-04-02T17:12:10.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.893+0000 7fdf7e000640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 0x7fdf640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:10.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.894+0000 7fdf7e000640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 0x7fdf640a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdf68005d00 tx=0x7fdf68005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:10.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:10.895+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdf700bede0 con 0x7fdf80102370 2024-04-02T17:12:11.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.109+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf8002d600 con 0x7fdf80102370 2024-04-02T17:12:11.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.110+0000 7fdf77400640 1 -- 172.21.15.59:0/422210190 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fdf700be6e0 con 0x7fdf80102370 2024-04-02T17:12:11.109 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 msgr2=0x7fdf640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 0x7fdf640a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdf68005d00 tx=0x7fdf68005c70 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 msgr2=0x7fdf80199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009230 tx=0x7fdf70005d50 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 shutdown_connections 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdf640a3c30 0x7fdf640a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf801075d0 0x7fdf801a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf80104ca0 0x7fdf8019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 --2- 172.21.15.59:0/422210190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf80102370 0x7fdf80199e00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 >> 172.21.15.59:0/422210190 conn(0x7fdf800fbda0 msgr2=0x7fdf80101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 shutdown_connections 2024-04-02T17:12:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.113+0000 7fdf7fe00640 1 -- 172.21.15.59:0/422210190 wait complete. 2024-04-02T17:12:11.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:11 smithi164 ceph-mon[36499]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:11 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1435567484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:11 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3697056296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 -- 172.21.15.59:0/2953751541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1701019e0 msgr2=0x7ff170101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 --2- 172.21.15.59:0/2953751541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1701019e0 0x7ff170101e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff164009900 tx=0x7ff16402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 -- 172.21.15.59:0/2953751541 shutdown_connections 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 --2- 172.21.15.59:0/2953751541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff170103590 0x7ff170109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 --2- 172.21.15.59:0/2953751541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 0x7ff170103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 --2- 172.21.15.59:0/2953751541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1701019e0 0x7ff170101e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 -- 172.21.15.59:0/2953751541 >> 172.21.15.59:0/2953751541 conn(0x7ff1700fcf70 msgr2=0x7ff1700ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 -- 172.21.15.59:0/2953751541 shutdown_connections 2024-04-02T17:12:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.361+0000 7ff176e00640 1 -- 172.21.15.59:0/2953751541 wait complete. 2024-04-02T17:12:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.362+0000 7ff176e00640 1 Processor -- start 2024-04-02T17:12:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.362+0000 7ff176e00640 1 -- start start 2024-04-02T17:12:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 0x7ff17019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff17010d6b0 con 0x7ff170103590 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff17010d530 con 0x7ff170102bd0 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff17010d830 con 0x7ff1701019e0 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff175a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff175000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 0x7ff17019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33216/0 (socket says 172.21.15.59:33216) 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff175a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43728/0 (socket says 172.21.15.59:43728) 2024-04-02T17:12:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.363+0000 7ff176400640 1 -- 172.21.15.59:0/3718788304 learned_addr learned my addr 172.21.15.59:0/3718788304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 -- 172.21.15.59:0/3718788304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 msgr2=0x7ff17019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 0x7ff17019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 -- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 msgr2=0x7ff1701a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 -- 172.21.15.59:0/3718788304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1701a1b40 con 0x7ff1701019e0 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff176400640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff175a00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff164009230 tx=0x7ff16402c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164043020 con 0x7ff1701019e0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1640054e0 con 0x7ff1701019e0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1701a1dd0 con 0x7ff1701019e0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164003b60 con 0x7ff1701019e0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.364+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1701a2240 con 0x7ff1701019e0 2024-04-02T17:12:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.366+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1701a2440 con 0x7ff1701019e0 2024-04-02T17:12:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.368+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff164046020 con 0x7ff1701019e0 2024-04-02T17:12:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.369+0000 7ff16e000640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 0x7ff1580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.369+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff1640f2a80 con 0x7ff1701019e0 2024-04-02T17:12:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.369+0000 7ff175000640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 0x7ff1580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.370+0000 7ff175000640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 0x7ff1580a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff17019b430 tx=0x7ff15c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.371+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff16404d020 con 0x7ff1701019e0 2024-04-02T17:12:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:11 smithi154 ceph-mon[36479]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:11 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1435567484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:11 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3697056296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:11 smithi059 ceph-mon[31664]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:11 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1435567484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:11 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3697056296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:11.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.576+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1701a27d0 con 0x7ff1701019e0 2024-04-02T17:12:11.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.577+0000 7ff16e000640 1 -- 172.21.15.59:0/3718788304 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff1640bf3a0 con 0x7ff1701019e0 2024-04-02T17:12:11.576 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:11.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 msgr2=0x7ff1580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 0x7ff1580a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff17019b430 tx=0x7ff15c005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:11.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 msgr2=0x7ff170199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff164009230 tx=0x7ff16402c5d0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 shutdown_connections 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1580a3c30 0x7ff1580a60f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff170103590 0x7ff1701a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff170102bd0 0x7ff17019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 --2- 172.21.15.59:0/3718788304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1701019e0 0x7ff170199f20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 >> 172.21.15.59:0/3718788304 conn(0x7ff1700fcf70 msgr2=0x7ff170073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 shutdown_connections 2024-04-02T17:12:11.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.580+0000 7ff176e00640 1 -- 172.21.15.59:0/3718788304 wait complete. 2024-04-02T17:12:11.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 -- 172.21.15.59:0/1525676211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9818102bf0 msgr2=0x7f9818103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 --2- 172.21.15.59:0/1525676211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9818102bf0 0x7f9818103070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9800009900 tx=0x7f980002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 -- 172.21.15.59:0/1525676211 shutdown_connections 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 --2- 172.21.15.59:0/1525676211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98181035b0 0x7f9818109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 --2- 172.21.15.59:0/1525676211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9818102bf0 0x7f9818103070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 --2- 172.21.15.59:0/1525676211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818101a00 0x7f9818101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 -- 172.21.15.59:0/1525676211 >> 172.21.15.59:0/1525676211 conn(0x7f98180fcf70 msgr2=0x7f98180ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 -- 172.21.15.59:0/1525676211 shutdown_connections 2024-04-02T17:12:11.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.820+0000 7f9817200640 1 -- 172.21.15.59:0/1525676211 wait complete. 2024-04-02T17:12:11.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.821+0000 7f9817200640 1 Processor -- start 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.821+0000 7f9817200640 1 -- start start 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 0x7f9818199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98181035b0 0x7f98181a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f981810d680 con 0x7f9818102bf0 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f981810d500 con 0x7f98181035b0 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9817200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f981810d800 con 0x7f9818101a00 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f980d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f980d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33234/0 (socket says 172.21.15.59:33234) 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f980d400640 1 -- 172.21.15.59:0/1382706019 learned_addr learned my addr 172.21.15.59:0/1382706019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9815e00640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 0x7f9818199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.822+0000 7f9816800640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98181035b0 0x7f98181a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 -- 172.21.15.59:0/1382706019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98181035b0 msgr2=0x7f98181a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98181035b0 0x7f98181a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 -- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 msgr2=0x7f981819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 -- 172.21.15.59:0/1382706019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98181a1b50 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f980d400640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f9815e00640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 0x7f9818199f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9808005fd0 tx=0x7f980800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.823+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808003970 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.824+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808004030 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.824+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98181a1e40 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.824+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080107f0 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.824+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98181a22a0 con 0x7f9818101a00 2024-04-02T17:12:11.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.825+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98181a2530 con 0x7f9818101a00 2024-04-02T17:12:11.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.827+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f98080041e0 con 0x7f9818101a00 2024-04-02T17:12:11.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.828+0000 7f980f400640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 0x7f97fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:11.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.828+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f98080d0c80 con 0x7f9818101a00 2024-04-02T17:12:11.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.828+0000 7f980d400640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 0x7f97fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:11.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.829+0000 7f980d400640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 0x7f97fc0a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f980002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:11.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:11.831+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f980809d5a0 con 0x7f9818101a00 2024-04-02T17:12:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.035+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98181a27a0 con 0x7f9818101a00 2024-04-02T17:12:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.036+0000 7f980f400640 1 -- 172.21.15.59:0/1382706019 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f980809cea0 con 0x7f9818101a00 2024-04-02T17:12:12.036 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 msgr2=0x7f97fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 0x7f97fc0a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f980002c880 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 msgr2=0x7f9818199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 0x7f9818199f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9808005fd0 tx=0x7f980800ef90 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 shutdown_connections 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f97fc0a3bb0 0x7f97fc0a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9818102bf0 0x7f981819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98181035b0 0x7f98181a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 --2- 172.21.15.59:0/1382706019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9818101a00 0x7f9818199f30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 >> 172.21.15.59:0/1382706019 conn(0x7f98180fcf70 msgr2=0x7f9818073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 shutdown_connections 2024-04-02T17:12:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.039+0000 7f9817200640 1 -- 172.21.15.59:0/1382706019 wait complete. 2024-04-02T17:12:12.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 -- 172.21.15.59:0/3312377036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28103b50 msgr2=0x7f6b28103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 --2- 172.21.15.59:0/3312377036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28103b50 0x7f6b28103f70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009be0 tx=0x7f6b1802bef0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 -- 172.21.15.59:0/3312377036 shutdown_connections 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 --2- 172.21.15.59:0/3312377036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28105700 0x7f6b2810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 --2- 172.21.15.59:0/3312377036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b281051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 --2- 172.21.15.59:0/3312377036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28103b50 0x7f6b28103f70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 -- 172.21.15.59:0/3312377036 >> 172.21.15.59:0/3312377036 conn(0x7f6b280ff0e0 msgr2=0x7f6b28101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 -- 172.21.15.59:0/3312377036 shutdown_connections 2024-04-02T17:12:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.280+0000 7f6b2d600640 1 -- 172.21.15.59:0/3312377036 wait complete. 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.281+0000 7f6b2d600640 1 Processor -- start 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 -- start start 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 0x7f6b28199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b2819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 0x7f6b281a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810f7c0 con 0x7f6b28103b50 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810f640 con 0x7f6b28105700 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b2d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810f940 con 0x7f6b28104d40 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.282+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 0x7f6b28199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 0x7f6b28199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33244/0 (socket says 172.21.15.59:33244) 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27e00640 1 -- 172.21.15.59:0/395414754 learned_addr learned my addr 172.21.15.59:0/395414754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:12.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b2819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b2cc00640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 0x7f6b281a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 -- 172.21.15.59:0/395414754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 msgr2=0x7f6b281a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 0x7f6b281a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 -- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 msgr2=0x7f6b28199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 0x7f6b28199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 -- 172.21.15.59:0/395414754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b281a1a80 con 0x7f6b28104d40 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b2cc00640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 0x7f6b281a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.283+0000 7f6b27400640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b2819a3a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009fd0 tx=0x7f6b1000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.284+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000ccc0 con 0x7f6b28104d40 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.284+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b281a1d70 con 0x7f6b28104d40 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.284+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1000ee70 con 0x7f6b28104d40 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.284+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10004a00 con 0x7f6b28104d40 2024-04-02T17:12:12.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.284+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a2210 con 0x7f6b28104d40 2024-04-02T17:12:12.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.285+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2810aef0 con 0x7f6b28104d40 2024-04-02T17:12:12.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.287+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6b10004ba0 con 0x7f6b28104d40 2024-04-02T17:12:12.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.289+0000 7f6b24c00640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.289+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6b100d0480 con 0x7f6b28104d40 2024-04-02T17:12:12.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.289+0000 7f6b27e00640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.290+0000 7f6b27e00640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006010 tx=0x7f6b18000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:12.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.292+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6b1009cda0 con 0x7f6b28104d40 2024-04-02T17:12:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:12 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/422210190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:12 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3718788304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:12 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1382706019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:12 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/422210190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:12 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3718788304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:12 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1382706019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.530+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b2802d370 con 0x7f6b28104d40 2024-04-02T17:12:12.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.531+0000 7f6b24c00640 1 -- 172.21.15.59:0/395414754 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f6b1009c6a0 con 0x7f6b28104d40 2024-04-02T17:12:12.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:12.531 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:12:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 msgr2=0x7f6b0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006010 tx=0x7f6b18000f40 comp rx=0 tx=0).stop 2024-04-02T17:12:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 msgr2=0x7f6b2819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b2819a3a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009fd0 tx=0x7f6b1000ec20 comp rx=0 tx=0).stop 2024-04-02T17:12:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 shutdown_connections 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b28105700 0x7f6b281a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b28104d40 0x7f6b2819a3a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6b0c0a3bb0 0x7f6b0c0a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 --2- 172.21.15.59:0/395414754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b28103b50 0x7f6b28199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 >> 172.21.15.59:0/395414754 conn(0x7f6b280ff0e0 msgr2=0x7f6b281080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 shutdown_connections 2024-04-02T17:12:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.534+0000 7f6b2d600640 1 -- 172.21.15.59:0/395414754 wait complete. 2024-04-02T17:12:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:12 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/422210190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:12 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3718788304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:12 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1382706019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:12.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 -- 172.21.15.59:0/784249814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 msgr2=0x7f9b38101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/784249814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38101d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009ca0 tx=0x7f9b2802c100 comp rx=0 tx=0).stop 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 -- 172.21.15.59:0/784249814 shutdown_connections 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/784249814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38103510 0x7f9b38109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/784249814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38102b50 0x7f9b38102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/784249814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38101d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 -- 172.21.15.59:0/784249814 >> 172.21.15.59:0/784249814 conn(0x7f9b380fced0 msgr2=0x7f9b380ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 -- 172.21.15.59:0/784249814 shutdown_connections 2024-04-02T17:12:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.762+0000 7f9b3ec00640 1 -- 172.21.15.59:0/784249814 wait complete. 2024-04-02T17:12:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.763+0000 7f9b3ec00640 1 Processor -- start 2024-04-02T17:12:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 -- start start 2024-04-02T17:12:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38103510 0x7f9b381a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d5b0 con 0x7f9b38101960 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d430 con 0x7f9b38102b50 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d730 con 0x7f9b38103510 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42664/0 (socket says 172.21.15.59:42664) 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3ce00640 1 -- 172.21.15.59:0/1675580728 learned_addr learned my addr 172.21.15.59:0/1675580728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.764+0000 7f9b3e200640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38103510 0x7f9b381a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 -- 172.21.15.59:0/1675580728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38103510 msgr2=0x7f9b381a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38103510 0x7f9b381a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 -- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 msgr2=0x7f9b38199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 -- 172.21.15.59:0/1675580728 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a1ab0 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3d800640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ce00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cd80 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b24010910 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a1da0 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24018990 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.765+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a2260 con 0x7f9b38102b50 2024-04-02T17:12:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.766+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b381a2490 con 0x7f9b38102b50 2024-04-02T17:12:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.769+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b24010ac0 con 0x7f9b38102b50 2024-04-02T17:12:12.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.770+0000 7f9b36000640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:12.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.771+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9b240d0b60 con 0x7f9b38102b50 2024-04-02T17:12:12.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.771+0000 7f9b3d800640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:12.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.772+0000 7f9b3d800640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9b2802c710 tx=0x7f9b28032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:12.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.772+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b2409ce30 con 0x7f9b38102b50 2024-04-02T17:12:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.984+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b381a27e0 con 0x7f9b38102b50 2024-04-02T17:12:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.986+0000 7f9b36000640 1 -- 172.21.15.59:0/1675580728 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f9b240a1130 con 0x7f9b38102b50 2024-04-02T17:12:12.986 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:12.986 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 msgr2=0x7f9b1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9b2802c710 tx=0x7f9b28032590 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 msgr2=0x7f9b3819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 shutdown_connections 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38103510 0x7f9b381a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102b50 0x7f9b3819a3d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 --2- 172.21.15.59:0/1675580728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38101960 0x7f9b38199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 >> 172.21.15.59:0/1675580728 conn(0x7f9b380fced0 msgr2=0x7f9b38073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.989+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 shutdown_connections 2024-04-02T17:12:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:12.990+0000 7f9b3ec00640 1 -- 172.21.15.59:0/1675580728 wait complete. 2024-04-02T17:12:13.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 -- 172.21.15.59:0/1631220106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40103b50 msgr2=0x7f4f40103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 --2- 172.21.15.59:0/1631220106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40103b50 0x7f4f40103f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009900 tx=0x7f4f3002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 -- 172.21.15.59:0/1631220106 shutdown_connections 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 --2- 172.21.15.59:0/1631220106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40105700 0x7f4f4010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 --2- 172.21.15.59:0/1631220106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40104d40 0x7f4f401051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 --2- 172.21.15.59:0/1631220106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40103b50 0x7f4f40103f70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 -- 172.21.15.59:0/1631220106 >> 172.21.15.59:0/1631220106 conn(0x7f4f400ff0e0 msgr2=0x7f4f40101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 -- 172.21.15.59:0/1631220106 shutdown_connections 2024-04-02T17:12:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.239+0000 7f4f45800640 1 -- 172.21.15.59:0/1631220106 wait complete. 2024-04-02T17:12:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.240+0000 7f4f45800640 1 Processor -- start 2024-04-02T17:12:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.240+0000 7f4f45800640 1 -- start start 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.240+0000 7f4f45800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f45800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40104d40 0x7f4f4019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f45800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40105700 0x7f4f401a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f45800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010f810 con 0x7f4f40104d40 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f45800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010f690 con 0x7f4f40105700 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f45800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010f990 con 0x7f4f40103b50 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f37400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40104d40 0x7f4f4019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43796/0 (socket says 172.21.15.59:43796) 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 -- 172.21.15.59:0/3217639805 learned_addr learned my addr 172.21.15.59:0/3217639805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f44e00640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40105700 0x7f4f401a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 -- 172.21.15.59:0/3217639805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40105700 msgr2=0x7f4f401a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40105700 0x7f4f401a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 -- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40104d40 msgr2=0x7f4f4019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40104d40 0x7f4f4019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.241+0000 7f4f3fe00640 1 -- 172.21.15.59:0/3217639805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f401a1aa0 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f3fe00640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f3002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30043020 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f300054e0 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401a1d30 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30003bd0 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.242+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a21d0 con 0x7f4f40103b50 2024-04-02T17:12:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.243+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4010ae70 con 0x7f4f40103b50 2024-04-02T17:12:13.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.245+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f30046020 con 0x7f4f40103b50 2024-04-02T17:12:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.246+0000 7f4f3d600640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 0x7f4f240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.247+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4f300f24c0 con 0x7f4f40103b50 2024-04-02T17:12:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.247+0000 7f4f37400640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 0x7f4f240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.247+0000 7f4f37400640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 0x7f4f240a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f4019b390 tx=0x7f4f28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.251+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4f3004d020 con 0x7f4f40103b50 2024-04-02T17:12:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:13 smithi059 ceph-mon[31664]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:13 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/395414754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:13 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1675580728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:13 smithi164 ceph-mon[36499]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:13 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/395414754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:13 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1675580728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:13 smithi154 ceph-mon[36479]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:13 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/395414754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:13 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1675580728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.499+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f400627a0 con 0x7f4f40103b50 2024-04-02T17:12:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.500+0000 7f4f3d600640 1 -- 172.21.15.59:0/3217639805 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f4f300bede0 con 0x7f4f40103b50 2024-04-02T17:12:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:13.501 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:12:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 msgr2=0x7f4f240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 0x7f4f240a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f4019b390 tx=0x7f4f28005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 msgr2=0x7f4f40199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f3002c650 comp rx=0 tx=0).stop 2024-04-02T17:12:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 shutdown_connections 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4f240a3bb0 0x7f4f240a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f40104d40 0x7f4f4019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40105700 0x7f4f401a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 --2- 172.21.15.59:0/3217639805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f40103b50 0x7f4f40199e80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 >> 172.21.15.59:0/3217639805 conn(0x7f4f400ff0e0 msgr2=0x7f4f40108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 shutdown_connections 2024-04-02T17:12:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.504+0000 7f4f45800640 1 -- 172.21.15.59:0/3217639805 wait complete. 2024-04-02T17:12:13.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T17:10:34.529898+0000', 'modified': '2024-04-02T17:10:43.085475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24584, 'mds_1': 24560, 'mds_2': 24547, 'mds_3': 24530, 'mds_4': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/3588884595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 3588884595}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 3588884595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6833/1241853080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 1241853080}, {'type': 'v1', 'addr': '172.21.15.154:6833', 'nonce': 1241853080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/448006801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 448006801}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 448006801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/1503733829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 1503733829}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 1503733829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2083325907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2083325907}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2083325907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.164:6837/316018252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 316018252}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 316018252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.154:6837/2695634895', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 2695634895}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 2695634895}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.164:6839/1157843957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1157843957}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1157843957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.59:6839/377916146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 377916146}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 377916146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6839/3218637046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 3218637046}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 3218637046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24560, 24584, 24547, 24530, 24529]} 2024-04-02T17:12:13.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T17:12:13.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 -- 172.21.15.59:0/35437757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d41078a0 msgr2=0x7f31d4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 --2- 172.21.15.59:0/35437757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d41078a0 0x7f31d4109c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009900 tx=0x7f31cc02bc40 comp rx=0 tx=0).stop 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 -- 172.21.15.59:0/35437757 shutdown_connections 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 --2- 172.21.15.59:0/35437757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4104f70 0x7f31d4107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 --2- 172.21.15.59:0/35437757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d41078a0 0x7f31d4109c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 --2- 172.21.15.59:0/35437757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d4102640 0x7f31d4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 -- 172.21.15.59:0/35437757 >> 172.21.15.59:0/35437757 conn(0x7f31d40fbfb0 msgr2=0x7f31d40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 -- 172.21.15.59:0/35437757 shutdown_connections 2024-04-02T17:12:13.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.751+0000 7f31d9e00640 1 -- 172.21.15.59:0/35437757 wait complete. 2024-04-02T17:12:13.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.752+0000 7f31d9e00640 1 Processor -- start 2024-04-02T17:12:13.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.752+0000 7f31d9e00640 1 -- start start 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4102640 0x7f31d419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d4104f70 0x7f31d419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31d410d7f0 con 0x7f31d4102640 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f31d410d670 con 0x7f31d41078a0 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31d410d970 con 0x7f31d4104f70 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4102640 0x7f31d419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42698/0 (socket says 172.21.15.59:42698) 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d4104f70 0x7f31d419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 -- 172.21.15.59:0/86434022 learned_addr learned my addr 172.21.15.59:0/86434022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 -- 172.21.15.59:0/86434022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d4104f70 msgr2=0x7f31d419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d4104f70 0x7f31d419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 -- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4102640 msgr2=0x7f31d419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.753+0000 7f31d3400640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4102640 0x7f31d419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d3400640 1 -- 172.21.15.59:0/86434022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d41a1c40 con 0x7f31d41078a0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d3400640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02c180 tx=0x7f31cc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d8a00640 1 -- 172.21.15.59:0/86434022 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc041070 con 0x7f31d41078a0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d8a00640 1 -- 172.21.15.59:0/86434022 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc0054e0 con 0x7f31d41078a0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d8a00640 1 -- 172.21.15.59:0/86434022 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc034440 con 0x7f31d41078a0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d41a1ed0 con 0x7f31d41078a0 2024-04-02T17:12:13.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.754+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d41a22e0 con 0x7f31d41078a0 2024-04-02T17:12:13.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.757+0000 7f31dcbb5740 1 Processor -- start 2024-04-02T17:12:13.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.758+0000 7f31d8a00640 1 -- 172.21.15.59:0/86434022 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f31cc044070 con 0x7f31d41078a0 2024-04-02T17:12:13.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.758+0000 7f31dcbb5740 1 -- start start 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.758+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459125d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.758+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55845908b060 con 0x55845912b450 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558459033560 con 0x5584591262e0 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558459087010 con 0x5584591258f0 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42712/0 (socket says 172.21.15.59:42712) 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 -- 172.21.15.59:0/2507295204 learned_addr learned my addr 172.21.15.59:0/2507295204 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2000640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459125d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2000640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 -- 172.21.15.59:0/2507295204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 msgr2=0x558459125d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459125d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 -- 172.21.15.59:0/2507295204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 msgr2=0x55845912d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2a00640 1 -- 172.21.15.59:0/2507295204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558459055fd0 con 0x5584591262e0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.759+0000 7f31d2000640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31d2a00640 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f31c40114f0 tx=0x7f31c40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=707dec22743da51b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31c3e00640 1 -- 172.21.15.59:0/2507295204 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c401e070 con 0x5584591262e0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31c3e00640 1 -- 172.21.15.59:0/2507295204 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4003c30 con 0x5584591262e0 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31d8a00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 0x7f31b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31c3e00640 1 -- 172.21.15.59:0/2507295204 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c400fa10 con 0x5584591262e0 2024-04-02T17:12:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31d2000640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 0x7f31b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.760+0000 7f31d8a00640 1 -- 172.21.15.59:0/86434022 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f31cc0f1ff0 con 0x7f31d41078a0 2024-04-02T17:12:13.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- 172.21.15.59:0/2507295204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 msgr2=0x55845912ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f31c40114f0 tx=0x7f31c40119b0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- 172.21.15.59:0/2507295204 shutdown_connections 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912ade0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/2507295204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459125d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- 172.21.15.59:0/2507295204 >> 172.21.15.59:0/2507295204 conn(0x558459120e60 msgr2=0x558459121260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31d2000640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 0x7f31b80a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558459126b40 tx=0x7f31bc013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- 172.21.15.59:0/2507295204 shutdown_connections 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- 172.21.15.59:0/2507295204 wait complete. 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.761+0000 7f31dcbb5740 1 -- start start 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 Processor -- start 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x55845912fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591262e0 0x5584591303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55845912e400 con 0x5584591262e0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558459046920 con 0x55845912b450 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55845912e580 con 0x5584591258f0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31dcbb5740 10 client.0 _ll_register_callbacks cb 0x558459046310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31d3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31d2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591262e0 0x5584591303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31d3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42716/0 (socket says 172.21.15.59:42716) 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31d3400640 1 -- 172.21.15.59:0/1933655144 learned_addr learned my addr 172.21.15.59:0/1933655144 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.762+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x55845912fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 -- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 msgr2=0x55845912fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x55845912fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 -- 172.21.15.59:0/1933655144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591262e0 msgr2=0x5584591303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591262e0 0x5584591303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 -- 172.21.15.59:0/1933655144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558458d9f2d0 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31d3400640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02c220 tx=0x7f31cc004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb1a79d94236e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc04b040 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f31cc0c69f0 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4003c30 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.763+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31cc0dc050 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 10 client.24986 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558458e696b0 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc09f080 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31cc0c2120 con 0x55845912b450 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 10 client.24986 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 10 client.24986 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T17:12:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 20 client.24986 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55845913cc40 0x55845913e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 4 client.24986 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.764+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584590985a0 con 0x55845913cc40 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.765+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55845913cc40 0x55845913e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.765+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55845913cc40 0x55845913e0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b47e6fdadfde21 server_cookie=238c49f99068b662 in_seq=0 out_seq=0 2024-04-02T17:12:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.765+0000 7f31c0c00640 10 client.24986 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.766+0000 7f31c0c00640 1 -- 172.21.15.59:0/1933655144 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584590985a0 con 0x55845913cc40 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.766+0000 7f31c0c00640 10 client.24986 handle_command_reply: tid=0 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.766+0000 7f31dcbb5740 1 client.24986 shutdown 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 msgr2=0x5584591373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02c220 tx=0x7f31cc004ee0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 shutdown_connections 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55845913cc40 0x55845913e0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591262e0 0x5584591303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x5584591373e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1933655144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x55845912fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 >> 172.21.15.59:0/1933655144 conn(0x558459120e60 msgr2=0x558459120080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31c0c00640 0 client.24986 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 shutdown_connections 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1933655144 wait complete. 2024-04-02T17:12:13.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.767+0000 7f31dcbb5740 20 client.24986 trim_cache size 0 max 16384 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 Processor -- start 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 -- start start 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459121240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55845913c3f0 con 0x55845912b450 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55845913cac0 con 0x5584591262e0 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.787+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55845913d190 con 0x5584591258f0 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33302/0 (socket says 172.21.15.59:33302) 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 -- 172.21.15.59:0/3355596622 learned_addr learned my addr 172.21.15.59:0/3355596622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d2a00640 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459121240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d2000640 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 -- 172.21.15.59:0/3355596622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 msgr2=0x558459121240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459121240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 -- 172.21.15.59:0/3355596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 msgr2=0x55845912f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 -- 172.21.15.59:0/3355596622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845911c340 con 0x55845912b450 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31d3400640 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0bdfc0 tx=0x7f31cc0bc7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942c2acd6eadeb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.788+0000 7f31c3400640 1 -- 172.21.15.59:0/3355596622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0a2070 con 0x55845912b450 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31c3400640 1 -- 172.21.15.59:0/3355596622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4003c30 con 0x55845912b450 2024-04-02T17:12:13.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31c3400640 1 -- 172.21.15.59:0/3355596622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc04b070 con 0x55845912b450 2024-04-02T17:12:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 -- 172.21.15.59:0/3355596622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 msgr2=0x55845912f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0bdfc0 tx=0x7f31cc0bc7d0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 -- 172.21.15.59:0/3355596622 shutdown_connections 2024-04-02T17:12:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55845912b450 0x55845912f670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584591262e0 0x55845912f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/3355596622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591258f0 0x558459121240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 -- 172.21.15.59:0/3355596622 >> 172.21.15.59:0/3355596622 conn(0x558459120e60 msgr2=0x55845913b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 -- 172.21.15.59:0/3355596622 shutdown_connections 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.789+0000 7f31dcbb5740 1 -- 172.21.15.59:0/3355596622 wait complete. 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 -- start start 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 Processor -- start 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 0x5584591363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x558459132f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55845913df30 con 0x5584591258f0 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55845913d860 con 0x55845912b450 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55845913e600 con 0x5584591262e0 2024-04-02T17:12:13.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.790+0000 7f31dcbb5740 10 client.0 _ll_register_callbacks cb 0x7f31cc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 0x5584591363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43858/0 (socket says 172.21.15.59:43858) 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2000640 1 -- 172.21.15.59:0/1407914457 learned_addr learned my addr 172.21.15.59:0/1407914457 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d3400640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x558459132f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 -- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 msgr2=0x558459136910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 -- 172.21.15.59:0/1407914457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 msgr2=0x558459132f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x558459132f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2a00640 1 -- 172.21.15.59:0/1407914457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558459133770 con 0x5584591258f0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d2000640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.791+0000 7f31d3400640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x558459132f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31d2a00640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 0x5584591363d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x558459121dd0 tx=0x7f31c4009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aec298ae186e2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c402e020 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4003c30 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f31c4004cc0 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31c402ae60 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31dcbb5740 10 client.14880 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584591a3ab0 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.792+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4003c30 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31c4032020 con 0x5584591258f0 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 10 client.14880 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 10 client.14880 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 20 client.14880 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5584591a8550 0x5584591a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 4 client.14880 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "status"}) -- 0x5584590928f0 con 0x5584591a8550 2024-04-02T17:12:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.793+0000 7f31d2000640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5584591a8550 0x5584591a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31d2000640 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5584591a8550 0x5584591a4990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76e0eb50f5703a99 server_cookie=108cbd644d997f2e in_seq=0 out_seq=0 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31c0c00640 10 client.14880 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31c0c00640 1 -- 172.21.15.59:0/1407914457 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5584590928f0 con 0x5584591a8550 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31c0c00640 10 client.14880 handle_command_reply: tid=0 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31dcbb5740 1 client.14880 shutdown 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 msgr2=0x5584591363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 0x5584591363d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x558459121dd0 tx=0x7f31c4009320 comp rx=0 tx=0).stop 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.794+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 shutdown_connections 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5584591a8550 0x5584591a4990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55845912b450 0x558459132f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584591262e0 0x558459136910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31c0c00640 0 client.14880 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 --2- 172.21.15.59:0/1407914457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584591258f0 0x5584591363d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 >> 172.21.15.59:0/1407914457 conn(0x558459120e60 msgr2=0x55845912f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 shutdown_connections 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 1 -- 172.21.15.59:0/1407914457 wait complete. 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31dcbb5740 20 client.14880 trim_cache size 0 max 16384 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 4, 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24529, 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 91.718839431000006, 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 120.11141882699999 2024-04-02T17:12:13.799 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 msgr2=0x7f31b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 0x7f31b80a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558459126b40 tx=0x7f31bc013040 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 msgr2=0x7f31d41a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.795+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02c180 tx=0x7f31cc02c660 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 shutdown_connections 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d41078a0 0x7f31d41a15d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31d4104f70 0x7f31d419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31b80a3c30 0x7f31b80a60f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 --2- 172.21.15.59:0/86434022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31d4102640 0x7f31d419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 >> 172.21.15.59:0/86434022 conn(0x7f31d40fbfb0 msgr2=0x7f31d4101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 shutdown_connections 2024-04-02T17:12:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:13.796+0000 7f31d9e00640 1 -- 172.21.15.59:0/86434022 wait complete. 2024-04-02T17:12:13.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T17:12:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.037+0000 7efe77e00640 1 -- 172.21.15.59:0/1846422128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe78104e10 msgr2=0x7efe78105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.037+0000 7efe77e00640 1 --2- 172.21.15.59:0/1846422128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe78104e10 0x7efe78105290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe60009900 tx=0x7efe6002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 -- 172.21.15.59:0/1846422128 shutdown_connections 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 --2- 172.21.15.59:0/1846422128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe78104e10 0x7efe78105290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 --2- 172.21.15.59:0/1846422128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe781057d0 0x7efe7810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 --2- 172.21.15.59:0/1846422128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78103c20 0x7efe78104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 -- 172.21.15.59:0/1846422128 >> 172.21.15.59:0/1846422128 conn(0x7efe780ff1b0 msgr2=0x7efe781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 -- 172.21.15.59:0/1846422128 shutdown_connections 2024-04-02T17:12:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.038+0000 7efe77e00640 1 -- 172.21.15.59:0/1846422128 wait complete. 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 Processor -- start 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 -- start start 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe78103c20 0x7efe78199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78104e10 0x7efe7819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810f8f0 con 0x7efe781057d0 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810f770 con 0x7efe78104e10 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810fa70 con 0x7efe78103c20 2024-04-02T17:12:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33332/0 (socket says 172.21.15.59:33332) 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe77400640 1 -- 172.21.15.59:0/3438978283 learned_addr learned my addr 172.21.15.59:0/3438978283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.039+0000 7efe76a00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe78103c20 0x7efe78199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe6fe00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78104e10 0x7efe7819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 -- 172.21.15.59:0/3438978283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe78103c20 msgr2=0x7efe78199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe78103c20 0x7efe78199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 -- 172.21.15.59:0/3438978283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78104e10 msgr2=0x7efe7819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78104e10 0x7efe7819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 -- 172.21.15.59:0/3438978283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe781a1b00 con 0x7efe781057d0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.040+0000 7efe77400640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe7000eb10 tx=0x7efe7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.041+0000 7efe7ca00640 1 -- 172.21.15.59:0/3438978283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004b20 con 0x7efe781057d0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.041+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe781a1df0 con 0x7efe781057d0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.041+0000 7efe7ca00640 1 -- 172.21.15.59:0/3438978283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70004cc0 con 0x7efe781057d0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.041+0000 7efe7ca00640 1 -- 172.21.15.59:0/3438978283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70010760 con 0x7efe781057d0 2024-04-02T17:12:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.041+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a22b0 con 0x7efe781057d0 2024-04-02T17:12:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.044+0000 7efe80a4a740 1 Processor -- start 2024-04-02T17:12:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.044+0000 7efe7ca00640 1 -- 172.21.15.59:0/3438978283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efe70005020 con 0x7efe781057d0 2024-04-02T17:12:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.044+0000 7efe80a4a740 1 -- start start 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fe2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650f5f40060 con 0x5650f5fdab10 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ee8560 con 0x5650f5fdb500 2024-04-02T17:12:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe80a4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650f5f3c010 con 0x5650f5fe0670 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33344/0 (socket says 172.21.15.59:33344) 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe6fe00640 1 -- 172.21.15.59:0/2071565794 learned_addr learned my addr 172.21.15.59:0/2071565794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe76a00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.045+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fe2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe7ca00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 0x7efe5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 -- 172.21.15.59:0/2071565794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 msgr2=0x5650f5fe2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fe2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 -- 172.21.15.59:0/2071565794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 msgr2=0x5650f5fe0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 -- 172.21.15.59:0/2071565794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f5f0afd0 con 0x5650f5fdab10 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe76a00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe7ca00640 1 -- 172.21.15.59:0/3438978283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efe700d0ae0 con 0x7efe781057d0 2024-04-02T17:12:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe60005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d453c89e427c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe77400640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 0x7efe5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe67e00640 1 -- 172.21.15.59:0/2071565794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6002cd20 con 0x5650f5fdab10 2024-04-02T17:12:14.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe67e00640 1 -- 172.21.15.59:0/2071565794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe600046c0 con 0x5650f5fdab10 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.046+0000 7efe67e00640 1 -- 172.21.15.59:0/2071565794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004f60 con 0x5650f5fdab10 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 -- 172.21.15.59:0/2071565794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 msgr2=0x5650f5fdaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe77400640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 0x7efe5c0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe70005ba0 tx=0x7efe7000efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe60005b50 comp rx=0 tx=0).stop 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 -- 172.21.15.59:0/2071565794 shutdown_connections 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fe2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2071565794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdab10 0x5650f5fdaf30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 -- 172.21.15.59:0/2071565794 >> 172.21.15.59:0/2071565794 conn(0x5650f5fd6080 msgr2=0x5650f5fd6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 -- 172.21.15.59:0/2071565794 shutdown_connections 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.047+0000 7efe80a4a740 1 -- 172.21.15.59:0/2071565794 wait complete. 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 -- start start 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 Processor -- start 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fdab10 0x5650f5fe5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650f5fe35c0 con 0x5650f5fe0670 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650f5efb920 con 0x5650f5fdb500 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.048+0000 7efe80a4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650f5fe3740 con 0x5650f5fdab10 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe80a4a740 10 client.0 _ll_register_callbacks cb 0x5650f5efb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33348/0 (socket says 172.21.15.59:33348) 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 -- 172.21.15.59:0/3677284839 learned_addr learned my addr 172.21.15.59:0/3677284839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fdab10 0x5650f5fe5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe76a00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 -- 172.21.15.59:0/3677284839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fdab10 msgr2=0x5650f5fe5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fdab10 0x5650f5fe5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 -- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 msgr2=0x5650f5fe55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 -- 172.21.15.59:0/3677284839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f5c542d0 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe76a00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe6fe00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe600096c0 tx=0x7efe6002df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e490e43036aa4276 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.049+0000 7efe64c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004fe0 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efe60005180 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe600046c0 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe60005380 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe80a4a740 10 client.14892 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650f5d1e6b0 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.050+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe600046c0 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe6005a020 con 0x5650f5fe0670 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 10 client.14892 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 10 client.14892 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T17:12:14.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 20 client.14892 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f5ff1940 0x5650f5ff2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 4 client.14892 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:14.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f5f4d5a0 con 0x5650f5ff1940 2024-04-02T17:12:14.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.051+0000 7efe76a00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f5ff1940 0x5650f5ff2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.052+0000 7efe76a00640 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f5ff1940 0x5650f5ff2db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f90f1cde3a28056d server_cookie=708263573e883972 in_seq=0 out_seq=0 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.052+0000 7efe64c00640 10 client.14892 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe64c00640 1 -- 172.21.15.59:0/3677284839 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650f5f4d5a0 con 0x5650f5ff1940 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe64c00640 10 client.14892 handle_command_reply: tid=0 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 client.14892 shutdown 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 msgr2=0x5650f5fec600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe600096c0 tx=0x7efe6002df90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 shutdown_connections 2024-04-02T17:12:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f5ff1940 0x5650f5ff2db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fe0670 0x5650f5fec600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdb500 0x5650f5fe55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe64c00640 0 client.14892 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 --2- 172.21.15.59:0/3677284839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fdab10 0x5650f5fe5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 >> 172.21.15.59:0/3677284839 conn(0x5650f5fd6080 msgr2=0x5650f5fd51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 shutdown_connections 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.053+0000 7efe80a4a740 1 -- 172.21.15.59:0/3677284839 wait complete. 2024-04-02T17:12:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.054+0000 7efe80a4a740 20 client.14892 trim_cache size 0 max 16384 2024-04-02T17:12:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.074+0000 7efe80a4a740 1 Processor -- start 2024-04-02T17:12:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 -- start start 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff10f0 con 0x5650f5fdb500 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff17c0 con 0x5650f5fdab10 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe80a4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff1e90 con 0x5650f5fe0670 2024-04-02T17:12:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42790/0 (socket says 172.21.15.59:42790) 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe6fe00640 1 -- 172.21.15.59:0/2665942166 learned_addr learned my addr 172.21.15.59:0/2665942166 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.075+0000 7efe76a00640 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 -- 172.21.15.59:0/2665942166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 msgr2=0x5650f5fec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 -- 172.21.15.59:0/2665942166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 msgr2=0x5650f5fe5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 -- 172.21.15.59:0/2665942166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f5fdfd60 con 0x5650f5fdab10 2024-04-02T17:12:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.076+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe60042d70 tx=0x7efe60005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba63f9e509f4cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.077+0000 7efe67400640 1 -- 172.21.15.59:0/2665942166 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004fe0 con 0x5650f5fdab10 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.077+0000 7efe67400640 1 -- 172.21.15.59:0/2665942166 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60005180 con 0x5650f5fdab10 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.077+0000 7efe67400640 1 -- 172.21.15.59:0/2665942166 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60005520 con 0x5650f5fdab10 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- 172.21.15.59:0/2665942166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 msgr2=0x5650f5ff0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe60042d70 tx=0x7efe60005b50 comp rx=0 tx=0).stop 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- 172.21.15.59:0/2665942166 shutdown_connections 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5fec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2665942166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5ff0af0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- 172.21.15.59:0/2665942166 >> 172.21.15.59:0/2665942166 conn(0x5650f5fd6080 msgr2=0x5650f5fe3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- 172.21.15.59:0/2665942166 shutdown_connections 2024-04-02T17:12:14.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- 172.21.15.59:0/2665942166 wait complete. 2024-04-02T17:12:14.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 -- start start 2024-04-02T17:12:14.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.078+0000 7efe80a4a740 1 Processor -- start 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5feb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff2c30 con 0x5650f5fdb500 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff2560 con 0x5650f5fdab10 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650f5ff3300 con 0x5650f5fe0670 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe80a4a740 10 client.0 _ll_register_callbacks cb 0x7efe60004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe6fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42802/0 (socket says 172.21.15.59:42802) 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe6fe00640 1 -- 172.21.15.59:0/2060886218 learned_addr learned my addr 172.21.15.59:0/2060886218 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.079+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5feb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 -- 172.21.15.59:0/2060886218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 msgr2=0x5650f5feb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5feb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 -- 172.21.15.59:0/2060886218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 msgr2=0x5650f5fe8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 -- 172.21.15.59:0/2060886218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f5febc10 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe6fe00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe60005c40 tx=0x7efe60004eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe932ae8e5895f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe64c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60005520 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe60005180 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.080+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efe600056c0 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe6005c8c0 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 10 client.25010 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650f6057d00 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60005180 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe6005a040 con 0x5650f5fdab10 2024-04-02T17:12:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 10 client.25010 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 10 client.25010 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 20 client.25010 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f605c430 0x5650f605b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 4 client.25010 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.081+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "status"}) -- 0x5650f5f478f0 con 0x5650f605c430 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.082+0000 7efe76a00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f605c430 0x5650f605b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.082+0000 7efe76a00640 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f605c430 0x5650f605b140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=593c47efa43dfd82 server_cookie=d39fec1f8211c92b in_seq=0 out_seq=0 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.082+0000 7efe64c00640 10 client.25010 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe64c00640 1 -- 172.21.15.59:0/2060886218 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5650f5f478f0 con 0x5650f605c430 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe64c00640 10 client.25010 handle_command_reply: tid=0 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 client.25010 shutdown 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 msgr2=0x5650f5fe80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe60005c40 tx=0x7efe60004eb0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 shutdown_connections 2024-04-02T17:12:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5650f605c430 0x5650f605b140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650f5fdb500 0x5650f5fe8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650f5fe0670 0x5650f5feb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 --2- 172.21.15.59:0/2060886218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650f5fdab10 0x5650f5fe80c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe64c00640 0 client.25010 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 >> 172.21.15.59:0/2060886218 conn(0x5650f5fd6080 msgr2=0x5650f5fd6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 shutdown_connections 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 1 -- 172.21.15.59:0/2060886218 wait complete. 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.083+0000 7efe80a4a740 20 client.25010 trim_cache size 0 max 16384 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:14.086 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 3, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24530, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 93.025202217, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 117.664802095 2024-04-02T17:12:14.087 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 msgr2=0x7efe5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 0x7efe5c0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe70005ba0 tx=0x7efe7000efd0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 msgr2=0x7efe781a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe7000eb10 tx=0x7efe7000c2d0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 shutdown_connections 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe5c0a3bb0 0x7efe5c0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe781057d0 0x7efe781a1490 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe78104e10 0x7efe7819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 --2- 172.21.15.59:0/3438978283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe78103c20 0x7efe78199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 >> 172.21.15.59:0/3438978283 conn(0x7efe780ff1b0 msgr2=0x7efe78108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 shutdown_connections 2024-04-02T17:12:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.084+0000 7efe77e00640 1 -- 172.21.15.59:0/3438978283 wait complete. 2024-04-02T17:12:14.101 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T17:12:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 -- 172.21.15.59:0/415640677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440109bd0 msgr2=0x7f5440110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 --2- 172.21.15.59:0/415640677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440109bd0 0x7f5440110460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5438009900 tx=0x7f543802bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 -- 172.21.15.59:0/415640677 shutdown_connections 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 --2- 172.21.15.59:0/415640677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440109bd0 0x7f5440110460 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 --2- 172.21.15.59:0/415640677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f5440109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 --2- 172.21.15.59:0/415640677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440108020 0x7f5440108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 -- 172.21.15.59:0/415640677 >> 172.21.15.59:0/415640677 conn(0x7f5440076fd0 msgr2=0x7f54400793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 -- 172.21.15.59:0/415640677 shutdown_connections 2024-04-02T17:12:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.311+0000 7f543fe00640 1 -- 172.21.15.59:0/415640677 wait complete. 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 Processor -- start 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 -- start start 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 0x7f544019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f544019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440109bd0 0x7f54401a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5440113d00 con 0x7f5440108020 2024-04-02T17:12:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5440113b80 con 0x7f5440109210 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.312+0000 7f543fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5440113e80 con 0x7f5440109bd0 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 0x7f544019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f544019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 0x7f544019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33374/0 (socket says 172.21.15.59:33374) 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543ea00640 1 -- 172.21.15.59:0/1850054666 learned_addr learned my addr 172.21.15.59:0/1850054666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543f400640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440109bd0 0x7f54401a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 -- 172.21.15.59:0/1850054666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440109bd0 msgr2=0x7f54401a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440109bd0 0x7f54401a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 -- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 msgr2=0x7f544019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 0x7f544019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 -- 172.21.15.59:0/1850054666 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a5e80 con 0x7f5440109210 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.313+0000 7f543e000640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f544019e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f543000eb00 tx=0x7f543000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.314+0000 7f5437400640 1 -- 172.21.15.59:0/1850054666 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430010430 con 0x7f5440109210 2024-04-02T17:12:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.314+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a6170 con 0x7f5440109210 2024-04-02T17:12:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.314+0000 7f5437400640 1 -- 172.21.15.59:0/1850054666 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5430010ab0 con 0x7f5440109210 2024-04-02T17:12:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.314+0000 7f5437400640 1 -- 172.21.15.59:0/1850054666 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430017bb0 con 0x7f5440109210 2024-04-02T17:12:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.314+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a6710 con 0x7f5440109210 2024-04-02T17:12:14.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 Processor -- start 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 -- start start 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f5437400640 1 -- 172.21.15.59:0/1850054666 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5430010c60 con 0x7f5440109210 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bdd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bddfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6bd3d060 con 0x563e6bddd670 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e6bce5560 con 0x563e6bdd7b10 2024-04-02T17:12:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.317+0000 7f54489e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e6bd39010 con 0x563e6bdd8500 2024-04-02T17:12:14.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bddfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43958/0 (socket says 172.21.15.59:43958) 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bdd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 -- 172.21.15.59:0/1574131095 learned_addr learned my addr 172.21.15.59:0/1574131095 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 -- 172.21.15.59:0/1574131095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 msgr2=0x563e6bdd7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bdd7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 -- 172.21.15.59:0/1574131095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 msgr2=0x563e6bddfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bddfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.318+0000 7f543ea00640 1 -- 172.21.15.59:0/1574131095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6bd07fd0 con 0x563e6bdd8500 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f543ea00640 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5428009d70 tx=0x7f5428011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbec3a34d41553fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f542fe00640 1 -- 172.21.15.59:0/1574131095 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428004220 con 0x563e6bdd8500 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f542fe00640 1 -- 172.21.15.59:0/1574131095 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54280043c0 con 0x563e6bdd8500 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f542fe00640 1 -- 172.21.15.59:0/1574131095 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428019950 con 0x563e6bdd8500 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f5437400640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f5437400640 1 -- 172.21.15.59:0/1850054666 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f54300d0e00 con 0x7f5440109210 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.319+0000 7f543f400640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- 172.21.15.59:0/1574131095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 msgr2=0x563e6bddd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5428009d70 tx=0x7f5428011810 comp rx=0 tx=0).stop 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- 172.21.15.59:0/1574131095 shutdown_connections 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bddfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bddd000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 --2- 172.21.15.59:0/1574131095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bdd7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- 172.21.15.59:0/1574131095 >> 172.21.15.59:0/1574131095 conn(0x563e6bdd3080 msgr2=0x563e6bdd3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- 172.21.15.59:0/1574131095 shutdown_connections 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- 172.21.15.59:0/1574131095 wait complete. 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f543f400640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5438036040 tx=0x7f543802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.320+0000 7f54489e0740 1 -- start start 2024-04-02T17:12:14.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 Processor -- start 2024-04-02T17:12:14.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd7b10 0x563e6bde1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd8500 0x563e6bde24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6bde0620 con 0x563e6bddd670 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e6bcf8920 con 0x563e6bdd8500 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e6bde07a0 con 0x563e6bdd7b10 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f54489e0740 10 client.0 _ll_register_callbacks cb 0x563e6bcf8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.321+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33388/0 (socket says 172.21.15.59:33388) 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 -- 172.21.15.59:0/1987406361 learned_addr learned my addr 172.21.15.59:0/1987406361 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543ea00640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd7b10 0x563e6bde1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543e000640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd8500 0x563e6bde24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 -- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd7b10 msgr2=0x563e6bde1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd7b10 0x563e6bde1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 -- 172.21.15.59:0/1987406361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd8500 msgr2=0x563e6bde24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd8500 0x563e6bde24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 -- 172.21.15.59:0/1987406361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6ba512d0 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f543f400640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f543802c8e0 tx=0x7f5438009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b3907b46d9c0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.322+0000 7f542cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438045070 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5438004480 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54280043c0 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54380035f0 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f54489e0740 10 client.14910 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e6bb1b6b0 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.323+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54280043c0 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f543805c020 con 0x563e6bddd670 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 10 client.14910 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 10 client.14910 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T17:12:14.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 20 client.14910 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6bdeeda0 0x563e6bdf0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 4 client.14910 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6bd4a5a0 con 0x563e6bdeeda0 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.324+0000 7f543ea00640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6bdeeda0 0x563e6bdf0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.325+0000 7f543ea00640 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6bdeeda0 0x563e6bdf0210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c2579a7d0188e46 server_cookie=b9c9ba6beeb7f2fb in_seq=0 out_seq=0 2024-04-02T17:12:14.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.325+0000 7f542cc00640 10 client.14910 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:14.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:14 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3217639805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:14.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.326+0000 7f542cc00640 1 -- 172.21.15.59:0/1987406361 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e6bd4a5a0 con 0x563e6bdeeda0 2024-04-02T17:12:14.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.326+0000 7f542cc00640 10 client.14910 handle_command_reply: tid=0 2024-04-02T17:12:14.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.326+0000 7f54489e0740 1 client.14910 shutdown 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.326+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 msgr2=0x563e6bde94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.326+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f543802c8e0 tx=0x7f5438009260 comp rx=0 tx=0).stop 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 shutdown_connections 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6bdeeda0 0x563e6bdf0210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde94f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd8500 0x563e6bde24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 --2- 172.21.15.59:0/1987406361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd7b10 0x563e6bde1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f542cc00640 0 client.14910 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 >> 172.21.15.59:0/1987406361 conn(0x563e6bdd3080 msgr2=0x563e6bdd21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 shutdown_connections 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 1 -- 172.21.15.59:0/1987406361 wait complete. 2024-04-02T17:12:14.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.327+0000 7f54489e0740 20 client.14910 trim_cache size 0 max 16384 2024-04-02T17:12:14.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 Processor -- start 2024-04-02T17:12:14.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 -- start start 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdee550 con 0x563e6bddd670 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdeec20 con 0x563e6bdd7b10 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.347+0000 7f54489e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdef2f0 con 0x563e6bdd8500 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33404/0 (socket says 172.21.15.59:33404) 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 -- 172.21.15.59:0/434289036 learned_addr learned my addr 172.21.15.59:0/434289036 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543e000640 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543ea00640 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 -- 172.21.15.59:0/434289036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 msgr2=0x563e6bde94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 -- 172.21.15.59:0/434289036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 msgr2=0x563e6bde0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 -- 172.21.15.59:0/434289036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6bdd16c0 con 0x563e6bddd670 2024-04-02T17:12:14.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.348+0000 7f543f400640 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f543805ead0 tx=0x7f5438004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c3a251c89e9fab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f542f400640 1 -- 172.21.15.59:0/434289036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543805e660 con 0x563e6bddd670 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f542f400640 1 -- 172.21.15.59:0/434289036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543805e800 con 0x563e6bddd670 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f542f400640 1 -- 172.21.15.59:0/434289036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034920 con 0x563e6bddd670 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 -- 172.21.15.59:0/434289036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 msgr2=0x563e6bdd3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f543805ead0 tx=0x7f5438004340 comp rx=0 tx=0).stop 2024-04-02T17:12:14.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 -- 172.21.15.59:0/434289036 shutdown_connections 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bdd3460 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 --2- 172.21.15.59:0/434289036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.349+0000 7f54489e0740 1 -- 172.21.15.59:0/434289036 >> 172.21.15.59:0/434289036 conn(0x563e6bdd3080 msgr2=0x563e6bdcdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 -- 172.21.15.59:0/434289036 shutdown_connections 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 -- 172.21.15.59:0/434289036 wait complete. 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 -- start start 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 Processor -- start 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.350+0000 7f54489e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f54489e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f54489e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdf0090 con 0x563e6bddd670 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f54489e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdef9c0 con 0x563e6bdd7b10 2024-04-02T17:12:14.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f54489e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e6bdf0760 con 0x563e6bdd8500 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f54489e0740 10 client.0 _ll_register_callbacks cb 0x7f543802cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33416/0 (socket says 172.21.15.59:33416) 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543f400640 1 -- 172.21.15.59:0/3198403696 learned_addr learned my addr 172.21.15.59:0/3198403696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543ea00640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 -- 172.21.15.59:0/3198403696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 msgr2=0x563e6bde4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 -- 172.21.15.59:0/3198403696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 msgr2=0x563e6bde7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543e000640 1 -- 172.21.15.59:0/3198403696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6bde83d0 con 0x563e6bdd8500 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.351+0000 7f543f400640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f543e000640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54300ce8f0 tx=0x7f54300d43f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe4264b0f8e4099 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543009d080 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f54300a10d0 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543805e800 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54300a1430 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f54489e0740 10 client.25036 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e6bde9790 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543805e800 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.352+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f543009c560 con 0x563e6bdd8500 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 10 client.25036 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 10 client.25036 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 20 client.25036 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6be596b0 0x563e6be583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 4 client.25036 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:14.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "status"}) -- 0x563e6bd448f0 con 0x563e6be596b0 2024-04-02T17:12:14.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f543ea00640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6be596b0 0x563e6be583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.353+0000 7f543ea00640 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6be596b0 0x563e6be583c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97d32f6cf452a23 server_cookie=57b42e0d0da75751 in_seq=0 out_seq=0 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f542cc00640 10 client.25036 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f542cc00640 1 -- 172.21.15.59:0/3198403696 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563e6bd448f0 con 0x563e6be596b0 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f542cc00640 10 client.25036 handle_command_reply: tid=0 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 client.25036 shutdown 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 msgr2=0x563e6bde4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54300ce8f0 tx=0x7f54300d43f0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 shutdown_connections 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x563e6be596b0 0x563e6be583c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e6bddd670 0x563e6bde7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e6bdd8500 0x563e6bde4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f542cc00640 0 client.25036 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 --2- 172.21.15.59:0/3198403696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e6bdd7b10 0x563e6bde4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 >> 172.21.15.59:0/3198403696 conn(0x563e6bdd3080 msgr2=0x563e6bdd3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 shutdown_connections 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 1 -- 172.21.15.59:0/3198403696 wait complete. 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.354+0000 7f54489e0740 20 client.25036 trim_cache size 0 max 16384 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2024-04-02T17:12:14.358 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24547, 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 94.307161629999996, 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 115.183196353 2024-04-02T17:12:14.359 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 msgr2=0x7f54240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5438036040 tx=0x7f543802df90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 msgr2=0x7f544019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f544019e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f543000eb00 tx=0x7f543000c2d0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 shutdown_connections 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440109bd0 0x7f54401a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440109210 0x7f544019e7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 --2- 172.21.15.59:0/1850054666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440108020 0x7f544019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 >> 172.21.15.59:0/1850054666 conn(0x7f5440076fd0 msgr2=0x7f544010c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 shutdown_connections 2024-04-02T17:12:14.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.355+0000 7f543fe00640 1 -- 172.21.15.59:0/1850054666 wait complete. 2024-04-02T17:12:14.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:14 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3217639805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:14.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-02T17:12:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:14 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3217639805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 -- 172.21.15.59:0/260324442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd8074560 msgr2=0x7f8bd80749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 --2- 172.21.15.59:0/260324442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd8074560 0x7f8bd80749e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009860 tx=0x7f8bc402bb90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 -- 172.21.15.59:0/260324442 shutdown_connections 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 --2- 172.21.15.59:0/260324442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd81098c0 0x7f8bd810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 --2- 172.21.15.59:0/260324442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd8074560 0x7f8bd80749e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 --2- 172.21.15.59:0/260324442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8075b40 0x7f8bd8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 -- 172.21.15.59:0/260324442 >> 172.21.15.59:0/260324442 conn(0x7f8bd80fcf70 msgr2=0x7f8bd80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 -- 172.21.15.59:0/260324442 shutdown_connections 2024-04-02T17:12:14.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.588+0000 7f8bdec00640 1 -- 172.21.15.59:0/260324442 wait complete. 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.589+0000 7f8bdec00640 1 Processor -- start 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 -- start start 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd8075b40 0x7f8bd819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd81098c0 0x7f8bd81a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810f860 con 0x7f8bd8074560 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810f6e0 con 0x7f8bd81098c0 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810f9e0 con 0x7f8bd8075b40 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33434/0 (socket says 172.21.15.59:33434) 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdd800640 1 -- 172.21.15.59:0/776189917 learned_addr learned my addr 172.21.15.59:0/776189917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bdce00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd8075b40 0x7f8bd819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.590+0000 7f8bde200640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd81098c0 0x7f8bd81a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 -- 172.21.15.59:0/776189917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd8075b40 msgr2=0x7f8bd819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd8075b40 0x7f8bd819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 -- 172.21.15.59:0/776189917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd81098c0 msgr2=0x7f8bd81a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd81098c0 0x7f8bd81a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 -- 172.21.15.59:0/776189917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd81a1b20 con 0x7f8bd8074560 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdd800640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e9c0 tx=0x7f8bcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bd6000640 1 -- 172.21.15.59:0/776189917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00cde0 con 0x7f8bd8074560 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd81a1e10 con 0x7f8bd8074560 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bd6000640 1 -- 172.21.15.59:0/776189917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc0035f0 con 0x7f8bd8074560 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.591+0000 7f8bd6000640 1 -- 172.21.15.59:0/776189917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010650 con 0x7f8bd8074560 2024-04-02T17:12:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.592+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a2280 con 0x7f8bd8074560 2024-04-02T17:12:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.595+0000 7f8be2fe9740 1 Processor -- start 2024-04-02T17:12:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.595+0000 7f8be2fe9740 1 -- start start 2024-04-02T17:12:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.595+0000 7f8bd6000640 1 -- 172.21.15.59:0/776189917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bcc0037a0 con 0x7f8bd8074560 2024-04-02T17:12:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6e500 0x55ac64b73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b73670 0x55ac64b75b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac64ad3060 con 0x55ac64b6db10 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac64a7b560 con 0x55ac64b73670 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac64acf010 con 0x55ac64b6e500 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33438/0 (socket says 172.21.15.59:33438) 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 -- 172.21.15.59:0/1298931980 learned_addr learned my addr 172.21.15.59:0/1298931980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bdce00640 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b73670 0x55ac64b75b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bdd800640 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6e500 0x55ac64b73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 -- 172.21.15.59:0/1298931980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6e500 msgr2=0x55ac64b73000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6e500 0x55ac64b73000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 -- 172.21.15.59:0/1298931980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b73670 msgr2=0x55ac64b75b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b73670 0x55ac64b75b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.596+0000 7f8bde200640 1 -- 172.21.15.59:0/1298931980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac64a9dfd0 con 0x55ac64b6db10 2024-04-02T17:12:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bde200640 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009fd0 tx=0x7f8bd00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3c20ee98451c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bcaa00640 1 -- 172.21.15.59:0/1298931980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0004d20 con 0x55ac64b6db10 2024-04-02T17:12:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bcaa00640 1 -- 172.21.15.59:0/1298931980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0004050 con 0x55ac64b6db10 2024-04-02T17:12:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bcaa00640 1 -- 172.21.15.59:0/1298931980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0005260 con 0x55ac64b6db10 2024-04-02T17:12:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bd6000640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 0x7f8bc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.597+0000 7f8bd6000640 1 -- 172.21.15.59:0/776189917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bcc0d07b0 con 0x7f8bd8074560 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8bdd800640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 0x7f8bc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1298931980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 msgr2=0x55ac64b6df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009fd0 tx=0x7f8bd00118d0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1298931980 shutdown_connections 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b73670 0x55ac64b75b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6e500 0x55ac64b73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1298931980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6db10 0x55ac64b6df30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1298931980 >> 172.21.15.59:0/1298931980 conn(0x55ac64b69080 msgr2=0x55ac64b69480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1298931980 shutdown_connections 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1298931980 wait complete. 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.598+0000 7f8bdd800640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 0x7f8bc00a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0993a0 tx=0x7f8bcc005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 -- start start 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 Processor -- start 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6db10 0x55ac64b77fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 0x55ac64b784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b76620 con 0x55ac64b73670 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac64a8e920 con 0x55ac64b6e500 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b767a0 con 0x55ac64b6db10 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.599+0000 7f8be2fe9740 10 client.0 _ll_register_callbacks cb 0x55ac64a8e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bde200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 0x55ac64b784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bde200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33452/0 (socket says 172.21.15.59:33452) 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bde200640 1 -- 172.21.15.59:0/2127117965 learned_addr learned my addr 172.21.15.59:0/2127117965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bde200640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6db10 0x55ac64b77fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 -- 172.21.15.59:0/2127117965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6db10 msgr2=0x55ac64b77fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6db10 0x55ac64b77fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 -- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 msgr2=0x55ac64b7f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 -- 172.21.15.59:0/2127117965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac647e72d0 con 0x55ac64b6e500 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.600+0000 7f8bdce00640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 0x55ac64b784e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40349e0 tx=0x7f8bc4005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d0291aa93620d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4043070 con 0x55ac64b6e500 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bc4004040 con 0x55ac64b6e500 2024-04-02T17:12:14.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd0004050 con 0x55ac64b6e500 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bc404e050 con 0x55ac64b6e500 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8be2fe9740 10 client.25040 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac648b16b0 con 0x55ac64b6e500 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.601+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0004050 con 0x55ac64b6e500 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bc4032760 con 0x55ac64b6e500 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 10 client.25040 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 10 client.25040 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 20 client.25040 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64b84cc0 0x55ac64b86130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 4 client.25040 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac64ae05a0 con 0x55ac64b84cc0 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.602+0000 7f8bde200640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64b84cc0 0x55ac64b86130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.603+0000 7f8bde200640 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64b84cc0 0x55ac64b86130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3870007da0a15b9d server_cookie=79eb878cc4741b10 in_seq=0 out_seq=0 2024-04-02T17:12:14.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.603+0000 7f8bbf400640 10 client.25040 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8bbf400640 1 -- 172.21.15.59:0/2127117965 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac64ae05a0 con 0x55ac64b84cc0 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8bbf400640 10 client.25040 handle_command_reply: tid=0 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 client.25040 shutdown 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 msgr2=0x55ac64b784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 0x55ac64b784e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40349e0 tx=0x7f8bc4005d90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 shutdown_connections 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64b84cc0 0x55ac64b86130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6e500 0x55ac64b784e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/2127117965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b6db10 0x55ac64b77fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 >> 172.21.15.59:0/2127117965 conn(0x55ac64b69080 msgr2=0x55ac64b681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8bbf400640 0 client.25040 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 shutdown_connections 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 1 -- 172.21.15.59:0/2127117965 wait complete. 2024-04-02T17:12:14.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.604+0000 7f8be2fe9740 20 client.25040 trim_cache size 0 max 16384 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.624+0000 7f8be2fe9740 1 Processor -- start 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 -- start start 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b73670 0x55ac64b67930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b84470 con 0x55ac64b6e500 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b84b40 con 0x55ac64b6db10 2024-04-02T17:12:14.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b85210 con 0x55ac64b73670 2024-04-02T17:12:14.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33468/0 (socket says 172.21.15.59:33468) 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8bde200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b73670 0x55ac64b67930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8bdce00640 1 -- 172.21.15.59:0/1673872819 learned_addr learned my addr 172.21.15.59:0/1673872819 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.625+0000 7f8bde200640 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 -- 172.21.15.59:0/1673872819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b73670 msgr2=0x55ac64b67930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b73670 0x55ac64b67930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 -- 172.21.15.59:0/1673872819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 msgr2=0x55ac64b72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 -- 172.21.15.59:0/1673872819 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac64b676c0 con 0x55ac64b6e500 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bdce00640 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009250 tx=0x7f8bc402bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e63bfb693f0d332 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bca000640 1 -- 172.21.15.59:0/1673872819 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc404b020 con 0x55ac64b6e500 2024-04-02T17:12:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bca000640 1 -- 172.21.15.59:0/1673872819 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc4005510 con 0x55ac64b6e500 2024-04-02T17:12:14.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.626+0000 7f8bca000640 1 -- 172.21.15.59:0/1673872819 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4043070 con 0x55ac64b6e500 2024-04-02T17:12:14.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1673872819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 msgr2=0x55ac64b7f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009250 tx=0x7f8bc402bb90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1673872819 shutdown_connections 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b6e500 0x55ac64b7f4e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b73670 0x55ac64b67930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/1673872819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1673872819 >> 172.21.15.59:0/1673872819 conn(0x55ac64b69080 msgr2=0x55ac64b69460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1673872819 shutdown_connections 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.627+0000 7f8be2fe9740 1 -- 172.21.15.59:0/1673872819 wait complete. 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 -- start start 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 Processor -- start 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b7b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 0x55ac64b7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b85fb0 con 0x55ac64b73670 2024-04-02T17:12:14.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b858e0 con 0x55ac64b6db10 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac64b86680 con 0x55ac64b7eb20 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8be2fe9740 10 client.0 _ll_register_callbacks cb 0x7f8bc4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33476/0 (socket says 172.21.15.59:33476) 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.628+0000 7f8bdce00640 1 -- 172.21.15.59:0/3011512948 learned_addr learned my addr 172.21.15.59:0/3011512948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bde200640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b7b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bde200640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 0x55ac64b7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 -- 172.21.15.59:0/3011512948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 msgr2=0x55ac64b7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 0x55ac64b7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 -- 172.21.15.59:0/3011512948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 msgr2=0x55ac64b7b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b7b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 -- 172.21.15.59:0/3011512948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac64b7f4e0 con 0x55ac64b73670 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bde200640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 0x55ac64b7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:14.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bdce00640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bc402c0d0 tx=0x7f8bc4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db29a0f9a76be790 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4058070 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bc402c730 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.629+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc4005510 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bc4043040 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 10 client.14934 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac64beba30 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc4005510 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bc403f930 con 0x55ac64b73670 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 10 client.14934 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 10 client.14934 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 20 client.14934 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64bf0240 0x55ac64beef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 4 client.14934 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.630+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "status"}) -- 0x55ac64ada8f0 con 0x55ac64bf0240 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8bde200640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64bf0240 0x55ac64beef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8bde200640 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64bf0240 0x55ac64beef50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6227580881a24c8 server_cookie=98ee66533348b70d in_seq=0 out_seq=0 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8bbf400640 10 client.14934 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8bbf400640 1 -- 172.21.15.59:0/3011512948 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ac64ada8f0 con 0x55ac64bf0240 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8bbf400640 10 client.14934 handle_command_reply: tid=0 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.631+0000 7f8be2fe9740 1 client.14934 shutdown 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 msgr2=0x55ac64b7bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bc402c0d0 tx=0x7f8bc4005a90 comp rx=0 tx=0).stop 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 shutdown_connections 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55ac64bf0240 0x55ac64beef50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac64b73670 0x55ac64b7bb00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac64b7eb20 0x55ac64b7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 --2- 172.21.15.59:0/3011512948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac64b6db10 0x55ac64b7b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8bbf400640 0 client.14934 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 >> 172.21.15.59:0/3011512948 conn(0x55ac64b69080 msgr2=0x55ac64b67930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 shutdown_connections 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 1 -- 172.21.15.59:0/3011512948 wait complete. 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.632+0000 7f8be2fe9740 20 client.14934 trim_cache size 0 max 16384 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24560, 2024-04-02T17:12:14.636 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 95.596818098, 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 111.51067778700001 2024-04-02T17:12:14.637 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 msgr2=0x7f8bc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 0x7f8bc00a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0993a0 tx=0x7f8bcc005c80 comp rx=0 tx=0).stop 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 msgr2=0x7f8bd8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e9c0 tx=0x7f8bcc00ee80 comp rx=0 tx=0).stop 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 shutdown_connections 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd81098c0 0x7f8bd81a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bd8075b40 0x7f8bd819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8bc00a3bb0 0x7f8bc00a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 --2- 172.21.15.59:0/776189917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd8074560 0x7f8bd8199f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 >> 172.21.15.59:0/776189917 conn(0x7f8bd80fcf70 msgr2=0x7f8bd81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 shutdown_connections 2024-04-02T17:12:14.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.633+0000 7f8bdec00640 1 -- 172.21.15.59:0/776189917 wait complete. 2024-04-02T17:12:14.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 -- 172.21.15.59:0/4000045743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4074560 msgr2=0x7f82a40749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 --2- 172.21.15.59:0/4000045743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4074560 0x7f82a40749e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8290009930 tx=0x7f829002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 -- 172.21.15.59:0/4000045743 shutdown_connections 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 --2- 172.21.15.59:0/4000045743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4074560 0x7f82a40749e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 --2- 172.21.15.59:0/4000045743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a410dfd0 0x7f82a41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 --2- 172.21.15.59:0/4000045743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a4075b40 0x7f82a4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 -- 172.21.15.59:0/4000045743 >> 172.21.15.59:0/4000045743 conn(0x7f82a40fcf70 msgr2=0x7f82a40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 -- 172.21.15.59:0/4000045743 shutdown_connections 2024-04-02T17:12:14.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.871+0000 7f82aa400640 1 -- 172.21.15.59:0/4000045743 wait complete. 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.872+0000 7f82aa400640 1 Processor -- start 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 -- start start 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4075b40 0x7f82a419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a410dfd0 0x7f82a41a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4113ca0 con 0x7f82a4075b40 2024-04-02T17:12:14.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4113b20 con 0x7f82a410dfd0 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82aa400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4113e20 con 0x7f82a4074560 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4075b40 0x7f82a419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82a9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82a9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44034/0 (socket says 172.21.15.59:44034) 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a410dfd0 0x7f82a41a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.873+0000 7f82a9000640 1 -- 172.21.15.59:0/836538300 learned_addr learned my addr 172.21.15.59:0/836538300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 -- 172.21.15.59:0/836538300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a410dfd0 msgr2=0x7f82a41a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a410dfd0 0x7f82a41a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 -- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4075b40 msgr2=0x7f82a419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4075b40 0x7f82a419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 -- 172.21.15.59:0/836538300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a41a5f70 con 0x7f82a4074560 2024-04-02T17:12:14.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a9000640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f829800ea60 tx=0x7f829800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a1600640 1 -- 172.21.15.59:0/836538300 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829800cde0 con 0x7f82a4074560 2024-04-02T17:12:14.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82a1600640 1 -- 172.21.15.59:0/836538300 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82980035f0 con 0x7f82a4074560 2024-04-02T17:12:14.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.874+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41a6260 con 0x7f82a4074560 2024-04-02T17:12:14.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.875+0000 7f82a1600640 1 -- 172.21.15.59:0/836538300 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298010670 con 0x7f82a4074560 2024-04-02T17:12:14.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.875+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a67d0 con 0x7f82a4074560 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82a1600640 1 -- 172.21.15.59:0/836538300 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f82980037a0 con 0x7f82a4074560 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 Processor -- start 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 -- start start 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 0x556d081fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d08201000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d08201670 0x556d08203b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d08161060 con 0x556d08201670 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d08109560 con 0x556d081fc500 2024-04-02T17:12:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.878+0000 7f82ae631740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d0815d010 con 0x556d081fbb10 2024-04-02T17:12:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d08201000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d08201670 0x556d08203b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 0x556d081fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d08201000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42926/0 (socket says 172.21.15.59:42926) 2024-04-02T17:12:14.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9000640 1 -- 172.21.15.59:0/3591815710 learned_addr learned my addr 172.21.15.59:0/3591815710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 -- 172.21.15.59:0/3591815710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 msgr2=0x556d08201000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d08201000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 -- 172.21.15.59:0/3591815710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d08201670 msgr2=0x556d08203b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d08201670 0x556d08203b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a9a00640 1 -- 172.21.15.59:0/3591815710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0812bfd0 con 0x556d081fbb10 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.879+0000 7f82a1600640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 0x7f828c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 0x556d081fbf30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ed0 tx=0x7f829c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eedcc66a05732bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f82a1600640 1 -- 172.21.15.59:0/836538300 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f82980d0800 con 0x7f82a4074560 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f82a9000640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 0x7f828c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f8296000640 1 -- 172.21.15.59:0/3591815710 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c003b70 con 0x556d081fbb10 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f8296000640 1 -- 172.21.15.59:0/3591815710 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c015070 con 0x556d081fbb10 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.880+0000 7f8296000640 1 -- 172.21.15.59:0/3591815710 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c011870 con 0x556d081fbb10 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- 172.21.15.59:0/3591815710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 msgr2=0x556d081fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82a9000640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 0x7f828c0a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82980d0aa0 tx=0x7f82980d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 0x556d081fbf30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ed0 tx=0x7f829c0107a0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- 172.21.15.59:0/3591815710 shutdown_connections 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d08201670 0x556d08203b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d08201000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 --2- 172.21.15.59:0/3591815710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d081fbb10 0x556d081fbf30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- 172.21.15.59:0/3591815710 >> 172.21.15.59:0/3591815710 conn(0x556d081f7080 msgr2=0x556d081f7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- 172.21.15.59:0/3591815710 shutdown_connections 2024-04-02T17:12:14.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- 172.21.15.59:0/3591815710 wait complete. 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 -- start start 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.881+0000 7f82ae631740 1 Processor -- start 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fbb10 0x556d08205fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d0820d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d08204620 con 0x556d081fc500 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d0811c920 con 0x556d081fbb10 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d082047a0 con 0x556d08201670 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82ae631740 10 client.0 _ll_register_callbacks cb 0x556d0811c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33520/0 (socket says 172.21.15.59:33520) 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82a3e00640 1 -- 172.21.15.59:0/1091079508 learned_addr learned my addr 172.21.15.59:0/1091079508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fbb10 0x556d08205fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.882+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d0820d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 -- 172.21.15.59:0/1091079508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 msgr2=0x556d0820d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d0820d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 -- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fbb10 msgr2=0x556d08205fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fbb10 0x556d08205fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 -- 172.21.15.59:0/1091079508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d07e752d0 con 0x556d081fc500 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d0820d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8290009900 tx=0x7f8290004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fb1a5bc31422d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f828aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290005d50 con 0x556d081fc500 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8290005040 con 0x556d081fc500 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.883+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c015070 con 0x556d081fc500 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8290005240 con 0x556d081fc500 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 10 client.14946 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d07f3f6b0 con 0x556d081fc500 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f829c015070 con 0x556d081fc500 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8290053050 con 0x556d081fc500 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 10 client.14946 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 10 client.14946 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 20 client.14946 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d08212d30 0x556d082141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 4 client.14946 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.884+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d0816e5a0 con 0x556d08212d30 2024-04-02T17:12:14.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.885+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d08212d30 0x556d082141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.885+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d08212d30 0x556d082141a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb6d77abb1fa355 server_cookie=80ac0dac3cbd473f in_seq=0 out_seq=0 2024-04-02T17:12:14.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.885+0000 7f828aa00640 10 client.14946 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:14.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.886+0000 7f828aa00640 1 -- 172.21.15.59:0/1091079508 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d0816e5a0 con 0x556d08212d30 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f828aa00640 10 client.14946 handle_command_reply: tid=0 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 client.14946 shutdown 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 msgr2=0x556d082064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8290009900 tx=0x7f8290004270 comp rx=0 tx=0).stop 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 shutdown_connections 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d08212d30 0x556d082141a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fc500 0x556d082064e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d0820d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 --2- 172.21.15.59:0/1091079508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fbb10 0x556d08205fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 >> 172.21.15.59:0/1091079508 conn(0x556d081f7080 msgr2=0x556d081f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f828aa00640 0 client.14946 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:14.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 shutdown_connections 2024-04-02T17:12:14.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 1 -- 172.21.15.59:0/1091079508 wait complete. 2024-04-02T17:12:14.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.887+0000 7f82ae631740 20 client.14946 trim_cache size 0 max 16384 2024-04-02T17:12:14.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 Processor -- start 2024-04-02T17:12:14.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 -- start start 2024-04-02T17:12:14.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08205bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d082124e0 con 0x556d081fbb10 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d08212bb0 con 0x556d081fc500 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82ae631740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d08213280 con 0x556d08201670 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.908+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33534/0 (socket says 172.21.15.59:33534) 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a9a00640 1 -- 172.21.15.59:0/1169988027 learned_addr learned my addr 172.21.15.59:0/1169988027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08205bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 -- 172.21.15.59:0/1169988027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 msgr2=0x556d08205bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08205bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 -- 172.21.15.59:0/1169988027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 msgr2=0x556d081f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 -- 172.21.15.59:0/1169988027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d081fcd90 con 0x556d081fc500 2024-04-02T17:12:14.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a9a00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:14.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f82a3e00640 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f829002c7b0 tx=0x7f8290000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56116fe3cd4e697 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f8295600640 1 -- 172.21.15.59:0/1169988027 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290059070 con 0x556d081fc500 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f8295600640 1 -- 172.21.15.59:0/1169988027 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8290003d40 con 0x556d081fc500 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.909+0000 7f8295600640 1 -- 172.21.15.59:0/1169988027 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290004bc0 con 0x556d081fc500 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 -- 172.21.15.59:0/1169988027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 msgr2=0x556d0820d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f829002c7b0 tx=0x7f8290000bc0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 -- 172.21.15.59:0/1169988027 shutdown_connections 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08205bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d4e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 --2- 172.21.15.59:0/1169988027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d081f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 -- 172.21.15.59:0/1169988027 >> 172.21.15.59:0/1169988027 conn(0x556d081f7080 msgr2=0x556d081f1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 -- 172.21.15.59:0/1169988027 shutdown_connections 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.910+0000 7f82ae631740 1 -- 172.21.15.59:0/1169988027 wait complete. 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 -- start start 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 Processor -- start 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08209860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d08214020 con 0x556d081fbb10 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d08213950 con 0x556d081fc500 2024-04-02T17:12:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d082146f0 con 0x556d08201670 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.911+0000 7f82ae631740 10 client.0 _ll_register_callbacks cb 0x7f8290009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33548/0 (socket says 172.21.15.59:33548) 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 -- 172.21.15.59:0/3279619257 learned_addr learned my addr 172.21.15.59:0/3279619257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a3e00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08209860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 -- 172.21.15.59:0/3279619257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 msgr2=0x556d08209860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08209860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 -- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 msgr2=0x556d0820d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 -- 172.21.15.59:0/3279619257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d08209f50 con 0x556d081fbb10 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a3e00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.912+0000 7f82a9a00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f829c004710 tx=0x7f829c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf50e38711fd2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:14.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f828aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c025040 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f829c003970 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8290003d40 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f829c004470 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f82ae631740 10 client.14952 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d08279920 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.913+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8290003d40 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f829c02b040 con 0x556d081fbb10 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 10 client.14952 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 10 client.14952 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 20 client.14952 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d0827e310 0x556d0827a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:14.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 4 client.14952 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "status"}) -- 0x556d081688f0 con 0x556d0827e310 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.914+0000 7f82a3e00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d0827e310 0x556d0827a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82a3e00640 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d0827e310 0x556d0827a750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82933696137edad7 server_cookie=64b7b1c19d1115fd in_seq=0 out_seq=0 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f828aa00640 10 client.14952 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f828aa00640 1 -- 172.21.15.59:0/3279619257 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x556d081688f0 con 0x556d0827e310 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f828aa00640 10 client.14952 handle_command_reply: tid=0 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 client.14952 shutdown 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 msgr2=0x556d0820cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f829c004710 tx=0x7f829c009500 comp rx=0 tx=0).stop 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 shutdown_connections 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556d0827e310 0x556d0827a750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d08201670 0x556d08209860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d081fc500 0x556d0820d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 --2- 172.21.15.59:0/3279619257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d081fbb10 0x556d0820cd70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 >> 172.21.15.59:0/3279619257 conn(0x556d081f7080 msgr2=0x556d081f7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.915+0000 7f828aa00640 0 client.14952 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 shutdown_connections 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82ae631740 1 -- 172.21.15.59:0/3279619257 wait complete. 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82ae631740 20 client.14952 trim_cache size 0 max 16384 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24584, 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:14.920 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 99.431962055, 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 105.094643992 2024-04-02T17:12:14.921 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 msgr2=0x7f828c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 0x7f828c0a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82980d0aa0 tx=0x7f82980d6530 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 msgr2=0x7f82a419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.916+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f829800ea60 tx=0x7f829800ef20 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 shutdown_connections 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f828c0a3c30 0x7f828c0a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82a4075b40 0x7f82a419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a410dfd0 0x7f82a41a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 --2- 172.21.15.59:0/836538300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a4074560 0x7f82a419e350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 >> 172.21.15.59:0/836538300 conn(0x7f82a40fcf70 msgr2=0x7f82a4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:14.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 shutdown_connections 2024-04-02T17:12:14.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:14.917+0000 7f82aa400640 1 -- 172.21.15.59:0/836538300 wait complete. 2024-04-02T17:12:14.934 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T17:12:14.934 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T17:12:14.947 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:12:14.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T17:12:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:15 smithi059 ceph-mon[31664]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:15 smithi154 ceph-mon[36479]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:15.414 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:15 smithi164 ceph-mon[36499]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:17.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:17 smithi059 ceph-mon[31664]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:17 smithi154 ceph-mon[36479]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 -- 172.21.15.59:0/1913301649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 msgr2=0x7f0d80074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 --2- 172.21.15.59:0/1913301649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d80074a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009990 tx=0x7f0d74009ca0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 -- 172.21.15.59:0/1913301649 shutdown_connections 2024-04-02T17:12:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 --2- 172.21.15.59:0/1913301649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d8010a440 0x7f0d8010c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 --2- 172.21.15.59:0/1913301649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d80074a00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 --2- 172.21.15.59:0/1913301649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80074fd0 0x7f0d80075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.566+0000 7f0d86e00700 1 -- 172.21.15.59:0/1913301649 >> 172.21.15.59:0/1913301649 conn(0x7f0d800fdb00 msgr2=0x7f0d800fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.567+0000 7f0d86e00700 1 -- 172.21.15.59:0/1913301649 shutdown_connections 2024-04-02T17:12:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.567+0000 7f0d86e00700 1 -- 172.21.15.59:0/1913301649 wait complete. 2024-04-02T17:12:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 Processor -- start 2024-04-02T17:12:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 -- start start 2024-04-02T17:12:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 0x7f0d801a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80110350 con 0x7f0d800765a0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801101d0 con 0x7f0d8010a440 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d86e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801104d0 con 0x7f0d80074fd0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d85000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d85a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d85000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33554/0 (socket says 172.21.15.59:33554) 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d85a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44090/0 (socket says 172.21.15.59:44090) 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.568+0000 7f0d85000700 1 -- 172.21.15.59:0/2943038004 learned_addr learned my addr 172.21.15.59:0/2943038004 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d86400700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 0x7f0d801a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 -- 172.21.15.59:0/2943038004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 msgr2=0x7f0d801a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 0x7f0d801a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 -- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 msgr2=0x7f0d8019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 -- 172.21.15.59:0/2943038004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d801a2570 con 0x7f0d80074fd0 2024-04-02T17:12:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85000700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d86400700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 0x7f0d801a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d85a00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d7400bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.569+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74026070 con 0x7f0d80074fd0 2024-04-02T17:12:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.570+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74003d70 con 0x7f0d80074fd0 2024-04-02T17:12:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.570+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a2800 con 0x7f0d80074fd0 2024-04-02T17:12:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.570+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74021410 con 0x7f0d80074fd0 2024-04-02T17:12:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.570+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a2c50 con 0x7f0d80074fd0 2024-04-02T17:12:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.571+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d800692b0 con 0x7f0d80074fd0 2024-04-02T17:12:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:17 smithi164 ceph-mon[36499]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:17.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.573+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0d74029070 con 0x7f0d80074fd0 2024-04-02T17:12:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.574+0000 7f0d7e000700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 0x7f0d680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.574+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0d740d18f0 con 0x7f0d80074fd0 2024-04-02T17:12:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.575+0000 7f0d85000700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 0x7f0d680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.575+0000 7f0d85000700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 0x7f0d680a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011550 tx=0x7f0d6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.577+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0d74030080 con 0x7f0d80074fd0 2024-04-02T17:12:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.829+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0d80062a30 con 0x7f0d680a3c40 2024-04-02T17:12:17.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.831+0000 7f0d7e000700 1 -- 172.21.15.59:0/2943038004 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0d80062a30 con 0x7f0d680a3c40 2024-04-02T17:12:17.831 INFO:teuthology.orchestra.run.smithi059.stdout:Backend: cephadm 2024-04-02T17:12:17.831 INFO:teuthology.orchestra.run.smithi059.stdout:Available: Yes 2024-04-02T17:12:17.831 INFO:teuthology.orchestra.run.smithi059.stdout:Paused: No 2024-04-02T17:12:17.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 msgr2=0x7f0d680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:17.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 0x7f0d680a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011550 tx=0x7f0d6c009260 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 msgr2=0x7f0d8019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d7400bb90 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 shutdown_connections 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8010a440 0x7f0d801a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d80074fd0 0x7f0d8019a910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0d680a3c40 0x7f0d680a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 --2- 172.21.15.59:0/2943038004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d800765a0 0x7f0d8019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:17.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 >> 172.21.15.59:0/2943038004 conn(0x7f0d800fdb00 msgr2=0x7f0d80106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:17.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 shutdown_connections 2024-04-02T17:12:17.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:17.835+0000 7f0d86e00700 1 -- 172.21.15.59:0/2943038004 wait complete. 2024-04-02T17:12:18.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T17:12:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:19.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:19 smithi059 ceph-mon[31664]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:19.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:19 smithi059 ceph-mon[31664]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:19 smithi154 ceph-mon[36479]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:19.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:19 smithi154 ceph-mon[36479]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:19 smithi164 ceph-mon[36499]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:19 smithi164 ceph-mon[36499]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 -- 172.21.15.59:0/1732695823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e4103740 msgr2=0x7f80e4103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 --2- 172.21.15.59:0/1732695823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e4103740 0x7f80e4103ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009920 tx=0x7f80cc009c30 comp rx=0 tx=0).stop 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 -- 172.21.15.59:0/1732695823 shutdown_connections 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 --2- 172.21.15.59:0/1732695823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e41040e0 0x7f80e410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 --2- 172.21.15.59:0/1732695823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4102540 0x7f80e4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 --2- 172.21.15.59:0/1732695823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e4103740 0x7f80e4103ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 -- 172.21.15.59:0/1732695823 >> 172.21.15.59:0/1732695823 conn(0x7f80e40fdb00 msgr2=0x7f80e40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:21.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 -- 172.21.15.59:0/1732695823 shutdown_connections 2024-04-02T17:12:21.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.263+0000 7f80e8c00700 1 -- 172.21.15.59:0/1732695823 wait complete. 2024-04-02T17:12:21.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 Processor -- start 2024-04-02T17:12:21.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 -- start start 2024-04-02T17:12:21.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4103740 0x7f80e419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 0x7f80e41a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410e100 con 0x7f80e4102540 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410df80 con 0x7f80e41040e0 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.265+0000 7f80e8c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410e280 con 0x7f80e4103740 2024-04-02T17:12:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57272/0 (socket says 172.21.15.59:57272) 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 -- 172.21.15.59:0/2785314703 learned_addr learned my addr 172.21.15.59:0/2785314703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3e00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 0x7f80e41a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80daa00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4103740 0x7f80e419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 -- 172.21.15.59:0/2785314703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4103740 msgr2=0x7f80e419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4103740 0x7f80e419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 -- 172.21.15.59:0/2785314703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 msgr2=0x7f80e41a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 0x7f80e41a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3400700 1 -- 172.21.15.59:0/2785314703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41a25f0 con 0x7f80e4102540 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.266+0000 7f80e3e00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 0x7f80e41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:21.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e3400700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d9b0 tx=0x7f80d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4016070 con 0x7f80e4102540 2024-04-02T17:12:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d400be30 con 0x7f80e4102540 2024-04-02T17:12:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400f3c0 con 0x7f80e4102540 2024-04-02T17:12:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41a28e0 con 0x7f80e4102540 2024-04-02T17:12:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.267+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41a2dd0 con 0x7f80e4102540 2024-04-02T17:12:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.268+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e40692b0 con 0x7f80e4102540 2024-04-02T17:12:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.270+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f80d4019070 con 0x7f80e4102540 2024-04-02T17:12:21.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.272+0000 7f80e0c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 0x7f80c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:21.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.272+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f80d40cff70 con 0x7f80e4102540 2024-04-02T17:12:21.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.272+0000 7f80daa00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 0x7f80c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:21.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.273+0000 7f80daa00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 0x7f80c80a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0098f0 tx=0x7f80cc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:21.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.278+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f80d4020030 con 0x7f80e4102540 2024-04-02T17:12:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:21 smithi154 ceph-mon[36479]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:21.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.524+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f80e4075f40 con 0x7f80c80a3cc0 2024-04-02T17:12:21.524 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:21 smithi059 ceph-mon[31664]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:21.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.567+0000 7f80e0c00700 1 -- 172.21.15.59:0/2785314703 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f80e4075f40 con 0x7f80c80a3cc0 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.a smithi059 running (114s) 25s ago 114s 12.3M - 19.0.0-2609-g191d963a f7301e070545 323fd44a3977 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.b smithi154 running (112s) 43s ago 111s 26.2M - 19.0.0-2609-g191d963a f7301e070545 7104f6875f3a 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.c smithi164 running (2m) 43s ago 2m 17.9M - 19.0.0-2609-g191d963a f7301e070545 6e81fcef5a12 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.d smithi059 running (2m) 25s ago 2m 15.4M - 19.0.0-2609-g191d963a f7301e070545 0f1ece883dc8 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.e smithi154 running (2m) 43s ago 2m 18.9M - 19.0.0-2609-g191d963a f7301e070545 cedc45bfbd01 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.f smithi164 running (2m) 43s ago 2m 19.4M - 19.0.0-2609-g191d963a f7301e070545 8297056f5b34 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.g smithi059 running (118s) 25s ago 118s 16.7M - 19.0.0-2609-g191d963a f7301e070545 609691d8ea22 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.h smithi154 running (2m) 43s ago 2m 17.9M - 19.0.0-2609-g191d963a f7301e070545 c5f867ee1cc4 2024-04-02T17:12:21.567 INFO:teuthology.orchestra.run.smithi059.stdout:mds.i smithi164 running (2m) 43s ago 2m 18.2M - 19.0.0-2609-g191d963a f7301e070545 994940dd911f 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mds.j smithi059 running (2m) 25s ago 2m 12.4M - 19.0.0-2609-g191d963a f7301e070545 d0735e66ce62 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mds.k smithi154 running (2m) 43s ago 2m 19.4M - 19.0.0-2609-g191d963a f7301e070545 54a6c83cd357 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mds.l smithi164 running (2m) 43s ago 2m 19.5M - 19.0.0-2609-g191d963a f7301e070545 48b62ec8ce0b 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.x smithi059 *:9283,8765 running (10m) 25s ago 10m 513M - 19.0.0-2609-g191d963a f7301e070545 fdd3dd274a97 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.y smithi154 *:8443,8765 running (8m) 43s ago 8m 454M - 19.0.0-2609-g191d963a f7301e070545 59f70b80d2b6 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.z smithi164 *:8443,8765 running (8m) 43s ago 8m 452M - 19.0.0-2609-g191d963a f7301e070545 4e13f0699e6d 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a smithi059 running (10m) 25s ago 10m 48.6M 2048M 19.0.0-2609-g191d963a f7301e070545 b4e1aa018a5e 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b smithi154 running (8m) 43s ago 8m 44.4M 2048M 19.0.0-2609-g191d963a f7301e070545 14d487407afd 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c smithi164 running (8m) 43s ago 8m 43.3M 2048M 19.0.0-2609-g191d963a f7301e070545 cd723794deac 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 smithi059 running (7m) 25s ago 7m 47.3M 4273M 19.0.0-2609-g191d963a f7301e070545 d7c5149a912e 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 smithi154 running (7m) 43s ago 7m 57.0M 4270M 19.0.0-2609-g191d963a f7301e070545 dd676ac9f031 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 smithi164 running (7m) 43s ago 6m 58.4M 4270M 19.0.0-2609-g191d963a f7301e070545 9944b704785b 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 smithi059 running (6m) 25s ago 6m 45.3M 4273M 19.0.0-2609-g191d963a f7301e070545 983df97c2100 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 smithi154 running (6m) 43s ago 6m 53.3M 4270M 19.0.0-2609-g191d963a f7301e070545 24c1c93cfc95 2024-04-02T17:12:21.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 smithi164 running (5m) 43s ago 5m 57.9M 4270M 19.0.0-2609-g191d963a f7301e070545 35b27f7280b1 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 smithi059 running (5m) 25s ago 5m 50.0M 4273M 19.0.0-2609-g191d963a f7301e070545 9513ee081eae 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 smithi154 running (4m) 43s ago 4m 49.5M 4270M 19.0.0-2609-g191d963a f7301e070545 e32b812c85de 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 smithi164 running (4m) 43s ago 4m 48.5M 4270M 19.0.0-2609-g191d963a f7301e070545 0b8f701355e1 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 smithi059 running (3m) 25s ago 3m 49.8M 4273M 19.0.0-2609-g191d963a f7301e070545 120e32f7acad 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 smithi154 running (3m) 43s ago 3m 55.4M 4270M 19.0.0-2609-g191d963a f7301e070545 8d74e0cffc0c 2024-04-02T17:12:21.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 smithi164 running (2m) 43s ago 2m 50.5M 4270M 19.0.0-2609-g191d963a f7301e070545 14621066e8bf 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 msgr2=0x7f80c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 0x7f80c80a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0098f0 tx=0x7f80cc019040 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 msgr2=0x7f80e419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d9b0 tx=0x7f80d400dcc0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 shutdown_connections 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80e41040e0 0x7f80e41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80c80a3cc0 0x7f80c80a6180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80e4102540 0x7f80e419a9e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 --2- 172.21.15.59:0/2785314703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e4103740 0x7f80e419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 >> 172.21.15.59:0/2785314703 conn(0x7f80e40fdb00 msgr2=0x7f80e4074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 shutdown_connections 2024-04-02T17:12:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:21.570+0000 7f80e8c00700 1 -- 172.21.15.59:0/2785314703 wait complete. 2024-04-02T17:12:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:21 smithi164 ceph-mon[36499]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:22.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T17:12:22.841 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:23 smithi059 ceph-mon[31664]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:23.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:23 smithi059 ceph-mon[31664]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:23 smithi154 ceph-mon[36479]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:23.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:23 smithi154 ceph-mon[36479]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:23 smithi164 ceph-mon[36499]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:23 smithi164 ceph-mon[36499]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 -- 172.21.15.59:0/3879866619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 msgr2=0x7efc241054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 --2- 172.21.15.59:0/3879866619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc241054a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc14009990 tx=0x7efc14009ca0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 -- 172.21.15.59:0/3879866619 shutdown_connections 2024-04-02T17:12:24.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 --2- 172.21.15.59:0/3879866619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc2410a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 --2- 172.21.15.59:0/3879866619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc241054a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 --2- 172.21.15.59:0/3879866619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 0x7efc24107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 -- 172.21.15.59:0/3879866619 >> 172.21.15.59:0/3879866619 conn(0x7efc240fcac0 msgr2=0x7efc240fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 -- 172.21.15.59:0/3879866619 shutdown_connections 2024-04-02T17:12:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.788+0000 7efc28e00700 1 -- 172.21.15.59:0/3879866619 wait complete. 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 Processor -- start 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 -- start start 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 0x7efc24101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc241015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e260 con 0x7efc241030b0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e0e0 con 0x7efc24108310 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc28e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e3e0 con 0x7efc241059e0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.790+0000 7efc21e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57280/0 (socket says 172.21.15.59:57280) 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 -- 172.21.15.59:0/3012817858 learned_addr learned my addr 172.21.15.59:0/3012817858 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc22800700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc241015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21400700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 0x7efc24101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 -- 172.21.15.59:0/3012817858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 msgr2=0x7efc24101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 0x7efc24101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 -- 172.21.15.59:0/3012817858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 msgr2=0x7efc241015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc241015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 -- 172.21.15.59:0/3012817858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24101e50 con 0x7efc241030b0 2024-04-02T17:12:24.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc22800700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc241015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.791+0000 7efc21e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b2e0 tx=0x7efc1400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.792+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14026070 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.792+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241a2a90 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.792+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1400f460 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.792+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14021670 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.792+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a2ee0 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.793+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc240692b0 con 0x7efc241030b0 2024-04-02T17:12:24.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.795+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc14029070 con 0x7efc241030b0 2024-04-02T17:12:24.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.796+0000 7efc1aa00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 0x7efc080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:24.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.797+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efc140d1170 con 0x7efc241030b0 2024-04-02T17:12:24.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.797+0000 7efc21400700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 0x7efc080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:24.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.797+0000 7efc21400700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 0x7efc080a6180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc10009c60 tx=0x7efc10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:24.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:24.806+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efc1405df20 con 0x7efc241030b0 2024-04-02T17:12:25.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.065+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efc24062a30 con 0x7efc080a3cc0 2024-04-02T17:12:25.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.073+0000 7efc1aa00700 1 -- 172.21.15.59:0/3012817858 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efc24062a30 con 0x7efc080a3cc0 2024-04-02T17:12:25.072 INFO:teuthology.orchestra.run.smithi059.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T17:12:25.072 INFO:teuthology.orchestra.run.smithi059.stdout:mds.all 12/12 46s ago 2m smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-02T17:12:25.072 INFO:teuthology.orchestra.run.smithi059.stdout:mgr 3/3 46s ago 8m smithi059=x;smithi154=y;smithi164=z;count:3 2024-04-02T17:12:25.072 INFO:teuthology.orchestra.run.smithi059.stdout:mon 3/3 46s ago 8m smithi059:172.21.15.59=a;smithi154:172.21.15.154=b;smithi164:172.21.15.164=c;count:3 2024-04-02T17:12:25.073 INFO:teuthology.orchestra.run.smithi059.stdout:osd 12 46s ago - 2024-04-02T17:12:25.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 msgr2=0x7efc080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:25.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 0x7efc080a6180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc10009c60 tx=0x7efc10009260 comp rx=0 tx=0).stop 2024-04-02T17:12:25.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 msgr2=0x7efc24102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:25.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b2e0 tx=0x7efc1400bbb0 comp rx=0 tx=0).stop 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 shutdown_connections 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc24108310 0x7efc241015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efc080a3cc0 0x7efc080a6180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc241030b0 0x7efc24102a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 --2- 172.21.15.59:0/3012817858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc241059e0 0x7efc24101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 >> 172.21.15.59:0/3012817858 conn(0x7efc240fcac0 msgr2=0x7efc24103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 shutdown_connections 2024-04-02T17:12:25.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:25.076+0000 7efc28e00700 1 -- 172.21.15.59:0/3012817858 wait complete. 2024-04-02T17:12:25.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:25 smithi059 ceph-mon[31664]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:25.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:25 smithi154 ceph-mon[36479]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:25 smithi164 ceph-mon[36499]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:25.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T17:12:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:26 smithi154 ceph-mon[36479]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:26.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:26 smithi059 ceph-mon[31664]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:26 smithi164 ceph-mon[36499]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:27 smithi154 ceph-mon[36479]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:27.542 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:27 smithi059 ceph-mon[31664]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:27 smithi164 ceph-mon[36499]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:28.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 -- 172.21.15.59:0/368489245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904105a70 msgr2=0x7f0904105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:28.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 --2- 172.21.15.59:0/368489245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904105a70 0x7f0904105ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009990 tx=0x7f08f0009ca0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 -- 172.21.15.59:0/368489245 shutdown_connections 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 --2- 172.21.15.59:0/368489245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904106410 0x7f090410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 --2- 172.21.15.59:0/368489245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f0904104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 --2- 172.21.15.59:0/368489245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904105a70 0x7f0904105ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.473+0000 7f090ac00700 1 -- 172.21.15.59:0/368489245 >> 172.21.15.59:0/368489245 conn(0x7f09040ffe10 msgr2=0x7f0904102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.474+0000 7f090ac00700 1 -- 172.21.15.59:0/368489245 shutdown_connections 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.474+0000 7f090ac00700 1 -- 172.21.15.59:0/368489245 wait complete. 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 Processor -- start 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 -- start start 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 0x7f090419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904106410 0x7f09041a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0904110460 con 0x7f0904104870 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09041102e0 con 0x7f0904105a70 2024-04-02T17:12:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.475+0000 7f090ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09041105e0 con 0x7f0904106410 2024-04-02T17:12:28.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:28.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57292/0 (socket says 172.21.15.59:57292) 2024-04-02T17:12:28.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 -- 172.21.15.59:0/2394777192 learned_addr learned my addr 172.21.15.59:0/2394777192 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:28.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903400700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 0x7f090419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:28.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0908c00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904106410 0x7f09041a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:28.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 -- 172.21.15.59:0/2394777192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904106410 msgr2=0x7f09041a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:28.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904106410 0x7f09041a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 -- 172.21.15.59:0/2394777192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 msgr2=0x7f090419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:28.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 0x7f090419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903e00700 1 -- 172.21.15.59:0/2394777192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09041a2670 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.476+0000 7f0903400700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 0x7f090419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f0903e00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9a0 tx=0x7f08f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004220 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f40043c0 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09041a2960 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4003df0 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.477+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a2ea0 con 0x7f0904104870 2024-04-02T17:12:28.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.478+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09040692b0 con 0x7f0904104870 2024-04-02T17:12:28.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.480+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f08f400c810 con 0x7f0904104870 2024-04-02T17:12:28.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.482+0000 7f0900c00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 0x7f08e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:28.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.482+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f08f40d0680 con 0x7f0904104870 2024-04-02T17:12:28.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.482+0000 7f0903400700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 0x7f08e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:28.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.483+0000 7f0903400700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 0x7f08e80a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b2e0 tx=0x7f08f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:28.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.487+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f08f409cfa0 con 0x7f0904104870 2024-04-02T17:12:28.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.728+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0904062a30 con 0x7f08e80a3c40 2024-04-02T17:12:28.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.731+0000 7f0900c00700 1 -- 172.21.15.59:0/2394777192 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0904062a30 con 0x7f08e80a3c40 2024-04-02T17:12:28.730 INFO:teuthology.orchestra.run.smithi059.stdout:HOST ADDR LABELS STATUS 2024-04-02T17:12:28.730 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 172.21.15.59 2024-04-02T17:12:28.731 INFO:teuthology.orchestra.run.smithi059.stdout:smithi154 172.21.15.154 2024-04-02T17:12:28.731 INFO:teuthology.orchestra.run.smithi059.stdout:smithi164 172.21.15.164 2024-04-02T17:12:28.731 INFO:teuthology.orchestra.run.smithi059.stdout:3 hosts in cluster 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 msgr2=0x7f08e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 0x7f08e80a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b2e0 tx=0x7f08f001a040 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 msgr2=0x7f090419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9a0 tx=0x7f08f400ecb0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 shutdown_connections 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904106410 0x7f09041a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c40 0x7f08e80a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904104870 0x7f090419aa10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 --2- 172.21.15.59:0/2394777192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105a70 0x7f090419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 >> 172.21.15.59:0/2394777192 conn(0x7f09040ffe10 msgr2=0x7f0904108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 shutdown_connections 2024-04-02T17:12:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:28.734+0000 7f090ac00700 1 -- 172.21.15.59:0/2394777192 wait complete. 2024-04-02T17:12:29.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:29 smithi059 ceph-mon[31664]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:29 smithi154 ceph-mon[36479]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:29.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T17:12:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:29 smithi164 ceph-mon[36499]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:30 smithi059 ceph-mon[31664]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:30 smithi154 ceph-mon[36479]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:30 smithi164 ceph-mon[36499]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:31.323 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:31 smithi059 ceph-mon[31664]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:31 smithi154 ceph-mon[36479]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:31 smithi164 ceph-mon[36499]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 -- 172.21.15.59:0/2792881947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c109e70 msgr2=0x7f9f7c10a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2792881947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c109e70 0x7f9f7c10a2d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68009cd0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 -- 172.21.15.59:0/2792881947 shutdown_connections 2024-04-02T17:12:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2792881947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2792881947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 0x7f9f7c109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2792881947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c109e70 0x7f9f7c10a2d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 -- 172.21.15.59:0/2792881947 >> 172.21.15.59:0/2792881947 conn(0x7f9f7c077c40 msgr2=0x7f9f7c07a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 -- 172.21.15.59:0/2792881947 shutdown_connections 2024-04-02T17:12:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.800+0000 7f9f80c00700 1 -- 172.21.15.59:0/2792881947 wait complete. 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.801+0000 7f9f80c00700 1 Processor -- start 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 -- start start 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 0x7f9f7c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c114880 con 0x7f9f7c10a810 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c114700 con 0x7f9f7c109e70 2024-04-02T17:12:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f80c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c114a00 con 0x7f9f7c108c70 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f79400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f7a800700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f79e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 0x7f9f7c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f79400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50806/0 (socket says 172.21.15.59:50806) 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.802+0000 7f9f79400700 1 -- 172.21.15.59:0/2991885877 learned_addr learned my addr 172.21.15.59:0/2991885877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 -- 172.21.15.59:0/2991885877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 msgr2=0x7f9f7c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 0x7f9f7c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 -- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 msgr2=0x7f9f7c1a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 -- 172.21.15.59:0/2991885877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1a6ac0 con 0x7f9f7c109e70 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f7a800700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f79400700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9f68003790 tx=0x7f9f6800bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68009e60 con 0x7f9f7c109e70 2024-04-02T17:12:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1a6d50 con 0x7f9f7c109e70 2024-04-02T17:12:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.803+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6800f460 con 0x7f9f7c109e70 2024-04-02T17:12:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.804+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68017610 con 0x7f9f7c109e70 2024-04-02T17:12:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.804+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1a71b0 con 0x7f9f7c109e70 2024-04-02T17:12:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.805+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c0692b0 con 0x7f9f7c109e70 2024-04-02T17:12:31.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.807+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9f6800f610 con 0x7f9f7c109e70 2024-04-02T17:12:31.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.808+0000 7f9f72a00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 0x7f9f600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.809+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9f680d11e0 con 0x7f9f7c109e70 2024-04-02T17:12:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.809+0000 7f9f79e00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 0x7f9f600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.809+0000 7f9f79e00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 0x7f9f600a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c109cd0 tx=0x7f9f6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:31.811+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f68013070 con 0x7f9f7c109e70 2024-04-02T17:12:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.050+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9f7c062a30 con 0x7f9f600a3cc0 2024-04-02T17:12:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.053+0000 7f9f72a00700 1 -- 172.21.15.59:0/2991885877 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9f7c062a30 con 0x7f9f600a3cc0 2024-04-02T17:12:32.054 INFO:teuthology.orchestra.run.smithi059.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T17:12:32.054 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T17:12:32.054 INFO:teuthology.orchestra.run.smithi059.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T17:12:32.054 INFO:teuthology.orchestra.run.smithi059.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-02T17:12:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 msgr2=0x7f9f600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 0x7f9f600a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c109cd0 tx=0x7f9f6c009260 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 msgr2=0x7f9f7c19f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9f68003790 tx=0x7f9f6800bee0 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 shutdown_connections 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9f600a3cc0 0x7f9f600a6180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f7c10a810 0x7f9f7c1a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f7c108c70 0x7f9f7c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 --2- 172.21.15.59:0/2991885877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c109e70 0x7f9f7c19f3a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.056+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 >> 172.21.15.59:0/2991885877 conn(0x7f9f7c077c40 msgr2=0x7f9f7c10d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.057+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 shutdown_connections 2024-04-02T17:12:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:32.057+0000 7f9f80c00700 1 -- 172.21.15.59:0/2991885877 wait complete. 2024-04-02T17:12:32.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T17:12:32.843 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:12:32.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T17:12:33.301 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:33 smithi154 ceph-mon[36479]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T17:12:33.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:33 smithi154 ceph-mon[36479]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:33.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:33 smithi059 ceph-mon[31664]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T17:12:33.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:33 smithi059 ceph-mon[31664]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:33 smithi164 ceph-mon[36499]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-02T17:12:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:33 smithi164 ceph-mon[36499]: from='client.25094 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T17:12:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 -- 172.21.15.59:0/3206370825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef40765a0 msgr2=0x7efef4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 --2- 172.21.15.59:0/3206370825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef40765a0 0x7efef4074a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efee4009990 tx=0x7efee4009ca0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 -- 172.21.15.59:0/3206370825 shutdown_connections 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 --2- 172.21.15.59:0/3206370825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 0x7efef410c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 --2- 172.21.15.59:0/3206370825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef40765a0 0x7efef4074a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 --2- 172.21.15.59:0/3206370825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef4074fd0 0x7efef4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 -- 172.21.15.59:0/3206370825 >> 172.21.15.59:0/3206370825 conn(0x7efef40fdb00 msgr2=0x7efef40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 -- 172.21.15.59:0/3206370825 shutdown_connections 2024-04-02T17:12:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.262+0000 7efef9000700 1 -- 172.21.15.59:0/3206370825 wait complete. 2024-04-02T17:12:35.264 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:35 smithi059 ceph-mon[31664]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.263+0000 7efef9000700 1 Processor -- start 2024-04-02T17:12:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 -- start start 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 0x7efef419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 0x7efef419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 0x7efef41a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efef4110370 con 0x7efef4074fd0 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efef41101f0 con 0x7efef410a460 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef9000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efef41104f0 con 0x7efef40765a0 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef3400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 0x7efef419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 0x7efef419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 0x7efef419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54714/0 (socket says 172.21.15.59:54714) 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.264+0000 7efef2a00700 1 -- 172.21.15.59:0/345790059 learned_addr learned my addr 172.21.15.59:0/345790059 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3e00700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 0x7efef41a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 -- 172.21.15.59:0/345790059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 msgr2=0x7efef419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 0x7efef419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 -- 172.21.15.59:0/345790059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 msgr2=0x7efef41a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 0x7efef41a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 -- 172.21.15.59:0/345790059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef41a25c0 con 0x7efef4074fd0 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.265+0000 7efef3400700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 0x7efef419a960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.266+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4026070 con 0x7efef4074fd0 2024-04-02T17:12:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.266+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4003d70 con 0x7efef4074fd0 2024-04-02T17:12:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.266+0000 7efef9000700 1 -- 172.21.15.59:0/345790059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41a2850 con 0x7efef4074fd0 2024-04-02T17:12:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.266+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4021410 con 0x7efef4074fd0 2024-04-02T17:12:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.266+0000 7efef9000700 1 -- 172.21.15.59:0/345790059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a2d00 con 0x7efef4074fd0 2024-04-02T17:12:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.267+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc005330 con 0x7efef4074fd0 2024-04-02T17:12:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.269+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efee4029070 con 0x7efef4074fd0 2024-04-02T17:12:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.270+0000 7efeebe00700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 0x7efed80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.271+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efee40d1210 con 0x7efef4074fd0 2024-04-02T17:12:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.271+0000 7efef2a00700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 0x7efed80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.271+0000 7efef2a00700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 0x7efed80a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d60 tx=0x7efee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.275+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efee405df20 con 0x7efef4074fd0 2024-04-02T17:12:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:35 smithi154 ceph-mon[36479]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.564+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efecc005e30 con 0x7efef4074fd0 2024-04-02T17:12:35.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.565+0000 7efeebe00700 1 -- 172.21.15.59:0/345790059 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3088 (secure 0 0 0) 0x7efee4013070 con 0x7efef4074fd0 2024-04-02T17:12:35.566 INFO:teuthology.orchestra.run.smithi059.stdout:e27 2024-04-02T17:12:35.566 INFO:teuthology.orchestra.run.smithi059.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T17:12:35.566 INFO:teuthology.orchestra.run.smithi059.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T17:12:35.566 INFO:teuthology.orchestra.run.smithi059.stdout:legacy client fscid: 1 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:Filesystem 'cephfs' (1) 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:fs_name cephfs 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 27 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-04-02T17:10:34.529898+0000 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-04-02T17:10:43.085475+0000 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:tableserver 0 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:root 0 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:session_timeout 300 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:session_autoclose 300 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:max_file_size 1099511627776 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:max_xattr_size 65536 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:required_client_features {} 2024-04-02T17:12:35.567 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure 0 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure_osd_epoch 0 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:max_mds 5 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:in 0,1,2,3,4 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:up {0=24584,1=24560,2=24547,3=24530,4=24529} 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:failed 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:damaged 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:stopped 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:data_pools [3] 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:metadata_pool 2 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:inline_data disabled 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:balancer 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:bal_rank_mask -1 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:standby_count_wanted 0 2024-04-02T17:12:35.568 INFO:teuthology.orchestra.run.smithi059.stdout:qdb_cluster leader: 24584 members: 24560,24584,24547,24530,24529 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b{0:24584} state up:active seq 3 addr [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.f{0:24517} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/2083325907,v1:172.21.15.164:6835/2083325907] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.g{1:24560} state up:active seq 5 addr [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.e{1:24503} state up:standby-replay seq 1 addr [v2:172.21.15.154:6834/1503733829,v1:172.21.15.154:6835/1503733829] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.i{2:24547} state up:active seq 7 addr [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d{2:24490} state up:standby-replay seq 1 addr [v2:172.21.15.59:6836/448006801,v1:172.21.15.59:6837/448006801] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.h{3:24530} state up:active seq 8 addr [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.k{3:24455} state up:standby-replay seq 1 addr [v2:172.21.15.154:6832/1241853080,v1:172.21.15.154:6833/1241853080] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c{4:24529} state up:active seq 9 addr [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.l{4:24454} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/3588884595,v1:172.21.15.164:6833/3588884595] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:Standby daemons: 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.j{-1:14388} state up:standby seq 1 addr [v2:172.21.15.59:6834/346805939,v1:172.21.15.59:6835/346805939] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.569 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a{-1:14466} state up:standby seq 1 addr [v2:172.21.15.59:6840/195975414,v1:172.21.15.59:6841/195975414] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T17:12:35.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 msgr2=0x7efed80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:35.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 0x7efed80a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d60 tx=0x7efee0009260 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 msgr2=0x7efef419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 0x7efef419a960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee400bb90 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 shutdown_connections 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efef410a460 0x7efef41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efed80a3cc0 0x7efed80a6180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef4074fd0 0x7efef419a960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 --2- 172.21.15.59:0/345790059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efef40765a0 0x7efef419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 >> 172.21.15.59:0/345790059 conn(0x7efef40fdb00 msgr2=0x7efef41063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 shutdown_connections 2024-04-02T17:12:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:35.570+0000 7efee9600700 1 -- 172.21.15.59:0/345790059 wait complete. 2024-04-02T17:12:35.572 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:35 smithi164 ceph-mon[36499]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-02T17:12:36.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-04-02T17:12:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:36 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/345790059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T17:12:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:36 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/345790059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T17:12:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:36 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/345790059' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T17:12:36.739 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:37.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:37 smithi059 ceph-mon[31664]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:37.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:37 smithi154 ceph-mon[36479]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:37 smithi164 ceph-mon[36499]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 -- 172.21.15.59:0/4196573513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 msgr2=0x7fd87c105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 --2- 172.21.15.59:0/4196573513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c105ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098f0 tx=0x7fd868009c00 comp rx=0 tx=0).stop 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 -- 172.21.15.59:0/4196573513 shutdown_connections 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 --2- 172.21.15.59:0/4196573513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c106410 0x7fd87c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 --2- 172.21.15.59:0/4196573513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c104870 0x7fd87c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 --2- 172.21.15.59:0/4196573513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c105ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 -- 172.21.15.59:0/4196573513 >> 172.21.15.59:0/4196573513 conn(0x7fd87c0ffe10 msgr2=0x7fd87c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 -- 172.21.15.59:0/4196573513 shutdown_connections 2024-04-02T17:12:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.831+0000 7fd881600700 1 -- 172.21.15.59:0/4196573513 wait complete. 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.832+0000 7fd881600700 1 Processor -- start 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.832+0000 7fd881600700 1 -- start start 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c106410 0x7fd87c1a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c110430 con 0x7fd87c104870 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1102b0 con 0x7fd87c106410 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd881600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c1105b0 con 0x7fd87c105a70 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd87aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd87aa00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35602/0 (socket says 172.21.15.59:35602) 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd87aa00700 1 -- 172.21.15.59:0/774521950 learned_addr learned my addr 172.21.15.59:0/774521950 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd87a000700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.833+0000 7fd87b400700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c106410 0x7fd87c1a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 -- 172.21.15.59:0/774521950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 msgr2=0x7fd87c19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 -- 172.21.15.59:0/774521950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c106410 msgr2=0x7fd87c1a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c106410 0x7fd87c1a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 -- 172.21.15.59:0/774521950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1a26b0 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd87aa00700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00d9b0 tx=0x7fd86c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c017030 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a29a0 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.834+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c004500 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.835+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c003910 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.835+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1a2f30 con 0x7fd87c104870 2024-04-02T17:12:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.836+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c0692b0 con 0x7fd87c104870 2024-04-02T17:12:38.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.838+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd86c01a070 con 0x7fd87c104870 2024-04-02T17:12:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.839+0000 7fd873400700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 0x7fd8600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.839+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fd86c0cf840 con 0x7fd87c104870 2024-04-02T17:12:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.839+0000 7fd87a000700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 0x7fd8600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.840+0000 7fd87a000700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 0x7fd8600a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd868009fd0 tx=0x7fd86801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:38.843+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd86c09c160 con 0x7fd87c104870 2024-04-02T17:12:39.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.079+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd87c063200 con 0x7fd87c104870 2024-04-02T17:12:39.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.081+0000 7fd873400700 1 -- 172.21.15.59:0/774521950 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5390 (secure 0 0 0) 0x7fd86c00fe80 con 0x7fd87c104870 2024-04-02T17:12:39.082 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 91 2024-04-02T17:12:39.082 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-04-02T17:01:33.642161+0000 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-04-02T17:11:52.214481+0000 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 26 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client luminous 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release squid 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-02T17:12:39.083 INFO:teuthology.orchestra.run.smithi059.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 12 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6802/1605463711,v1:172.21.15.59:6803/1605463711] [v2:172.21.15.59:6804/1605463711,v1:172.21.15.59:6805/1605463711] exists,up cb8100a6-a589-4808-b2c2-870b291c27de 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/212515491,v1:172.21.15.154:6801/212515491] [v2:172.21.15.154:6802/212515491,v1:172.21.15.154:6803/212515491] exists,up e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/4028786479,v1:172.21.15.164:6801/4028786479] [v2:172.21.15.164:6802/4028786479,v1:172.21.15.164:6803/4028786479] exists,up 8fde8bb0-3569-424d-8271-a0f77a83f346 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6810/3330665578,v1:172.21.15.59:6811/3330665578] [v2:172.21.15.59:6812/3330665578,v1:172.21.15.59:6813/3330665578] exists,up 73897ded-e965-4239-b88f-776b7af2eb73 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/2249512865,v1:172.21.15.154:6809/2249512865] [v2:172.21.15.154:6810/2249512865,v1:172.21.15.154:6811/2249512865] exists,up e81e729a-401f-4f5c-9749-6f88a4c93261 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/3065900784,v1:172.21.15.164:6809/3065900784] [v2:172.21.15.164:6810/3065900784,v1:172.21.15.164:6811/3065900784] exists,up 287aef0f-1e4b-4121-8e66-fad24be440d5 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6818/2899017867,v1:172.21.15.59:6819/2899017867] [v2:172.21.15.59:6820/2899017867,v1:172.21.15.59:6821/2899017867] exists,up 005113f3-2b16-467c-be8c-a79457eaab2c 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 up in weight 1 up_from 52 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/3912867880,v1:172.21.15.154:6817/3912867880] [v2:172.21.15.154:6818/3912867880,v1:172.21.15.154:6819/3912867880] exists,up d5a328eb-01df-4384-ba8a-ec41925acd17 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/1918014378,v1:172.21.15.164:6817/1918014378] [v2:172.21.15.164:6818/1918014378,v1:172.21.15.164:6819/1918014378] exists,up 77628eb3-b084-412c-9e27-2c9d9bdc3aeb 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 up in weight 1 up_from 68 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6826/980178556,v1:172.21.15.59:6827/980178556] [v2:172.21.15.59:6828/980178556,v1:172.21.15.59:6829/980178556] exists,up bbeabc6e-28cb-4639-9ef1-5982c75d2e25 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 up in weight 1 up_from 74 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/2125299053,v1:172.21.15.154:6825/2125299053] [v2:172.21.15.154:6826/2125299053,v1:172.21.15.154:6827/2125299053] exists,up 7079db2d-8f1f-4aea-b0dd-b7171210f38d 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/589108001,v1:172.21.15.164:6825/589108001] [v2:172.21.15.164:6826/589108001,v1:172.21.15.164:6827/589108001] exists,up 360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4 2024-04-02T17:12:39.084 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.0 [8,4] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.b [2,4] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.e [2,1] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.12 [5,11] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1d [8,11] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.2 [3,9] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.5 [3,9] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.e [4,8,5,9] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.1d [4,6] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.21 [4,6] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.23 [4,7] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.39 [5,8] 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/2274685378 expires 2024-04-03T17:02:49.350524+0000 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/2274685378 expires 2024-04-03T17:02:49.350524+0000 2024-04-02T17:12:39.085 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2569029589 expires 2024-04-03T17:02:15.389808+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2590435245 expires 2024-04-03T17:02:49.350524+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2036014369 expires 2024-04-03T17:02:49.350524+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2195394001 expires 2024-04-03T17:02:15.389808+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2817128980 expires 2024-04-03T17:02:49.350524+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/3477889503 expires 2024-04-03T17:02:15.389808+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/3477889503 expires 2024-04-03T17:02:15.389808+0000 2024-04-02T17:12:39.086 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3686798564 expires 2024-04-03T17:02:15.389808+0000 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 msgr2=0x7fd8600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 0x7fd8600a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd868009fd0 tx=0x7fd86801a040 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 msgr2=0x7fd87c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00d9b0 tx=0x7fd86c00dcc0 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 shutdown_connections 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd87c106410 0x7fd87c1a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd8600a3c40 0x7fd8600a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd87c104870 0x7fd87c19aa50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 --2- 172.21.15.59:0/774521950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87c105a70 0x7fd87c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 >> 172.21.15.59:0/774521950 conn(0x7fd87c0ffe10 msgr2=0x7fd87c108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 shutdown_connections 2024-04-02T17:12:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:39.084+0000 7fd881600700 1 -- 172.21.15.59:0/774521950 wait complete. 2024-04-02T17:12:39.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:39 smithi059 ceph-mon[31664]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:39.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:39 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/774521950' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T17:12:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:39 smithi164 ceph-mon[36499]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:39 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/774521950' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T17:12:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:39 smithi154 ceph-mon[36479]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-02T17:12:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:39 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/774521950' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T17:12:39.856 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-02T17:12:39.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 -- 172.21.15.59:0/3970312291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90104d40 msgr2=0x7fba901051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 --2- 172.21.15.59:0/3970312291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90104d40 0x7fba901051c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009900 tx=0x7fba7c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 -- 172.21.15.59:0/3970312291 shutdown_connections 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 --2- 172.21.15.59:0/3970312291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90104d40 0x7fba901051c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 --2- 172.21.15.59:0/3970312291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 0x7fba9010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 --2- 172.21.15.59:0/3970312291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90103b50 0x7fba90103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 -- 172.21.15.59:0/3970312291 >> 172.21.15.59:0/3970312291 conn(0x7fba900ff0e0 msgr2=0x7fba90101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:40.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.103+0000 7fba95600640 1 -- 172.21.15.59:0/3970312291 shutdown_connections 2024-04-02T17:12:40.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.104+0000 7fba95600640 1 -- 172.21.15.59:0/3970312291 wait complete. 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.104+0000 7fba95600640 1 Processor -- start 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 -- start start 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90104d40 0x7fba9019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 0x7fba901a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba9010f820 con 0x7fba90103b50 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba9010f6a0 con 0x7fba90104d40 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba95600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba9010f9a0 con 0x7fba90105700 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35622/0 (socket says 172.21.15.59:35622) 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba8fe00640 1 -- 172.21.15.59:0/1217018214 learned_addr learned my addr 172.21.15.59:0/1217018214 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:40.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba94c00640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 0x7fba901a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.105+0000 7fba8f400640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90104d40 0x7fba9019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 -- 172.21.15.59:0/1217018214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 msgr2=0x7fba901a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 0x7fba901a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 -- 172.21.15.59:0/1217018214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90104d40 msgr2=0x7fba9019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90104d40 0x7fba9019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 -- 172.21.15.59:0/1217018214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba901a1a30 con 0x7fba90103b50 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8fe00640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d770 tx=0x7fba8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80017070 con 0x7fba90103b50 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba901a1d20 con 0x7fba90103b50 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba80009c30 con 0x7fba90103b50 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80003dd0 con 0x7fba90103b50 2024-04-02T17:12:40.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.106+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba901a2180 con 0x7fba90103b50 2024-04-02T17:12:40.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.107+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9010ae10 con 0x7fba90103b50 2024-04-02T17:12:40.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.109+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fba80003f70 con 0x7fba90103b50 2024-04-02T17:12:40.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.111+0000 7fba8cc00640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 0x7fba740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.111+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fba800cfde0 con 0x7fba90103b50 2024-04-02T17:12:40.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.111+0000 7fba8f400640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 0x7fba740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.112+0000 7fba8f400640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 0x7fba740a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c240 tx=0x7fba7c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:40.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.116+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fba80009de0 con 0x7fba90103b50 2024-04-02T17:12:40.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.324+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba900627a0 con 0x7fba90103b50 2024-04-02T17:12:40.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.327+0000 7fba8cc00640 1 -- 172.21.15.59:0/1217018214 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7fba8009c0b0 con 0x7fba90103b50 2024-04-02T17:12:40.328 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:40.329 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 msgr2=0x7fba740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 0x7fba740a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c240 tx=0x7fba7c02df90 comp rx=0 tx=0).stop 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 msgr2=0x7fba90199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d770 tx=0x7fba8000dc30 comp rx=0 tx=0).stop 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 shutdown_connections 2024-04-02T17:12:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba90105700 0x7fba901a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba90104d40 0x7fba9019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fba740a3bb0 0x7fba740a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 --2- 172.21.15.59:0/1217018214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba90103b50 0x7fba90199e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 >> 172.21.15.59:0/1217018214 conn(0x7fba900ff0e0 msgr2=0x7fba90107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 shutdown_connections 2024-04-02T17:12:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.331+0000 7fba95600640 1 -- 172.21.15.59:0/1217018214 wait complete. 2024-04-02T17:12:40.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:40.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 -- 172.21.15.59:0/1935568539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 msgr2=0x7f535c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 --2- 172.21.15.59:0/1935568539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c073f90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5350009900 tx=0x7f535002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 -- 172.21.15.59:0/1935568539 shutdown_connections 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 --2- 172.21.15.59:0/1935568539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 --2- 172.21.15.59:0/1935568539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c0749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 --2- 172.21.15.59:0/1935568539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c073f90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.587+0000 7f5364600640 1 -- 172.21.15.59:0/1935568539 >> 172.21.15.59:0/1935568539 conn(0x7f535c0fd090 msgr2=0x7f535c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.588+0000 7f5364600640 1 -- 172.21.15.59:0/1935568539 shutdown_connections 2024-04-02T17:12:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.588+0000 7f5364600640 1 -- 172.21.15.59:0/1935568539 wait complete. 2024-04-02T17:12:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.589+0000 7f5364600640 1 Processor -- start 2024-04-02T17:12:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.589+0000 7f5364600640 1 -- start start 2024-04-02T17:12:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f535c10f9d0 con 0x7f535c075b40 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f535c10f850 con 0x7f535c1099e0 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5364600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f535c10fb50 con 0x7f535c074560 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54772/0 (socket says 172.21.15.59:54772) 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361c00640 1 -- 172.21.15.59:0/2212883319 learned_addr learned my addr 172.21.15.59:0/2212883319 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5362600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361c00640 1 -- 172.21.15.59:0/2212883319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 msgr2=0x7f535c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.590+0000 7f5361c00640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5361c00640 1 -- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 msgr2=0x7f535c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5361c00640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5361c00640 1 -- 172.21.15.59:0/2212883319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535c1a1c30 con 0x7f535c074560 2024-04-02T17:12:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5362600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5361200640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5361c00640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f535002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350043020 con 0x7f535c074560 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535c1a1ec0 con 0x7f535c074560 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53500054e0 con 0x7f535c074560 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.591+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350003c40 con 0x7f535c074560 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.592+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535c1a2330 con 0x7f535c074560 2024-04-02T17:12:40.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.593+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f535c108710 con 0x7f535c074560 2024-04-02T17:12:40.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.595+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5350046020 con 0x7f535c074560 2024-04-02T17:12:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.596+0000 7f535a000640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 0x7f53440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.596+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f53500f2c40 con 0x7f535c074560 2024-04-02T17:12:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.596+0000 7f5361200640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 0x7f53440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.597+0000 7f5361200640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 0x7f53440a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f535c19b520 tx=0x7f5348005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:40.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.599+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f535004d020 con 0x7f535c074560 2024-04-02T17:12:40.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.806+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f535c02d5c0 con 0x7f535c074560 2024-04-02T17:12:40.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.807+0000 7f535a000640 1 -- 172.21.15.59:0/2212883319 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f53500bf560 con 0x7f535c074560 2024-04-02T17:12:40.807 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:40.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 msgr2=0x7f53440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 0x7f53440a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f535c19b520 tx=0x7f5348005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 msgr2=0x7f535c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f535002c650 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 shutdown_connections 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53440a3bb0 0x7f53440a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535c075b40 0x7f535c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c1099e0 0x7f535c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 --2- 172.21.15.59:0/2212883319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f535c074560 0x7f535c19a010 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 >> 172.21.15.59:0/2212883319 conn(0x7f535c0fd090 msgr2=0x7f535c1058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 shutdown_connections 2024-04-02T17:12:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:40.810+0000 7f5364600640 1 -- 172.21.15.59:0/2212883319 wait complete. 2024-04-02T17:12:40.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 -- 172.21.15.59:0/3534557008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109bd0 msgr2=0x7f1300110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 --2- 172.21.15.59:0/3534557008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109bd0 0x7f1300110460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009860 tx=0x7f12f802bb70 comp rx=0 tx=0).stop 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 -- 172.21.15.59:0/3534557008 shutdown_connections 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 --2- 172.21.15.59:0/3534557008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300109210 0x7f1300109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 --2- 172.21.15.59:0/3534557008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109bd0 0x7f1300110460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 --2- 172.21.15.59:0/3534557008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300108020 0x7f1300108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 -- 172.21.15.59:0/3534557008 >> 172.21.15.59:0/3534557008 conn(0x7f1300076fd0 msgr2=0x7f13000793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 -- 172.21.15.59:0/3534557008 shutdown_connections 2024-04-02T17:12:41.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.057+0000 7f1304a00640 1 -- 172.21.15.59:0/3534557008 wait complete. 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.058+0000 7f1304a00640 1 Processor -- start 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.058+0000 7f1304a00640 1 -- start start 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300108020 0x7f130019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109210 0x7f130019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1300113cb0 con 0x7f1300108020 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1300113b30 con 0x7f1300109bd0 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f1304a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1300113e30 con 0x7f1300109210 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12ff200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12fe800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300108020 0x7f130019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12ff200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38564/0 (socket says 172.21.15.59:38564) 2024-04-02T17:12:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12ff200640 1 -- 172.21.15.59:0/1067113739 learned_addr learned my addr 172.21.15.59:0/1067113739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109210 0x7f130019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12ff200640 1 -- 172.21.15.59:0/1067113739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109210 msgr2=0x7f130019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.059+0000 7f12ff200640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109210 0x7f130019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12ff200640 1 -- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300108020 msgr2=0x7f130019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12ff200640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300108020 0x7f130019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12ff200640 1 -- 172.21.15.59:0/1067113739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13001a5ea0 con 0x7f1300109bd0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12ff200640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12f8034b10 tx=0x7f12f8032470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8041030 con 0x7f1300109bd0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13001a6130 con 0x7f1300109bd0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f802c510 con 0x7f1300109bd0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8039040 con 0x7f1300109bd0 2024-04-02T17:12:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.060+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13001a6620 con 0x7f1300109bd0 2024-04-02T17:12:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.062+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130010f340 con 0x7f1300109bd0 2024-04-02T17:12:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.063+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f12f8044070 con 0x7f1300109bd0 2024-04-02T17:12:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.065+0000 7f12f7e00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 0x7f12e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.065+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f12f80f1900 con 0x7f1300109bd0 2024-04-02T17:12:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.065+0000 7f12fe800640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 0x7f12e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.066+0000 7f12fe800640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 0x7f12e40a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1300109070 tx=0x7f12f0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.069+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f12f80be220 con 0x7f1300109bd0 2024-04-02T17:12:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.279+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1300062700 con 0x7f1300109bd0 2024-04-02T17:12:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.279+0000 7f12f7e00640 1 -- 172.21.15.59:0/1067113739 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f12f80bdb20 con 0x7f1300109bd0 2024-04-02T17:12:41.280 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:41.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 msgr2=0x7f12e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 0x7f12e40a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1300109070 tx=0x7f12f0010040 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 msgr2=0x7f13001a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12f8034b10 tx=0x7f12f8032470 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 shutdown_connections 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1300109bd0 0x7f13001a5830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1300109210 0x7f130019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12e40a3bb0 0x7f12e40a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 --2- 172.21.15.59:0/1067113739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1300108020 0x7f130019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 >> 172.21.15.59:0/1067113739 conn(0x7f1300076fd0 msgr2=0x7f130010c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 shutdown_connections 2024-04-02T17:12:41.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.282+0000 7f1304a00640 1 -- 172.21.15.59:0/1067113739 wait complete. 2024-04-02T17:12:41.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 -- 172.21.15.59:0/3487688283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0103d20 msgr2=0x7fa1d0104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 --2- 172.21.15.59:0/3487688283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0103d20 0x7fa1d0104140 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009900 tx=0x7fa1c402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 -- 172.21.15.59:0/3487688283 shutdown_connections 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 --2- 172.21.15.59:0/3487688283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d010c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 --2- 172.21.15.59:0/3487688283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0104f10 0x7fa1d0105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 --2- 172.21.15.59:0/3487688283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0103d20 0x7fa1d0104140 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.532+0000 7fa1d7600640 1 -- 172.21.15.59:0/3487688283 >> 172.21.15.59:0/3487688283 conn(0x7fa1d00ff2b0 msgr2=0x7fa1d01016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.533+0000 7fa1d7600640 1 -- 172.21.15.59:0/3487688283 shutdown_connections 2024-04-02T17:12:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.533+0000 7fa1d7600640 1 -- 172.21.15.59:0/3487688283 wait complete. 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 Processor -- start 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 -- start start 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0103d20 0x7fa1d019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d01a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010f9f0 con 0x7fa1d0104f10 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010f870 con 0x7fa1d0103d20 2024-04-02T17:12:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d7600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010fb70 con 0x7fa1d01058d0 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38288/0 (socket says 172.21.15.59:38288) 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1cfe00640 1 -- 172.21.15.59:0/2645709091 learned_addr learned my addr 172.21.15.59:0/2645709091 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d5600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d01a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.534+0000 7fa1d4c00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0103d20 0x7fa1d019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 -- 172.21.15.59:0/2645709091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 msgr2=0x7fa1d01a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d01a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 -- 172.21.15.59:0/2645709091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0103d20 msgr2=0x7fa1d019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0103d20 0x7fa1d019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 -- 172.21.15.59:0/2645709091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01a1c30 con 0x7fa1d0104f10 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1d5600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00e9f0 tx=0x7fa1bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.535+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00cd80 con 0x7fa1d0104f10 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.536+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1bc010910 con 0x7fa1d0104f10 2024-04-02T17:12:41.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.536+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a1f20 con 0x7fa1d0104f10 2024-04-02T17:12:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.536+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0188f0 con 0x7fa1d0104f10 2024-04-02T17:12:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.536+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a23e0 con 0x7fa1d0104f10 2024-04-02T17:12:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.537+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d010b040 con 0x7fa1d0104f10 2024-04-02T17:12:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.539+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1bc010ac0 con 0x7fa1d0104f10 2024-04-02T17:12:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.541+0000 7fa1cd600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.541+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa1bc0d0830 con 0x7fa1d0104f10 2024-04-02T17:12:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.541+0000 7fa1d4c00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:41.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.542+0000 7fa1d4c00640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa1c402c170 tx=0x7fa1c402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:41.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.545+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1bc09cb00 con 0x7fa1d0104f10 2024-04-02T17:12:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:41 smithi059 ceph-mon[31664]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1217018214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2212883319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:41 smithi164 ceph-mon[36499]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1217018214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2212883319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:41 smithi154 ceph-mon[36479]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1217018214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:41.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2212883319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:41.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.752+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1d00627a0 con 0x7fa1d0104f10 2024-04-02T17:12:41.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.753+0000 7fa1cd600640 1 -- 172.21.15.59:0/2645709091 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fa1bc0a0e00 con 0x7fa1d0104f10 2024-04-02T17:12:41.753 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 msgr2=0x7fa1b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa1c402c170 tx=0x7fa1c402df90 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 msgr2=0x7fa1d019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00e9f0 tx=0x7fa1bc00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 shutdown_connections 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d0104f10 0x7fa1d019a550 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1d01058d0 0x7fa1d01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 --2- 172.21.15.59:0/2645709091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1d0103d20 0x7fa1d019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 >> 172.21.15.59:0/2645709091 conn(0x7fa1d00ff2b0 msgr2=0x7fa1d0108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 shutdown_connections 2024-04-02T17:12:41.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:41.756+0000 7fa1d7600640 1 -- 172.21.15.59:0/2645709091 wait complete. 2024-04-02T17:12:41.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 -- 172.21.15.59:0/2658186533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c074560 msgr2=0x7fc00c0749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 --2- 172.21.15.59:0/2658186533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c074560 0x7fc00c0749e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009900 tx=0x7fbff402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 -- 172.21.15.59:0/2658186533 shutdown_connections 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 --2- 172.21.15.59:0/2658186533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c074560 0x7fc00c0749e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 --2- 172.21.15.59:0/2658186533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 --2- 172.21.15.59:0/2658186533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c075b40 0x7fc00c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 -- 172.21.15.59:0/2658186533 >> 172.21.15.59:0/2658186533 conn(0x7fc00c0fce90 msgr2=0x7fc00c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 -- 172.21.15.59:0/2658186533 shutdown_connections 2024-04-02T17:12:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.002+0000 7fc00b200640 1 -- 172.21.15.59:0/2658186533 wait complete. 2024-04-02T17:12:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.003+0000 7fc00b200640 1 Processor -- start 2024-04-02T17:12:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.003+0000 7fc00b200640 1 -- start start 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.003+0000 7fc00b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.003+0000 7fc00b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 0x7fc00c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f7f0 con 0x7fc00c075b40 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f670 con 0x7fc00c109800 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f970 con 0x7fc00c074560 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc003e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 0x7fc00c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40002/0 (socket says 172.21.15.59:40002) 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc003e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 0x7fc00c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38310/0 (socket says 172.21.15.59:38310) 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 -- 172.21.15.59:0/452587943 learned_addr learned my addr 172.21.15.59:0/452587943 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00a800640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 -- 172.21.15.59:0/452587943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 msgr2=0x7fc00c1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 -- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 msgr2=0x7fc00c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 0x7fc00c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc009e00640 1 -- 172.21.15.59:0/452587943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a1a60 con 0x7fc00c074560 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.004+0000 7fc00a800640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc009e00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e9f0 tx=0x7fbffc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00cd80 con 0x7fc00c074560 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a1d50 con 0x7fc00c074560 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc005500 con 0x7fc00c074560 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc010690 con 0x7fc00c074560 2024-04-02T17:12:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.005+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a21c0 con 0x7fc00c074560 2024-04-02T17:12:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.006+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c108530 con 0x7fc00c074560 2024-04-02T17:12:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.008+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbffc0046c0 con 0x7fc00c074560 2024-04-02T17:12:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.010+0000 7fc002a00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 0x7fbff00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.010+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fbffc0d0950 con 0x7fc00c074560 2024-04-02T17:12:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.010+0000 7fc003e00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 0x7fbff00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.011+0000 7fc003e00640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 0x7fbff00a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbff402c240 tx=0x7fbff402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.014+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbffc09d270 con 0x7fc00c074560 2024-04-02T17:12:42.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.269+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc00c0627a0 con 0x7fc00c074560 2024-04-02T17:12:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.270+0000 7fc002a00640 1 -- 172.21.15.59:0/452587943 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7fbffc09cb70 con 0x7fc00c074560 2024-04-02T17:12:42.270 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:42.271 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:12:42.271 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.272+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 msgr2=0x7fbff00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.272+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 0x7fbff00a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbff402c240 tx=0x7fbff402df90 comp rx=0 tx=0).stop 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.272+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 msgr2=0x7fc00c199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.272+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e9f0 tx=0x7fbffc00edf0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 shutdown_connections 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbff00a3bb0 0x7fbff00a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00c075b40 0x7fc00c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc00c109800 0x7fc00c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 --2- 172.21.15.59:0/452587943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c074560 0x7fc00c199e40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 >> 172.21.15.59:0/452587943 conn(0x7fc00c0fce90 msgr2=0x7fc00c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 shutdown_connections 2024-04-02T17:12:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.273+0000 7fc00b200640 1 -- 172.21.15.59:0/452587943 wait complete. 2024-04-02T17:12:42.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 -- 172.21.15.59:0/1741435329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c1036c0 msgr2=0x7fb92c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 --2- 172.21.15.59:0/1741435329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c1036c0 0x7fb92c109f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 -- 172.21.15.59:0/1741435329 shutdown_connections 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 --2- 172.21.15.59:0/1741435329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c1036c0 0x7fb92c109f50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 --2- 172.21.15.59:0/1741435329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c102d00 0x7fb92c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 --2- 172.21.15.59:0/1741435329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c101b10 0x7fb92c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 -- 172.21.15.59:0/1741435329 >> 172.21.15.59:0/1741435329 conn(0x7fb92c0fd0e0 msgr2=0x7fb92c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 -- 172.21.15.59:0/1741435329 shutdown_connections 2024-04-02T17:12:42.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.516+0000 7fb934000640 1 -- 172.21.15.59:0/1741435329 wait complete. 2024-04-02T17:12:42.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.517+0000 7fb934000640 1 Processor -- start 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.517+0000 7fb934000640 1 -- start start 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c101b10 0x7fb92c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c102d00 0x7fb92c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10d7c0 con 0x7fb92c102d00 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10d640 con 0x7fb92c101b10 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb934000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10d940 con 0x7fb92c1036c0 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb932000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb932000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40024/0 (socket says 172.21.15.59:40024) 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb932000640 1 -- 172.21.15.59:0/681239283 learned_addr learned my addr 172.21.15.59:0/681239283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb930c00640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c102d00 0x7fb92c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.518+0000 7fb931600640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c101b10 0x7fb92c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 -- 172.21.15.59:0/681239283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c101b10 msgr2=0x7fb92c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c101b10 0x7fb92c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 -- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c102d00 msgr2=0x7fb92c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c102d00 0x7fb92c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 -- 172.21.15.59:0/681239283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c19f9f0 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb932000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb92402c8e0 tx=0x7fb92402c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924043020 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9240054e0 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c19fc80 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924003d00 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.519+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c1a01d0 con 0x7fb92c1036c0 2024-04-02T17:12:42.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.520+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c108dd0 con 0x7fb92c1036c0 2024-04-02T17:12:42.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.523+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb924046020 con 0x7fb92c1036c0 2024-04-02T17:12:42.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.524+0000 7fb92a000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 0x7fb9140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.524+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb9240f2a40 con 0x7fb92c1036c0 2024-04-02T17:12:42.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.524+0000 7fb931600640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 0x7fb9140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.525+0000 7fb931600640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 0x7fb9140a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb92c102b60 tx=0x7fb920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.526+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb92404d020 con 0x7fb92c1036c0 2024-04-02T17:12:42.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:42 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1067113739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:42 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2645709091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:42 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1067113739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:42 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2645709091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:42 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1067113739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:42 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2645709091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.733+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb92c02d4f0 con 0x7fb92c1036c0 2024-04-02T17:12:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.735+0000 7fb92a000640 1 -- 172.21.15.59:0/681239283 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7fb9240dde20 con 0x7fb92c1036c0 2024-04-02T17:12:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:42.736 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:42.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 msgr2=0x7fb9140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 0x7fb9140a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb92c102b60 tx=0x7fb920005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 msgr2=0x7fb92c19f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb92402c8e0 tx=0x7fb92402c640 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 shutdown_connections 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9140a3bb0 0x7fb9140a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c102d00 0x7fb92c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb92c1036c0 0x7fb92c19f380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 --2- 172.21.15.59:0/681239283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb92c101b10 0x7fb92c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 >> 172.21.15.59:0/681239283 conn(0x7fb92c0fd0e0 msgr2=0x7fb92c105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 shutdown_connections 2024-04-02T17:12:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.738+0000 7fb934000640 1 -- 172.21.15.59:0/681239283 wait complete. 2024-04-02T17:12:42.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:42.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 -- 172.21.15.59:0/3456797608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc154103d20 msgr2=0x7fc154104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 --2- 172.21.15.59:0/3456797608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc154103d20 0x7fc154104140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc144009900 tx=0x7fc14402bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 -- 172.21.15.59:0/3456797608 shutdown_connections 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 --2- 172.21.15.59:0/3456797608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1541058d0 0x7fc15410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 --2- 172.21.15.59:0/3456797608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154104f10 0x7fc154105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 --2- 172.21.15.59:0/3456797608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc154103d20 0x7fc154104140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 -- 172.21.15.59:0/3456797608 >> 172.21.15.59:0/3456797608 conn(0x7fc1540ff2b0 msgr2=0x7fc1541016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 -- 172.21.15.59:0/3456797608 shutdown_connections 2024-04-02T17:12:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.980+0000 7fc15aa00640 1 -- 172.21.15.59:0/3456797608 wait complete. 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.981+0000 7fc15aa00640 1 Processor -- start 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 -- start start 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc154104f10 0x7fc15419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1541058d0 0x7fc1541a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc15410f9c0 con 0x7fc154104f10 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc15410f840 con 0x7fc154103d20 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc15aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc15410fb40 con 0x7fc1541058d0 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc153e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc153e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38620/0 (socket says 172.21.15.59:38620) 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc153e00640 1 -- 172.21.15.59:0/191220982 learned_addr learned my addr 172.21.15.59:0/191220982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.982+0000 7fc153400640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc154104f10 0x7fc15419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc158a00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1541058d0 0x7fc1541a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 -- 172.21.15.59:0/191220982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1541058d0 msgr2=0x7fc1541a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1541058d0 0x7fc1541a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 -- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc154104f10 msgr2=0x7fc15419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc154104f10 0x7fc15419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 -- 172.21.15.59:0/191220982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1541a1c50 con 0x7fc154103d20 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc153e00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc144005ab0 tx=0x7fc14402c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144041070 con 0x7fc154103d20 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.983+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc144004450 con 0x7fc154103d20 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.984+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144034440 con 0x7fc154103d20 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.984+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1541a1ee0 con 0x7fc154103d20 2024-04-02T17:12:42.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.984+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1541a23a0 con 0x7fc154103d20 2024-04-02T17:12:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.985+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15410b040 con 0x7fc154103d20 2024-04-02T17:12:42.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.987+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc144044070 con 0x7fc154103d20 2024-04-02T17:12:42.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.988+0000 7fc150c00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 0x7fc1380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:42.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.989+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc1440f2300 con 0x7fc154103d20 2024-04-02T17:12:42.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.989+0000 7fc153400640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 0x7fc1380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:42.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.989+0000 7fc153400640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 0x7fc1380a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc15419b540 tx=0x7fc13c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:42.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:42.990+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc1440be5d0 con 0x7fc154103d20 2024-04-02T17:12:43.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.202+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc15402d500 con 0x7fc154103d20 2024-04-02T17:12:43.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.203+0000 7fc150c00640 1 -- 172.21.15.59:0/191220982 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fc1440c28d0 con 0x7fc154103d20 2024-04-02T17:12:43.206 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.205+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 msgr2=0x7fc1380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.205+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 0x7fc1380a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc15419b540 tx=0x7fc13c00b390 comp rx=0 tx=0).stop 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 msgr2=0x7fc15419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc144005ab0 tx=0x7fc14402c6e0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 shutdown_connections 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1380a3c30 0x7fc1380a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc154104f10 0x7fc15419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1541058d0 0x7fc1541a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 --2- 172.21.15.59:0/191220982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc154103d20 0x7fc15419a030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 >> 172.21.15.59:0/191220982 conn(0x7fc1540ff2b0 msgr2=0x7fc154108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:43.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 shutdown_connections 2024-04-02T17:12:43.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.206+0000 7fc15aa00640 1 -- 172.21.15.59:0/191220982 wait complete. 2024-04-02T17:12:43.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.446+0000 7f76e1c00640 1 -- 172.21.15.59:0/2601679941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc077700 msgr2=0x7f76dc077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.446+0000 7f76e1c00640 1 --2- 172.21.15.59:0/2601679941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc077700 0x7f76dc077b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009900 tx=0x7f76c802bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 -- 172.21.15.59:0/2601679941 shutdown_connections 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 --2- 172.21.15.59:0/2601679941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc077700 0x7f76dc077b60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 --2- 172.21.15.59:0/2601679941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc0780a0 0x7f76dc078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 --2- 172.21.15.59:0/2601679941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0764c0 0x7f76dc0768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 -- 172.21.15.59:0/2601679941 >> 172.21.15.59:0/2601679941 conn(0x7f76dc0ff2b0 msgr2=0x7f76dc1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 -- 172.21.15.59:0/2601679941 shutdown_connections 2024-04-02T17:12:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.447+0000 7f76e1c00640 1 -- 172.21.15.59:0/2601679941 wait complete. 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 Processor -- start 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 -- start start 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc077700 0x7f76dc19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 0x7f76dc1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc113da0 con 0x7f76dc0764c0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc113c20 con 0x7f76dc0780a0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.448+0000 7f76e1c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc113f20 con 0x7f76dc077700 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76daa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76daa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38346/0 (socket says 172.21.15.59:38346) 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76daa00640 1 -- 172.21.15.59:0/3819133856 learned_addr learned my addr 172.21.15.59:0/3819133856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 0x7f76dc1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76da000640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc077700 0x7f76dc19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 -- 172.21.15.59:0/3819133856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc077700 msgr2=0x7f76dc19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc077700 0x7f76dc19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 -- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 msgr2=0x7f76dc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 -- 172.21.15.59:0/3819133856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc1a6090 con 0x7f76dc0780a0 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76daa00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.449+0000 7f76db400640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 0x7f76dc1a5a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ec50 tx=0x7f76d400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.450+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4003b70 con 0x7f76dc0780a0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.450+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc1a6380 con 0x7f76dc0780a0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.450+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d4003d10 con 0x7f76dc0780a0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.450+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4010890 con 0x7f76dc0780a0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.450+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc1a6850 con 0x7f76dc0780a0 2024-04-02T17:12:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.451+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc1108a0 con 0x7f76dc0780a0 2024-04-02T17:12:43.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.453+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76d4003fc0 con 0x7f76dc0780a0 2024-04-02T17:12:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.454+0000 7f76d3400640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 0x7f76c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.454+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f76d40d0a40 con 0x7f76dc0780a0 2024-04-02T17:12:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.455+0000 7f76daa00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 0x7f76c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.455+0000 7f76daa00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 0x7f76c00a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011550 tx=0x7f76cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:43.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.459+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f76d409d360 con 0x7f76dc0780a0 2024-04-02T17:12:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:43 smithi059 ceph-mon[31664]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:43 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/452587943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:43 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/681239283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:43 smithi164 ceph-mon[36499]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:43 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/452587943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:43 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/681239283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:43 smithi154 ceph-mon[36479]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:43 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/452587943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:43 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/681239283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:43.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.666+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76dc062700 con 0x7f76dc0780a0 2024-04-02T17:12:43.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.667+0000 7f76d3400640 1 -- 172.21.15.59:0/3819133856 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f76d40bbe20 con 0x7f76dc0780a0 2024-04-02T17:12:43.668 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 msgr2=0x7f76c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 0x7f76c00a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76cc011550 tx=0x7f76cc005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 msgr2=0x7f76dc1a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 0x7f76dc1a5a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ec50 tx=0x7f76d400c6a0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 shutdown_connections 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76dc0780a0 0x7f76dc1a5a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76dc077700 0x7f76dc19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76c00a3bb0 0x7f76c00a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 --2- 172.21.15.59:0/3819133856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc0764c0 0x7f76dc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.670+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 >> 172.21.15.59:0/3819133856 conn(0x7f76dc0ff2b0 msgr2=0x7f76dc1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.671+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 shutdown_connections 2024-04-02T17:12:43.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.671+0000 7f76e1c00640 1 -- 172.21.15.59:0/3819133856 wait complete. 2024-04-02T17:12:43.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 -- 172.21.15.59:0/3314499411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 msgr2=0x7fa4b8109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 --2- 172.21.15.59:0/3314499411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8109b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009ca0 tx=0x7fa4b002c100 comp rx=0 tx=0).stop 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 -- 172.21.15.59:0/3314499411 shutdown_connections 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 --2- 172.21.15.59:0/3314499411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8109b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 --2- 172.21.15.59:0/3314499411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8104db0 0x7fa4b81071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 --2- 172.21.15.59:0/3314499411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8102480 0x7fa4b8104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 -- 172.21.15.59:0/3314499411 >> 172.21.15.59:0/3314499411 conn(0x7fa4b80fbe70 msgr2=0x7fa4b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 -- 172.21.15.59:0/3314499411 shutdown_connections 2024-04-02T17:12:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.919+0000 7fa4bde00640 1 -- 172.21.15.59:0/3314499411 wait complete. 2024-04-02T17:12:43.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.920+0000 7fa4bde00640 1 Processor -- start 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.920+0000 7fa4bde00640 1 -- start start 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8102480 0x7fa4b81966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8104db0 0x7fa4b8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810d680 con 0x7fa4b81076e0 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810d500 con 0x7fa4b8102480 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810d800 con 0x7fa4b8104db0 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38358/0 (socket says 172.21.15.59:38358) 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 -- 172.21.15.59:0/637027390 learned_addr learned my addr 172.21.15.59:0/637027390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4afe00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8104db0 0x7fa4b8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bca00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8102480 0x7fa4b81966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 -- 172.21.15.59:0/637027390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8104db0 msgr2=0x7fa4b8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8104db0 0x7fa4b8196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 -- 172.21.15.59:0/637027390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8102480 msgr2=0x7fa4b81966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8102480 0x7fa4b81966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.921+0000 7fa4bd400640 1 -- 172.21.15.59:0/637027390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8193b00 con 0x7fa4b81076e0 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4bd400640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0037040 tx=0x7fa4b0035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0039040 con 0x7fa4b81076e0 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81a6300 con 0x7fa4b81076e0 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0044070 con 0x7fa4b81076e0 2024-04-02T17:12:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00043a0 con 0x7fa4b81076e0 2024-04-02T17:12:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.922+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a66b0 con 0x7fa4b81076e0 2024-04-02T17:12:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.923+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b81091d0 con 0x7fa4b81076e0 2024-04-02T17:12:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.925+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa4b0004540 con 0x7fa4b81076e0 2024-04-02T17:12:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.927+0000 7fa4b6000640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.927+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa4b00f28b0 con 0x7fa4b81076e0 2024-04-02T17:12:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.927+0000 7fa4bca00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.928+0000 7fa4bca00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009bd0 tx=0x7fa4a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:43.931+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa4b004e020 con 0x7fa4b81076e0 2024-04-02T17:12:44.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.139+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4b8062700 con 0x7fa4b81076e0 2024-04-02T17:12:44.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.139+0000 7fa4b6000640 1 -- 172.21.15.59:0/637027390 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fa4b00bf1d0 con 0x7fa4b81076e0 2024-04-02T17:12:44.140 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:44.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 msgr2=0x7fa49c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009bd0 tx=0x7fa4a8005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 msgr2=0x7fa4b8193240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0037040 tx=0x7fa4b0035d90 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 shutdown_connections 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4b81076e0 0x7fa4b8193240 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4b8104db0 0x7fa4b8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 --2- 172.21.15.59:0/637027390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4b8102480 0x7fa4b81966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 >> 172.21.15.59:0/637027390 conn(0x7fa4b80fbe70 msgr2=0x7fa4b8101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 shutdown_connections 2024-04-02T17:12:44.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.142+0000 7fa4bde00640 1 -- 172.21.15.59:0/637027390 wait complete. 2024-04-02T17:12:44.164 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 -- 172.21.15.59:0/4271065780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8105700 msgr2=0x7fe4c810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/4271065780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8105700 0x7fe4c810bf90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 -- 172.21.15.59:0/4271065780 shutdown_connections 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/4271065780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8105700 0x7fe4c810bf90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/4271065780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8104d40 0x7fe4c81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/4271065780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 0x7fe4c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 -- 172.21.15.59:0/4271065780 >> 172.21.15.59:0/4271065780 conn(0x7fe4c80ff0e0 msgr2=0x7fe4c8101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 -- 172.21.15.59:0/4271065780 shutdown_connections 2024-04-02T17:12:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.388+0000 7fe4c7e00640 1 -- 172.21.15.59:0/4271065780 wait complete. 2024-04-02T17:12:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.389+0000 7fe4c7e00640 1 Processor -- start 2024-04-02T17:12:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.389+0000 7fe4c7e00640 1 -- start start 2024-04-02T17:12:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 0x7fe4c8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 0x7fe4c81a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810f7d0 con 0x7fe4c8104d40 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810f650 con 0x7fe4c8105700 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810f950 con 0x7fe4c8103b50 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 0x7fe4c8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38376/0 (socket says 172.21.15.59:38376) 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 -- 172.21.15.59:0/3320578679 learned_addr learned my addr 172.21.15.59:0/3320578679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7400640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 0x7fe4c81a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 -- 172.21.15.59:0/3320578679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 msgr2=0x7fe4c8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 0x7fe4c8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 -- 172.21.15.59:0/3320578679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 msgr2=0x7fe4c81a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 0x7fe4c81a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4be000640 1 -- 172.21.15.59:0/3320578679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c81a1aa0 con 0x7fe4c8104d40 2024-04-02T17:12:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.390+0000 7fe4c7400640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 0x7fe4c81a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4be000640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e9f0 tx=0x7fe4b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000cd80 con 0x7fe4c8104d40 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c81a1d90 con 0x7fe4c8104d40 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0010910 con 0x7fe4c8104d40 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00188f0 con 0x7fe4c8104d40 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.391+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a2260 con 0x7fe4c8104d40 2024-04-02T17:12:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.392+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c810ae10 con 0x7fe4c8104d40 2024-04-02T17:12:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.394+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe4b0010ac0 con 0x7fe4c8104d40 2024-04-02T17:12:44.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.396+0000 7fe4cca00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 0x7fe4ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.396+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe4b00d08f0 con 0x7fe4c8104d40 2024-04-02T17:12:44.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.396+0000 7fe4c6a00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 0x7fe4ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.397+0000 7fe4c6a00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 0x7fe4ac0a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8104ba0 tx=0x7fe4b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:44.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.400+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe4b009d210 con 0x7fe4c8104d40 2024-04-02T17:12:44.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:44 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/191220982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:44 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3819133856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:44 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/637027390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:44 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/191220982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:44 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3819133856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:44 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/637027390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:44 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/191220982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:44 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3819133856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:44 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/637027390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:44.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.650+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4c80627a0 con 0x7fe4c8104d40 2024-04-02T17:12:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.651+0000 7fe4cca00640 1 -- 172.21.15.59:0/3320578679 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7fe4b009cb10 con 0x7fe4c8104d40 2024-04-02T17:12:44.652 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:44.652 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:10:43.085475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:12:44.652 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-02T17:12:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 msgr2=0x7fe4ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 0x7fe4ac0a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8104ba0 tx=0x7fe4b8009500 comp rx=0 tx=0).stop 2024-04-02T17:12:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 msgr2=0x7fe4c819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e9f0 tx=0x7fe4b000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 shutdown_connections 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe4ac0a3bb0 0x7fe4ac0a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c8104d40 0x7fe4c819a3c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4c8105700 0x7fe4c81a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 --2- 172.21.15.59:0/3320578679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4c8103b50 0x7fe4c8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 >> 172.21.15.59:0/3320578679 conn(0x7fe4c80ff0e0 msgr2=0x7fe4c8107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.654+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 shutdown_connections 2024-04-02T17:12:44.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.655+0000 7fe4c7e00640 1 -- 172.21.15.59:0/3320578679 wait complete. 2024-04-02T17:12:44.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 -- 172.21.15.59:0/3852046562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 msgr2=0x7f23f41097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 --2- 172.21.15.59:0/3852046562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f41097b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009860 tx=0x7f23dc02bb70 comp rx=0 tx=0).stop 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 -- 172.21.15.59:0/3852046562 shutdown_connections 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 --2- 172.21.15.59:0/3852046562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 0x7f23f4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 --2- 172.21.15.59:0/3852046562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f41097b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 --2- 172.21.15.59:0/3852046562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f4108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 -- 172.21.15.59:0/3852046562 >> 172.21.15.59:0/3852046562 conn(0x7f23f4077110 msgr2=0x7f23f4079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 -- 172.21.15.59:0/3852046562 shutdown_connections 2024-04-02T17:12:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.897+0000 7f23fa600640 1 -- 172.21.15.59:0/3852046562 wait complete. 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 Processor -- start 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 -- start start 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 0x7f23f41a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4113dd0 con 0x7f23f4108140 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4113c50 con 0x7f23f4109cf0 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.898+0000 7f23fa600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4113f50 con 0x7f23f4109330 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38394/0 (socket says 172.21.15.59:38394) 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 -- 172.21.15.59:0/1219467380 learned_addr learned my addr 172.21.15.59:0/1219467380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23ebe00640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3e00640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 0x7f23f41a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 -- 172.21.15.59:0/1219467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 msgr2=0x7f23f419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 -- 172.21.15.59:0/1219467380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 msgr2=0x7f23f41a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 0x7f23f41a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 -- 172.21.15.59:0/1219467380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f41a6020 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.899+0000 7f23f3400640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23e400e9f0 tx=0x7f23e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.900+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e400ccc0 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.900+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e40035f0 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.900+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f41a6310 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.900+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e400ce60 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.900+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a6830 con 0x7f23f4108140 2024-04-02T17:12:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.901+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f410f460 con 0x7f23f4108140 2024-04-02T17:12:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.903+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f23e40037a0 con 0x7f23f4108140 2024-04-02T17:12:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.904+0000 7f23f0c00640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 0x7f23d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.904+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f23e40d06f0 con 0x7f23f4108140 2024-04-02T17:12:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.905+0000 7f23ebe00640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 0x7f23d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.905+0000 7f23ebe00640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 0x7f23d80a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009830 tx=0x7f23dc034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:44.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:44.909+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f23e409d010 con 0x7f23f4108140 2024-04-02T17:12:45.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.116+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23f4062700 con 0x7f23f4108140 2024-04-02T17:12:45.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.118+0000 7f23f0c00640 1 -- 172.21.15.59:0/1219467380 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f23e409c910 con 0x7f23f4108140 2024-04-02T17:12:45.120 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:45.120 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:45.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 msgr2=0x7f23d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 0x7f23d80a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009830 tx=0x7f23dc034ab0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 msgr2=0x7f23f419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23e400e9f0 tx=0x7f23e400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 shutdown_connections 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f4109cf0 0x7f23f41a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f4109330 0x7f23f419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23d80a3bb0 0x7f23d80a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 --2- 172.21.15.59:0/1219467380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f4108140 0x7f23f419e400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 >> 172.21.15.59:0/1219467380 conn(0x7f23f4077110 msgr2=0x7f23f410c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 shutdown_connections 2024-04-02T17:12:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.121+0000 7f23fa600640 1 -- 172.21.15.59:0/1219467380 wait complete. 2024-04-02T17:12:45.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T17:10:34.529898+0000', 'modified': '2024-04-02T17:10:43.085475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24584, 'mds_1': 24560, 'mds_2': 24547, 'mds_3': 24530, 'mds_4': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24454': {'gid': 24454, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/3588884595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 3588884595}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 3588884595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6833/1241853080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 1241853080}, {'type': 'v1', 'addr': '172.21.15.154:6833', 'nonce': 1241853080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/448006801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 448006801}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 448006801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/1503733829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 1503733829}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 1503733829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2083325907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2083325907}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2083325907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.164:6837/316018252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 316018252}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 316018252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.154:6837/2695634895', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 2695634895}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 2695634895}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.164:6839/1157843957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1157843957}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1157843957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.59:6839/377916146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 377916146}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 377916146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6839/3218637046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 3218637046}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 3218637046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24560, 24584, 24547, 24530, 24529]} 2024-04-02T17:12:45.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T17:12:45.142 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 -- 172.21.15.59:0/1893784971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 msgr2=0x7f2ef0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/1893784971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0101e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009900 tx=0x7f2ee402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 -- 172.21.15.59:0/1893784971 shutdown_connections 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/1893784971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0103590 0x7f2ef0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/1893784971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0102bd0 0x7f2ef0103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/1893784971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0101e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.349+0000 7f2ef6c00640 1 -- 172.21.15.59:0/1893784971 >> 172.21.15.59:0/1893784971 conn(0x7f2ef00fcf70 msgr2=0x7f2ef00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.350+0000 7f2ef6c00640 1 -- 172.21.15.59:0/1893784971 shutdown_connections 2024-04-02T17:12:45.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.350+0000 7f2ef6c00640 1 -- 172.21.15.59:0/1893784971 wait complete. 2024-04-02T17:12:45.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 Processor -- start 2024-04-02T17:12:45.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 -- start start 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 0x7f2ef019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010d630 con 0x7f2ef0102bd0 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010d4b0 con 0x7f2ef01019e0 2024-04-02T17:12:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.351+0000 7f2ef6c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010d7b0 con 0x7f2ef0103590 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40140/0 (socket says 172.21.15.59:40140) 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 -- 172.21.15.59:0/3440466700 learned_addr learned my addr 172.21.15.59:0/3440466700 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38696/0 (socket says 172.21.15.59:38696) 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 0x7f2ef019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 -- 172.21.15.59:0/3440466700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 msgr2=0x7f2ef0199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 -- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 msgr2=0x7f2ef019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 0x7f2ef019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef6200640 1 -- 172.21.15.59:0/3440466700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01a1b60 con 0x7f2ef0103590 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.352+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 0x7f2ef019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:45.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2ef6200640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eb10 tx=0x7f2ee800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2eee000640 1 -- 172.21.15.59:0/3440466700 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004b20 con 0x7f2ef0103590 2024-04-02T17:12:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2eee000640 1 -- 172.21.15.59:0/3440466700 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee8004cc0 con 0x7f2ef0103590 2024-04-02T17:12:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01a1e50 con 0x7f2ef0103590 2024-04-02T17:12:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2eee000640 1 -- 172.21.15.59:0/3440466700 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8010760 con 0x7f2ef0103590 2024-04-02T17:12:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.353+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01a22c0 con 0x7f2ef0103590 2024-04-02T17:12:45.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.356+0000 7f2efadef740 1 Processor -- start 2024-04-02T17:12:45.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.356+0000 7f2eee000640 1 -- 172.21.15.59:0/3440466700 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ee8005020 con 0x7f2ef0103590 2024-04-02T17:12:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 -- start start 2024-04-02T17:12:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922837f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x55992283d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55992279d060 con 0x559922838500 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559922745560 con 0x55992283d670 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2efadef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559922799010 con 0x559922837b10 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x55992283d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38708/0 (socket says 172.21.15.59:38708) 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2ef4e00640 1 -- 172.21.15.59:0/2649236399 learned_addr learned my addr 172.21.15.59:0/2649236399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.357+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922837f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 -- 172.21.15.59:0/2649236399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 msgr2=0x559922837f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922837f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 -- 172.21.15.59:0/2649236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 msgr2=0x55992283d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x55992283d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 -- 172.21.15.59:0/2649236399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559922767fd0 con 0x55992283d670 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2eee000640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 0x7f2ed80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2eee000640 1 -- 172.21.15.59:0/3440466700 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2ee80d0ae0 con 0x7f2ef0103590 2024-04-02T17:12:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef5800640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 0x7f2ed80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009560 tx=0x7f2edc009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c461a3d4bd98ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ee2a00640 1 -- 172.21.15.59:0/2649236399 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc012e40 con 0x55992283d670 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.358+0000 7f2ee2a00640 1 -- 172.21.15.59:0/2649236399 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2edc010cb0 con 0x55992283d670 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2ee2a00640 1 -- 172.21.15.59:0/2649236399 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01ebd0 con 0x55992283d670 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 -- 172.21.15.59:0/2649236399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 msgr2=0x55992283fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2ef5800640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 0x7f2ed80a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009560 tx=0x7f2edc009760 comp rx=0 tx=0).stop 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 -- 172.21.15.59:0/2649236399 shutdown_connections 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x55992283d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x55992283fb00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 --2- 172.21.15.59:0/2649236399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922837f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 -- 172.21.15.59:0/2649236399 >> 172.21.15.59:0/2649236399 conn(0x559922833080 msgr2=0x559922833480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 -- 172.21.15.59:0/2649236399 shutdown_connections 2024-04-02T17:12:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.359+0000 7f2efadef740 1 -- 172.21.15.59:0/2649236399 wait complete. 2024-04-02T17:12:45.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.360+0000 7f2efadef740 1 -- start start 2024-04-02T17:12:45.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.360+0000 7f2efadef740 1 Processor -- start 2024-04-02T17:12:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.360+0000 7f2efadef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.360+0000 7f2efadef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 0x559922842530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2efadef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922849570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2efadef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559922840620 con 0x559922837b10 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2efadef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559922758920 con 0x55992283d670 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2efadef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5599228407a0 con 0x559922838500 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2efadef740 10 client.0 _ll_register_callbacks cb 0x559922758310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 0x559922842530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38430/0 (socket says 172.21.15.59:38430) 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef5800640 1 -- 172.21.15.59:0/555618655 learned_addr learned my addr 172.21.15.59:0/555618655 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef6200640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922849570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 -- 172.21.15.59:0/555618655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 msgr2=0x559922849570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922849570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 -- 172.21.15.59:0/555618655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 msgr2=0x559922841ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef4e00640 1 -- 172.21.15.59:0/555618655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599224b12d0 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef6200640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922849570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.361+0000 7f2ef5800640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 0x559922842530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009560 tx=0x7f2edc009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5da854c0379d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2ed7400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01e990 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2edc010cb0 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2edc01eb30 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2edc01ed70 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2efadef740 10 client.25192 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.362+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55992257b6b0 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2edc010cb0 con 0x559922838500 2024-04-02T17:12:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2edc026030 con 0x559922838500 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 10 client.25192 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 10 client.25192 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 20 client.25192 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55992284eb30 0x55992284ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 4 client.25192 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599227aa5a0 con 0x55992284eb30 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.363+0000 7f2ef5800640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55992284eb30 0x55992284ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.364+0000 7f2ef5800640 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55992284eb30 0x55992284ffa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70f1199c85ce38d5 server_cookie=a640b433aaed291c in_seq=0 out_seq=0 2024-04-02T17:12:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.364+0000 7f2ed7400640 10 client.25192 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2ed7400640 1 -- 172.21.15.59:0/555618655 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599227aa5a0 con 0x55992284eb30 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2ed7400640 10 client.25192 handle_command_reply: tid=0 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 client.25192 shutdown 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 msgr2=0x559922842530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 0x559922842530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009560 tx=0x7f2edc009e40 comp rx=0 tx=0).stop 2024-04-02T17:12:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 shutdown_connections 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55992284eb30 0x55992284ffa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922849570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922838500 0x559922842530 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 --2- 172.21.15.59:0/555618655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922837b10 0x559922841ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2ed7400640 0 client.25192 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 >> 172.21.15.59:0/555618655 conn(0x559922833080 msgr2=0x5599228321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.365+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 shutdown_connections 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.366+0000 7f2efadef740 1 -- 172.21.15.59:0/555618655 wait complete. 2024-04-02T17:12:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.366+0000 7f2efadef740 20 client.25192 trim_cache size 0 max 16384 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 Processor -- start 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 -- start start 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559922838500 0x559922849530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55992284e2e0 con 0x55992283d670 2024-04-02T17:12:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55992284e9b0 con 0x559922838500 2024-04-02T17:12:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.386+0000 7f2efadef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55992284f080 con 0x559922837b10 2024-04-02T17:12:45.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559922838500 0x559922849530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40166/0 (socket says 172.21.15.59:40166) 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 -- 172.21.15.59:0/2685152361 learned_addr learned my addr 172.21.15.59:0/2685152361 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef6200640 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38432/0 (socket says 172.21.15.59:38432) 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 -- 172.21.15.59:0/2685152361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559922838500 msgr2=0x559922849530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559922838500 0x559922849530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 -- 172.21.15.59:0/2685152361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 msgr2=0x559922839a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 -- 172.21.15.59:0/2685152361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559922833df0 con 0x559922837b10 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef6200640 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.387+0000 7f2ef5800640 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4036040 tx=0x7f2ee4043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a23428039538fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2ee2000640 1 -- 172.21.15.59:0/2685152361 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4051b50 con 0x559922837b10 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2ee2000640 1 -- 172.21.15.59:0/2685152361 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee4051890 con 0x559922837b10 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2ee2000640 1 -- 172.21.15.59:0/2685152361 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40035f0 con 0x559922837b10 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 -- 172.21.15.59:0/2685152361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 msgr2=0x559922833460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4036040 tx=0x7f2ee4043000 comp rx=0 tx=0).stop 2024-04-02T17:12:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 -- 172.21.15.59:0/2685152361 shutdown_connections 2024-04-02T17:12:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55992283d670 0x559922839a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559922838500 0x559922849530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 --2- 172.21.15.59:0/2685152361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x559922833460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.388+0000 7f2efadef740 1 -- 172.21.15.59:0/2685152361 >> 172.21.15.59:0/2685152361 conn(0x559922833080 msgr2=0x55992282dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 -- 172.21.15.59:0/2685152361 shutdown_connections 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 -- 172.21.15.59:0/2685152361 wait complete. 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 -- start start 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 Processor -- start 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.389+0000 7f2efadef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2efadef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922847d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2efadef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55992284fe20 con 0x559922838500 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2efadef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55992284f750 con 0x55992283d670 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2efadef740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5599228504f0 con 0x559922837b10 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2efadef740 10 client.0 _ll_register_callbacks cb 0x7f2edc012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38438/0 (socket says 172.21.15.59:38438) 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40176/0 (socket says 172.21.15.59:40176) 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 -- 172.21.15.59:0/3311590168 learned_addr learned my addr 172.21.15.59:0/3311590168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef6200640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922847d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 -- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 msgr2=0x5599228447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 -- 172.21.15.59:0/3311590168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 msgr2=0x559922847d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922847d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef4e00640 1 -- 172.21.15.59:0/3311590168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559922848330 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef5800640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.390+0000 7f2ef6200640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922847d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ef4e00640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0092e0 tx=0x7f2edc009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b588b7d9721d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ed7400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004890 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2edc012040 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee4051890 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2edc012240 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2efadef740 10 client.15078 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559922849630 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.391+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee4051890 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2edc026040 con 0x559922838500 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 10 client.15078 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 20 client.15078 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5599228b9210 0x5599228b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 4 client.15078 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "status"}) -- 0x5599227a48f0 con 0x5599228b9210 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.392+0000 7f2ef5800640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5599228b9210 0x5599228b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2ef5800640 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5599228b9210 0x5599228b5d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23bcb5cd65951c36 server_cookie=8739a5341635b348 in_seq=0 out_seq=0 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2ed7400640 10 client.15078 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2ed7400640 1 -- 172.21.15.59:0/3311590168 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5599227a48f0 con 0x5599228b9210 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2ed7400640 10 client.15078 handle_command_reply: tid=0 2024-04-02T17:12:45.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2efadef740 1 client.15078 shutdown 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 msgr2=0x559922844d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0092e0 tx=0x7f2edc009be0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 shutdown_connections 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.393+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x5599228b9210 0x5599228b5d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559922838500 0x559922844d20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55992283d670 0x559922847d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 --2- 172.21.15.59:0/3311590168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559922837b10 0x5599228447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2ed7400640 0 client.15078 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 >> 172.21.15.59:0/3311590168 conn(0x559922833080 msgr2=0x559922833460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 shutdown_connections 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 1 -- 172.21.15.59:0/3311590168 wait complete. 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2efadef740 20 client.15078 trim_cache size 0 max 16384 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 4, 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24529, 2024-04-02T17:12:45.398 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 123.31879409299999, 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 151.71137343199999 2024-04-02T17:12:45.399 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 msgr2=0x7f2ed80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 0x7f2ed80a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee4005fd0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 msgr2=0x7f2ef01a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.394+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eb10 tx=0x7f2ee800c2d0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 shutdown_connections 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ed80a3bb0 0x7f2ed80a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef0102bd0 0x7f2ef019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ef0103590 0x7f2ef01a14f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 --2- 172.21.15.59:0/3440466700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef01019e0 0x7f2ef0199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 >> 172.21.15.59:0/3440466700 conn(0x7f2ef00fcf70 msgr2=0x7f2ef0073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 shutdown_connections 2024-04-02T17:12:45.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.395+0000 7f2ef6c00640 1 -- 172.21.15.59:0/3440466700 wait complete. 2024-04-02T17:12:45.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:45 smithi059 ceph-mon[31664]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:45.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:45 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3320578679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:45 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1219467380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T17:12:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:45 smithi164 ceph-mon[36499]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:45 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3320578679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:45 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1219467380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 -- 172.21.15.59:0/1919384325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 msgr2=0x7eff40108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 --2- 172.21.15.59:0/1919384325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff40108440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7eff30009860 tx=0x7eff3002bb70 comp rx=0 tx=0).stop 2024-04-02T17:12:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 -- 172.21.15.59:0/1919384325 shutdown_connections 2024-04-02T17:12:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 --2- 172.21.15.59:0/1919384325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 0x7eff40109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 --2- 172.21.15.59:0/1919384325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff40110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 --2- 172.21.15.59:0/1919384325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff40108440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.616+0000 7eff3f200640 1 -- 172.21.15.59:0/1919384325 >> 172.21.15.59:0/1919384325 conn(0x7eff40076fd0 msgr2=0x7eff400793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.617+0000 7eff3f200640 1 -- 172.21.15.59:0/1919384325 shutdown_connections 2024-04-02T17:12:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.617+0000 7eff3f200640 1 -- 172.21.15.59:0/1919384325 wait complete. 2024-04-02T17:12:45.679 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:45 smithi154 ceph-mon[36479]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:45.679 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:45 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3320578679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.679 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:45 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1219467380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 Processor -- start 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 -- start start 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff4019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 0x7eff4019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113c70 con 0x7eff40109210 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113af0 con 0x7eff40108020 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.618+0000 7eff3f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113df0 con 0x7eff40109bd0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 0x7eff4019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40190/0 (socket says 172.21.15.59:40190) 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 -- 172.21.15.59:0/2090879291 learned_addr learned my addr 172.21.15.59:0/2090879291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3de00640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff4019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 -- 172.21.15.59:0/2090879291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 msgr2=0x7eff4019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff4019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 -- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 msgr2=0x7eff4019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 0x7eff4019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 -- 172.21.15.59:0/2090879291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff401a5f40 con 0x7eff40109bd0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.619+0000 7eff3e800640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff38009fd0 tx=0x7eff3800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.620+0000 7eff37400640 1 -- 172.21.15.59:0/2090879291 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38004b20 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.620+0000 7eff37400640 1 -- 172.21.15.59:0/2090879291 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff38003e50 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.620+0000 7eff37400640 1 -- 172.21.15.59:0/2090879291 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3800b630 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.620+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff401a6230 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.620+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff401a6730 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.623+0000 7eff47676740 1 Processor -- start 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.623+0000 7eff37400640 1 -- 172.21.15.59:0/2090879291 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7eff3801d020 con 0x7eff40109bd0 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.623+0000 7eff47676740 1 -- start start 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.623+0000 7eff47676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65eef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff47676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff47676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf65f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff47676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf6554060 con 0x557bf65ef500 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff47676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bf64fc560 con 0x557bf65f4670 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff47676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557bf6550010 con 0x557bf65eeb10 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38460/0 (socket says 172.21.15.59:38460) 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff35400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 -- 172.21.15.59:0/3936944377 learned_addr learned my addr 172.21.15.59:0/3936944377 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff35400640 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf65f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 -- 172.21.15.59:0/3936944377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 msgr2=0x557bf65eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 -- 172.21.15.59:0/3936944377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 msgr2=0x557bf65f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf65f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff3de00640 1 -- 172.21.15.59:0/3936944377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf651efd0 con 0x557bf65ef500 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.624+0000 7eff35400640 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff37400640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 0x7eff240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff3de00640 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff300057c0 tx=0x7eff30009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36af1d1f6ab1185 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff37400640 1 -- 172.21.15.59:0/2090879291 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7eff380d03f0 con 0x7eff40109bd0 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff35400640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 0x7eff240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff2f000640 1 -- 172.21.15.59:0/3936944377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30043020 con 0x557bf65ef500 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff2f000640 1 -- 172.21.15.59:0/3936944377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff300323c0 con 0x557bf65ef500 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.625+0000 7eff2f000640 1 -- 172.21.15.59:0/3936944377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30039040 con 0x557bf65ef500 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 -- 172.21.15.59:0/3936944377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 msgr2=0x557bf65f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff35400640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 0x7eff240a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x557bf65efd60 tx=0x7eff28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff300057c0 tx=0x7eff30009490 comp rx=0 tx=0).stop 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 -- 172.21.15.59:0/3936944377 shutdown_connections 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf65f4000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf65f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 --2- 172.21.15.59:0/3936944377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 -- 172.21.15.59:0/3936944377 >> 172.21.15.59:0/3936944377 conn(0x557bf65ea080 msgr2=0x557bf65ea480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 -- 172.21.15.59:0/3936944377 shutdown_connections 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.626+0000 7eff47676740 1 -- 172.21.15.59:0/3936944377 wait complete. 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 -- start start 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 Processor -- start 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65ef500 0x557bf65f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65f4670 0x557bf6600580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf65f75c0 con 0x557bf65eeb10 2024-04-02T17:12:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bf650f920 con 0x557bf65ef500 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557bf65f7740 con 0x557bf65f4670 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.627+0000 7eff47676740 10 client.0 _ll_register_callbacks cb 0x557bf650f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38472/0 (socket says 172.21.15.59:38472) 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 -- 172.21.15.59:0/386962369 learned_addr learned my addr 172.21.15.59:0/386962369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff35400640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65ef500 0x557bf65f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3e800640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65f4670 0x557bf6600580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 -- 172.21.15.59:0/386962369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65f4670 msgr2=0x557bf6600580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65f4670 0x557bf6600580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 -- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65ef500 msgr2=0x557bf65f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65ef500 0x557bf65f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 -- 172.21.15.59:0/386962369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf62682d0 con 0x557bf65eeb10 2024-04-02T17:12:45.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff3de00640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c0b0 tx=0x7eff300043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4495064a1587743 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff23400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.628+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30043030 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7eff30034640 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff300323c0 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff3005a050 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff47676740 10 client.15090 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf63326b0 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff300323c0 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff30032a20 con 0x557bf65eeb10 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.629+0000 7eff47676740 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff47676740 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff47676740 20 client.15090 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6605900 0x557bf6606d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff47676740 4 client.15090 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf65615a0 con 0x557bf6605900 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.630+0000 7eff35400640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6605900 0x557bf6606d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.631+0000 7eff35400640 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6605900 0x557bf6606d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21384111638f2749 server_cookie=5ebfaa1877731c5e in_seq=0 out_seq=0 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.631+0000 7eff23400640 10 client.15090 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff23400640 1 -- 172.21.15.59:0/386962369 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bf65615a0 con 0x557bf6605900 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff23400640 10 client.15090 handle_command_reply: tid=0 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 client.15090 shutdown 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 msgr2=0x557bf65f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c0b0 tx=0x7eff300043d0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 shutdown_connections 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6605900 0x557bf6606d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65f4670 0x557bf6600580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65ef500 0x557bf65f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff23400640 0 client.15090 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 --2- 172.21.15.59:0/386962369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65eeb10 0x557bf65f9000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 >> 172.21.15.59:0/386962369 conn(0x557bf65ea080 msgr2=0x557bf65e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 shutdown_connections 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 1 -- 172.21.15.59:0/386962369 wait complete. 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.632+0000 7eff47676740 20 client.15090 trim_cache size 0 max 16384 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 Processor -- start 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 -- start start 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65f4670 0x557bf65e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf66050b0 con 0x557bf65f4670 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bf6605780 con 0x557bf65eeb10 2024-04-02T17:12:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff47676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557bf6605e50 con 0x557bf65ef500 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff3e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65f4670 0x557bf65e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff3de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff3de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38788/0 (socket says 172.21.15.59:38788) 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff3de00640 1 -- 172.21.15.59:0/1610597742 learned_addr learned my addr 172.21.15.59:0/1610597742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.653+0000 7eff35400640 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 -- 172.21.15.59:0/1610597742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65ef500 msgr2=0x557bf66004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 -- 172.21.15.59:0/1610597742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65f4670 msgr2=0x557bf65e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65f4670 0x557bf65e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 -- 172.21.15.59:0/1610597742 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf65eadf0 con 0x557bf65eeb10 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff3de00640 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c180 tx=0x7eff30005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ba356065248bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff2e600640 1 -- 172.21.15.59:0/1610597742 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3004c020 con 0x557bf65eeb10 2024-04-02T17:12:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff2e600640 1 -- 172.21.15.59:0/1610597742 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30005850 con 0x557bf65eeb10 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.654+0000 7eff2e600640 1 -- 172.21.15.59:0/1610597742 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30043050 con 0x557bf65eeb10 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- 172.21.15.59:0/1610597742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 msgr2=0x557bf65f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c180 tx=0x7eff30005bd0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- 172.21.15.59:0/1610597742 shutdown_connections 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65f4670 0x557bf65e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 --2- 172.21.15.59:0/1610597742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65eeb10 0x557bf65f6a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- 172.21.15.59:0/1610597742 >> 172.21.15.59:0/1610597742 conn(0x557bf65ea080 msgr2=0x557bf65f7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- 172.21.15.59:0/1610597742 shutdown_connections 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- 172.21.15.59:0/1610597742 wait complete. 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 -- start start 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.655+0000 7eff47676740 1 Processor -- start 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf66698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557bf6606bf0 con 0x557bf65ef500 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bf6606520 con 0x557bf65f4670 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557bf66072c0 con 0x557bf65eeb10 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff47676740 10 client.0 _ll_register_callbacks cb 0x7eff38009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38494/0 (socket says 172.21.15.59:38494) 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff35400640 1 -- 172.21.15.59:0/508676751 learned_addr learned my addr 172.21.15.59:0/508676751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff3e800640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf66698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.656+0000 7eff3de00640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 -- 172.21.15.59:0/508676751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 msgr2=0x557bf65f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 -- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 msgr2=0x557bf66698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf66698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 -- 172.21.15.59:0/508676751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf666a030 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff3e800640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf66698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff35400640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff28009fd0 tx=0x7eff280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c523e7c65c07640 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff23400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2802d750 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7eff28011790 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.657+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30005850 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff2802d8f0 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 10 client.15096 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf666b430 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30005850 con 0x557bf65ef500 2024-04-02T17:12:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff2803e020 con 0x557bf65ef500 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 20 client.15096 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6670000 0x557bf666ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 4 client.15096 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "status"}) -- 0x557bf655b8f0 con 0x557bf6670000 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.658+0000 7eff3de00640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6670000 0x557bf666ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff3de00640 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6670000 0x557bf666ed10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=916b72cf84d1bf30 server_cookie=274a3401541036e3 in_seq=0 out_seq=0 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff23400640 10 client.15096 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff23400640 1 -- 172.21.15.59:0/508676751 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557bf655b8f0 con 0x557bf6670000 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff23400640 10 client.15096 handle_command_reply: tid=0 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 client.15096 shutdown 2024-04-02T17:12:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 msgr2=0x557bf66004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff28009fd0 tx=0x7eff280046c0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 shutdown_connections 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x557bf6670000 0x557bf666ed10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557bf65ef500 0x557bf66004a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bf65f4670 0x557bf66698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 --2- 172.21.15.59:0/508676751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557bf65eeb10 0x557bf65f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff23400640 0 client.15096 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.659+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 >> 172.21.15.59:0/508676751 conn(0x557bf65ea080 msgr2=0x557bf65e4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 shutdown_connections 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff47676740 1 -- 172.21.15.59:0/508676751 wait complete. 2024-04-02T17:12:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff47676740 20 client.15096 trim_cache size 0 max 16384 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 3, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24530, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 124.602759804, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 149.242358325 2024-04-02T17:12:45.708 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 msgr2=0x7eff240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 0x7eff240a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x557bf65efd60 tx=0x7eff28005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 msgr2=0x7eff401a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.660+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff38009fd0 tx=0x7eff3800c800 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 shutdown_connections 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7eff240a3c30 0x7eff240a60f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff40109210 0x7eff4019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff40109bd0 0x7eff401a58d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 --2- 172.21.15.59:0/2090879291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff40108020 0x7eff4019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 >> 172.21.15.59:0/2090879291 conn(0x7eff40076fd0 msgr2=0x7eff4010c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 shutdown_connections 2024-04-02T17:12:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.661+0000 7eff3f200640 1 -- 172.21.15.59:0/2090879291 wait complete. 2024-04-02T17:12:45.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 -- 172.21.15.59:0/4256021421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 msgr2=0x7fab10108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 --2- 172.21.15.59:0/4256021421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab10108540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab00009900 tx=0x7fab0002bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 -- 172.21.15.59:0/4256021421 shutdown_connections 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 --2- 172.21.15.59:0/4256021421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab10109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 --2- 172.21.15.59:0/4256021421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 0x7fab10110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 --2- 172.21.15.59:0/4256021421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab10108540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 -- 172.21.15.59:0/4256021421 >> 172.21.15.59:0/4256021421 conn(0x7fab100770c0 msgr2=0x7fab10079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.939+0000 7fab16a00640 1 -- 172.21.15.59:0/4256021421 shutdown_connections 2024-04-02T17:12:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.940+0000 7fab16a00640 1 -- 172.21.15.59:0/4256021421 wait complete. 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 Processor -- start 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 -- start start 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab1019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 0x7fab101a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab10113d80 con 0x7fab10109310 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab10113c00 con 0x7fab10109cd0 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab16a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab10113f00 con 0x7fab10108120 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab1019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab14a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 0x7fab101a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40262/0 (socket says 172.21.15.59:40262) 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.941+0000 7fab0fe00640 1 -- 172.21.15.59:0/35227955 learned_addr learned my addr 172.21.15.59:0/35227955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 -- 172.21.15.59:0/35227955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 msgr2=0x7fab101a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 0x7fab101a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 -- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 msgr2=0x7fab1019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab1019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 -- 172.21.15.59:0/35227955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab101a60e0 con 0x7fab10108120 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab07e00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab1019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0fe00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab00000c00 tx=0x7fab0002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0d600640 1 -- 172.21.15.59:0/35227955 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00042020 con 0x7fab10108120 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab0d600640 1 -- 172.21.15.59:0/35227955 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00004450 con 0x7fab10108120 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.942+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab101a6370 con 0x7fab10108120 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.943+0000 7fab0d600640 1 -- 172.21.15.59:0/35227955 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0003e460 con 0x7fab10108120 2024-04-02T17:12:45.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.943+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a6910 con 0x7fab10108120 2024-04-02T17:12:45.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 Processor -- start 2024-04-02T17:12:45.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 -- start start 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab0d600640 1 -- 172.21.15.59:0/35227955 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab00045020 con 0x7fab10108120 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177352e0 0x559217739de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x55921773c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55921769a060 con 0x5592177348f0 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559217642560 con 0x5592177352e0 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.947+0000 7fab19781740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559217696010 con 0x55921773a450 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38516/0 (socket says 172.21.15.59:38516) 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 -- 172.21.15.59:0/3613880160 learned_addr learned my addr 172.21.15.59:0/3613880160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab07e00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x55921773c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab0fe00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177352e0 0x559217739de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 -- 172.21.15.59:0/3613880160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 msgr2=0x55921773c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x55921773c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 -- 172.21.15.59:0/3613880160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177352e0 msgr2=0x559217739de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177352e0 0x559217739de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab14a00640 1 -- 172.21.15.59:0/3613880160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559217664fd0 con 0x5592177348f0 2024-04-02T17:12:45.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.948+0000 7fab07e00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x55921773c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab14a00640 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab08009fd0 tx=0x7fab080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f44be8fb77dd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab05200640 1 -- 172.21.15.59:0/3613880160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08004d20 con 0x5592177348f0 2024-04-02T17:12:45.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab05200640 1 -- 172.21.15.59:0/3613880160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab08004050 con 0x5592177348f0 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab05200640 1 -- 172.21.15.59:0/3613880160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08005240 con 0x5592177348f0 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab0d600640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.949+0000 7fab0d600640 1 -- 172.21.15.59:0/35227955 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fab000f2350 con 0x7fab10108120 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab0fe00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- 172.21.15.59:0/3613880160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 msgr2=0x559217734d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab08009fd0 tx=0x7fab080118d0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- 172.21.15.59:0/3613880160 shutdown_connections 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x55921773c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177352e0 0x559217739de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 --2- 172.21.15.59:0/3613880160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177348f0 0x559217734d10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- 172.21.15.59:0/3613880160 >> 172.21.15.59:0/3613880160 conn(0x55921772fe60 msgr2=0x559217730260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- 172.21.15.59:0/3613880160 shutdown_connections 2024-04-02T17:12:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- 172.21.15.59:0/3613880160 wait complete. 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab0fe00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fab000f0160 tx=0x7fab000f5e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.950+0000 7fab19781740 1 -- start start 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 Processor -- start 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x55921773f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55921773d400 con 0x55921773a450 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559217655920 con 0x5592177348f0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55921773d580 con 0x5592177352e0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.951+0000 7fab19781740 10 client.0 _ll_register_callbacks cb 0x559217655310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab14a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab14a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab14a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38820/0 (socket says 172.21.15.59:38820) 2024-04-02T17:12:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab14a00640 1 -- 172.21.15.59:0/723612654 learned_addr learned my addr 172.21.15.59:0/723612654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x55921773f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 -- 172.21.15.59:0/723612654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 msgr2=0x55921773ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 -- 172.21.15.59:0/723612654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 msgr2=0x5592177463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 -- 172.21.15.59:0/723612654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592173ae2d0 con 0x5592177352e0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab14a00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7fab07e00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x55921773f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab1019f9d0 tx=0x7faaf800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13df26a96f4a3f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.952+0000 7faafd600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80120d0 con 0x5592177352e0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7faaf8004030 con 0x5592177352e0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab08004050 con 0x5592177352e0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faaf8012270 con 0x5592177352e0 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7fab19781740 10 client.25228 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592174786b0 con 0x5592177352e0 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.953+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab08004050 con 0x5592177352e0 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faaf801c460 con 0x5592177352e0 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 10 client.25228 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 10 client.25228 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 20 client.25228 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55921774bd50 0x55921774d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 4 client.25228 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592176a75a0 con 0x55921774bd50 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.954+0000 7fab14a00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55921774bd50 0x55921774d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.955+0000 7fab14a00640 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55921774bd50 0x55921774d1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b3604ec02239460 server_cookie=b5ab1b46126d7bc5 in_seq=0 out_seq=0 2024-04-02T17:12:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.955+0000 7faafd600640 10 client.25228 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.955+0000 7faafd600640 1 -- 172.21.15.59:0/723612654 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592176a75a0 con 0x55921774bd50 2024-04-02T17:12:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.955+0000 7faafd600640 10 client.25228 handle_command_reply: tid=0 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.955+0000 7fab19781740 1 client.25228 shutdown 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 msgr2=0x55921773f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x55921773f370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab1019f9d0 tx=0x7faaf800f790 comp rx=0 tx=0).stop 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 shutdown_connections 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55921774bd50 0x55921774d1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x55921773f370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 --2- 172.21.15.59:0/723612654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 >> 172.21.15.59:0/723612654 conn(0x55921772fe60 msgr2=0x55921772f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7faafd600640 0 client.25228 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 shutdown_connections 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 1 -- 172.21.15.59:0/723612654 wait complete. 2024-04-02T17:12:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.956+0000 7fab19781740 20 client.25228 trim_cache size 0 max 16384 2024-04-02T17:12:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 Processor -- start 2024-04-02T17:12:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 -- start start 2024-04-02T17:12:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55921774b500 con 0x55921773a450 2024-04-02T17:12:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55921774bbd0 con 0x5592177348f0 2024-04-02T17:12:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab19781740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55921774c2a0 con 0x5592177352e0 2024-04-02T17:12:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.976+0000 7fab07e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40280/0 (socket says 172.21.15.59:40280) 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 -- 172.21.15.59:0/2225740462 learned_addr learned my addr 172.21.15.59:0/2225740462 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab14a00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab14a00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 -- 172.21.15.59:0/2225740462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 msgr2=0x55921773c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 -- 172.21.15.59:0/2225740462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 msgr2=0x5592177367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 -- 172.21.15.59:0/2225740462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592177356c0 con 0x5592177352e0 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab14a00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7fab07e00640 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faaf801c950 tx=0x7faaf801df20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53d36dd36cb92c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.977+0000 7faaffe00640 1 -- 172.21.15.59:0/2225740462 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf801c460 con 0x5592177352e0 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7faaffe00640 1 -- 172.21.15.59:0/2225740462 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf801c600 con 0x5592177352e0 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7faaffe00640 1 -- 172.21.15.59:0/2225740462 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003850 con 0x5592177352e0 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 -- 172.21.15.59:0/2225740462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 msgr2=0x559217746370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faaf801c950 tx=0x7faaf801df20 comp rx=0 tx=0).stop 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 -- 172.21.15.59:0/2225740462 shutdown_connections 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55921773a450 0x5592177367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592177352e0 0x559217746370 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 --2- 172.21.15.59:0/2225740462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x55921773c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 -- 172.21.15.59:0/2225740462 >> 172.21.15.59:0/2225740462 conn(0x55921772fe60 msgr2=0x55921774ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 -- 172.21.15.59:0/2225740462 shutdown_connections 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.978+0000 7fab19781740 1 -- 172.21.15.59:0/2225740462 wait complete. 2024-04-02T17:12:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 -- start start 2024-04-02T17:12:45.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 Processor -- start 2024-04-02T17:12:45.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x559217741c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x5592177451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55921774d040 con 0x5592177352e0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55921774c970 con 0x5592177348f0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55921774d710 con 0x55921773a450 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.979+0000 7fab19781740 10 client.0 _ll_register_callbacks cb 0x7faaf8012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38536/0 (socket says 172.21.15.59:38536) 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 -- 172.21.15.59:0/3484925417 learned_addr learned my addr 172.21.15.59:0/3484925417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab14a00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x5592177451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab14a00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x559217741c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 -- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 msgr2=0x5592177451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x5592177451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 -- 172.21.15.59:0/3484925417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 msgr2=0x559217741c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x559217741c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 -- 172.21.15.59:0/3484925417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559217745790 con 0x5592177352e0 2024-04-02T17:12:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab14a00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x559217741c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7fab07e00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faaf800fd90 tx=0x7faaf8003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697b6fb1a7d7d240 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.980+0000 7faafd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8032070 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7faaf80050b0 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf801c600 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faaf803e050 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7fab19781740 10 client.15114 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592177b1ee0 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf801c600 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.981+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faaf802e750 con 0x5592177352e0 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T17:12:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 20 client.15114 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5592177b6760 0x5592177b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 4 client.15114 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "status"}) -- 0x5592176a18f0 con 0x5592177b6760 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab14a00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5592177b6760 0x5592177b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7fab14a00640 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5592177b6760 0x5592177b5470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29ea91d4266e5193 server_cookie=7f8ade21edd166e6 in_seq=0 out_seq=0 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.982+0000 7faafd600640 10 client.15114 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7faafd600640 1 -- 172.21.15.59:0/3484925417 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5592176a18f0 con 0x5592177b6760 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7faafd600640 10 client.15114 handle_command_reply: tid=0 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 client.15114 shutdown 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 msgr2=0x5592177421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faaf800fd90 tx=0x7faaf8003c50 comp rx=0 tx=0).stop 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 shutdown_connections 2024-04-02T17:12:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5592177b6760 0x5592177b5470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592177352e0 0x5592177421d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55921773a450 0x5592177451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7faafd600640 0 client.15114 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 --2- 172.21.15.59:0/3484925417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592177348f0 0x559217741c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 >> 172.21.15.59:0/3484925417 conn(0x55921772fe60 msgr2=0x5592177367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 shutdown_connections 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 1 -- 172.21.15.59:0/3484925417 wait complete. 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.983+0000 7fab19781740 20 client.15114 trim_cache size 0 max 16384 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:45.988 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24547, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 125.936956209, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 146.812985696 2024-04-02T17:12:45.989 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 msgr2=0x7faaf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fab000f0160 tx=0x7fab000f5e30 comp rx=0 tx=0).stop 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 msgr2=0x7fab1019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab00000c00 tx=0x7fab0002c660 comp rx=0 tx=0).stop 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 shutdown_connections 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab10109310 0x7fab1019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab10109cd0 0x7fab101a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 --2- 172.21.15.59:0/35227955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab10108120 0x7fab1019e4c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:45.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 >> 172.21.15.59:0/35227955 conn(0x7fab100770c0 msgr2=0x7fab1010c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:45.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 shutdown_connections 2024-04-02T17:12:45.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:45.984+0000 7fab16a00640 1 -- 172.21.15.59:0/35227955 wait complete. 2024-04-02T17:12:46.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-02T17:12:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 -- 172.21.15.59:0/3839604470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8108020 msgr2=0x7f76e8108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 --2- 172.21.15.59:0/3839604470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8108020 0x7f76e8108440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009900 tx=0x7f76d802bc40 comp rx=0 tx=0).stop 2024-04-02T17:12:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 -- 172.21.15.59:0/3839604470 shutdown_connections 2024-04-02T17:12:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 --2- 172.21.15.59:0/3839604470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8109210 0x7f76e8109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 --2- 172.21.15.59:0/3839604470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 0x7f76e8110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 --2- 172.21.15.59:0/3839604470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8108020 0x7f76e8108440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.221+0000 7f76ece00640 1 -- 172.21.15.59:0/3839604470 >> 172.21.15.59:0/3839604470 conn(0x7f76e8076fd0 msgr2=0x7f76e80793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.222+0000 7f76ece00640 1 -- 172.21.15.59:0/3839604470 shutdown_connections 2024-04-02T17:12:46.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.222+0000 7f76ece00640 1 -- 172.21.15.59:0/3839604470 wait complete. 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 Processor -- start 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 -- start start 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8109210 0x7f76e819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 0x7f76e81a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113d00 con 0x7f76e8108020 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113b80 con 0x7f76e8109210 2024-04-02T17:12:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.223+0000 7f76ece00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8113e80 con 0x7f76e8109bd0 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38552/0 (socket says 172.21.15.59:38552) 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 -- 172.21.15.59:0/2886607257 learned_addr learned my addr 172.21.15.59:0/2886607257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 0x7f76e81a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e6a00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8109210 0x7f76e819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 -- 172.21.15.59:0/2886607257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 msgr2=0x7f76e81a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 0x7f76e81a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 -- 172.21.15.59:0/2886607257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8109210 msgr2=0x7f76e819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8109210 0x7f76e819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 -- 172.21.15.59:0/2886607257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a5f20 con 0x7f76e8108020 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.224+0000 7f76e7400640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76d802efd0 tx=0x7f76d802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.225+0000 7f76dfe00640 1 -- 172.21.15.59:0/2886607257 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8041070 con 0x7f76e8108020 2024-04-02T17:12:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.225+0000 7f76dfe00640 1 -- 172.21.15.59:0/2886607257 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d8004450 con 0x7f76e8108020 2024-04-02T17:12:46.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.225+0000 7f76dfe00640 1 -- 172.21.15.59:0/2886607257 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003e50 con 0x7f76e8108020 2024-04-02T17:12:46.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.225+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a61b0 con 0x7f76e8108020 2024-04-02T17:12:46.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.225+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a6670 con 0x7f76e8108020 2024-04-02T17:12:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.228+0000 7f76f11a8740 1 Processor -- start 2024-04-02T17:12:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.228+0000 7f76dfe00640 1 -- 172.21.15.59:0/2886607257 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76d8044070 con 0x7f76e8108020 2024-04-02T17:12:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.228+0000 7f76f11a8740 1 -- start start 2024-04-02T17:12:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcedb500 0x55bbdcee0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 0x55bbdcee2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbdce40060 con 0x55bbdcedab10 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcde8560 con 0x55bbdcee0670 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76f11a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbdce3c010 con 0x55bbdcedb500 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcedb500 0x55bbdcee0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38562/0 (socket says 172.21.15.59:38562) 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76e7e00640 1 -- 172.21.15.59:0/1750301727 learned_addr learned my addr 172.21.15.59:0/1750301727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.229+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 0x55bbdcee2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 -- 172.21.15.59:0/1750301727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcedb500 msgr2=0x55bbdcee0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcedb500 0x55bbdcee0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 -- 172.21.15.59:0/1750301727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 msgr2=0x55bbdcee2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 0x55bbdcee2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 -- 172.21.15.59:0/1750301727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdce0afd0 con 0x55bbdcedab10 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 0x55bbdcee2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009fd0 tx=0x7f76e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f785a92af4c8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76dfe00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 0x7f76cc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76dcc00640 1 -- 172.21.15.59:0/1750301727 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0004d20 con 0x55bbdcedab10 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76dfe00640 1 -- 172.21.15.59:0/2886607257 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f76d80f1fb0 con 0x7f76e8108020 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76dcc00640 1 -- 172.21.15.59:0/1750301727 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e0004050 con 0x55bbdcedab10 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76dcc00640 1 -- 172.21.15.59:0/1750301727 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0005260 con 0x55bbdcedab10 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.230+0000 7f76e7400640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 0x7f76cc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 -- 172.21.15.59:0/1750301727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 msgr2=0x55bbdcedaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009fd0 tx=0x7f76e00118d0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76e7400640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 0x7f76cc0a6210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76d80efdc0 tx=0x7f76d80f5a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 -- 172.21.15.59:0/1750301727 shutdown_connections 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcee0670 0x55bbdcee2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcedb500 0x55bbdcee0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1750301727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedab10 0x55bbdcedaf30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 -- 172.21.15.59:0/1750301727 >> 172.21.15.59:0/1750301727 conn(0x55bbdced6080 msgr2=0x55bbdced6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 -- 172.21.15.59:0/1750301727 shutdown_connections 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.231+0000 7f76f11a8740 1 -- 172.21.15.59:0/1750301727 wait complete. 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 -- start start 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 Processor -- start 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdceec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.232+0000 7f76f11a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcee3620 con 0x55bbdcedb500 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76f11a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcdfb920 con 0x55bbdcedab10 2024-04-02T17:12:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76f11a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcee37a0 con 0x55bbdcee0670 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76f11a8740 10 client.0 _ll_register_callbacks cb 0x55bbdcdfb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdceec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38568/0 (socket says 172.21.15.59:38568) 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 -- 172.21.15.59:0/1792560774 learned_addr learned my addr 172.21.15.59:0/1792560774 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 -- 172.21.15.59:0/1792560774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 msgr2=0x55bbdceec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdceec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 -- 172.21.15.59:0/1792560774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 msgr2=0x55bbdcee5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 -- 172.21.15.59:0/1792560774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdcb542d0 con 0x55bbdcedb500 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76d4012700 tx=0x7f76d4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3db3bf36d1058b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.233+0000 7f76d1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4004030 con 0x55bbdcedb500 2024-04-02T17:12:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f76d40041d0 con 0x55bbdcedb500 2024-04-02T17:12:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e0004050 con 0x55bbdcedb500 2024-04-02T17:12:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76d4004410 con 0x55bbdcedb500 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76f11a8740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbdcc1e6b0 con 0x55bbdcedb500 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.234+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e0004050 con 0x55bbdcedb500 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f76d4025050 con 0x55bbdcedb500 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 10 client.15126 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 20 client.15126 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcef1e60 0x55bbdcef32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 4 client.15126 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbdce4d5a0 con 0x55bbdcef1e60 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcef1e60 0x55bbdcef32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcef1e60 0x55bbdcef32d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=374005fabb91e787 server_cookie=a9673da8cc6992c in_seq=0 out_seq=0 2024-04-02T17:12:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.235+0000 7f76d1600640 10 client.15126 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.236+0000 7f76d1600640 1 -- 172.21.15.59:0/1792560774 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbdce4d5a0 con 0x55bbdcef1e60 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.236+0000 7f76d1600640 10 client.15126 handle_command_reply: tid=0 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.236+0000 7f76f11a8740 1 client.15126 shutdown 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 msgr2=0x55bbdcee55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76d4012700 tx=0x7f76d4012bc0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 shutdown_connections 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcef1e60 0x55bbdcef32d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdcee55a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdceec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1792560774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76d1600640 0 client.15126 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 >> 172.21.15.59:0/1792560774 conn(0x55bbdced6080 msgr2=0x55bbdced51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 shutdown_connections 2024-04-02T17:12:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 1 -- 172.21.15.59:0/1792560774 wait complete. 2024-04-02T17:12:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.237+0000 7f76f11a8740 20 client.15126 trim_cache size 0 max 16384 2024-04-02T17:12:46.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 Processor -- start 2024-04-02T17:12:46.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 -- start start 2024-04-02T17:12:46.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdced0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdced6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef1610 con 0x55bbdcedb500 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef1ce0 con 0x55bbdcedab10 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76f11a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef23b0 con 0x55bbdcee0670 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.257+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38576/0 (socket says 172.21.15.59:38576) 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 -- 172.21.15.59:0/1414797034 learned_addr learned my addr 172.21.15.59:0/1414797034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdced0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 -- 172.21.15.59:0/1414797034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 msgr2=0x55bbdced6460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdced6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 -- 172.21.15.59:0/1414797034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 msgr2=0x55bbdced0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdced0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 -- 172.21.15.59:0/1414797034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdced6df0 con 0x55bbdcedb500 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e7e00640 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdced0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76e6a00640 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98be20197164bc17 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.258+0000 7f76d3e00640 1 -- 172.21.15.59:0/1414797034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d402b030 con 0x55bbdcedb500 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76d3e00640 1 -- 172.21.15.59:0/1414797034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d40035f0 con 0x55bbdcedb500 2024-04-02T17:12:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76d3e00640 1 -- 172.21.15.59:0/1414797034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d400fbe0 con 0x55bbdcedb500 2024-04-02T17:12:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 -- 172.21.15.59:0/1414797034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 msgr2=0x55bbdceec5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d40096a0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 -- 172.21.15.59:0/1414797034 shutdown_connections 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcedb500 0x55bbdceec5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdcee0670 0x55bbdced6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 --2- 172.21.15.59:0/1414797034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdced0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.259+0000 7f76f11a8740 1 -- 172.21.15.59:0/1414797034 >> 172.21.15.59:0/1414797034 conn(0x55bbdced6080 msgr2=0x55bbdcee3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 -- 172.21.15.59:0/1414797034 shutdown_connections 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 -- 172.21.15.59:0/1414797034 wait complete. 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 -- start start 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 Processor -- start 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.260+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76f11a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdceebb00 0x55bbdceebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76f11a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef3150 con 0x55bbdcee0670 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76f11a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef2a80 con 0x55bbdcedab10 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76f11a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcef3820 con 0x55bbdceebb00 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76f11a8740 10 client.0 _ll_register_callbacks cb 0x7f76d4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38584/0 (socket says 172.21.15.59:38584) 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 -- 172.21.15.59:0/2587581942 learned_addr learned my addr 172.21.15.59:0/2587581942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdceebb00 0x55bbdceebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 -- 172.21.15.59:0/2587581942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdceebb00 msgr2=0x55bbdceebf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdceebb00 0x55bbdceebf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 -- 172.21.15.59:0/2587581942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 msgr2=0x55bbdcee8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 -- 172.21.15.59:0/2587581942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdceec4f0 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76e6a00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76d4012420 tx=0x7f76d40041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a018da62ff13d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76d1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.261+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d400fd30 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f76d4005040 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d40035f0 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76d4005240 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 10 client.15132 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbdcf58a90 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d40035f0 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f76d4025070 con 0x55bbdcee0670 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 10 client.15132 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.262+0000 7f76f11a8740 20 client.15132 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcf5d420 0x55bbdcf5c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76f11a8740 4 client.15132 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "status"}) -- 0x55bbdce478f0 con 0x55bbdcf5d420 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcf5d420 0x55bbdcf5c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76e7e00640 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcf5d420 0x55bbdcf5c130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2daf3d51e12ab9 server_cookie=48ec60ea89ed54af in_seq=0 out_seq=0 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.263+0000 7f76d1600640 10 client.15132 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76d1600640 1 -- 172.21.15.59:0/2587581942 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bbdce478f0 con 0x55bbdcf5d420 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76d1600640 10 client.15132 handle_command_reply: tid=0 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 client.15132 shutdown 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 msgr2=0x55bbdcee8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76d4012420 tx=0x7f76d40041e0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 shutdown_connections 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55bbdcf5d420 0x55bbdcf5c130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbdcee0670 0x55bbdcee8c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbdceebb00 0x55bbdceebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 --2- 172.21.15.59:0/2587581942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbdcedab10 0x55bbdcee8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76d1600640 0 client.15132 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 >> 172.21.15.59:0/2587581942 conn(0x55bbdced6080 msgr2=0x55bbdced0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 shutdown_connections 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 1 -- 172.21.15.59:0/2587581942 wait complete. 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.264+0000 7f76f11a8740 20 client.15132 trim_cache size 0 max 16384 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2024-04-02T17:12:46.269 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24560, 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 127.229955408, 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 143.14381442199999 2024-04-02T17:12:46.270 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 msgr2=0x7f76cc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 0x7f76cc0a6210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76d80efdc0 tx=0x7f76d80f5a90 comp rx=0 tx=0).stop 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 msgr2=0x7f76e819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f76d802efd0 tx=0x7f76d802c660 comp rx=0 tx=0).stop 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 shutdown_connections 2024-04-02T17:12:46.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8109bd0 0x7f76e81a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e8109210 0x7f76e819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f76cc0a3d50 0x7f76cc0a6210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 --2- 172.21.15.59:0/2886607257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e8108020 0x7f76e819e300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 >> 172.21.15.59:0/2886607257 conn(0x7f76e8076fd0 msgr2=0x7f76e810c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 shutdown_connections 2024-04-02T17:12:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.265+0000 7f76ece00640 1 -- 172.21.15.59:0/2886607257 wait complete. 2024-04-02T17:12:46.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 -- 172.21.15.59:0/2301468334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 msgr2=0x7f8bb410be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 --2- 172.21.15.59:0/2301468334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb410be70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009900 tx=0x7f8bac02bcd0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 -- 172.21.15.59:0/2301468334 shutdown_connections 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 --2- 172.21.15.59:0/2301468334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb410be70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 --2- 172.21.15.59:0/2301468334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40744e0 0x7f8bb4074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 --2- 172.21.15.59:0/2301468334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb4075ac0 0x7f8bb4073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 -- 172.21.15.59:0/2301468334 >> 172.21.15.59:0/2301468334 conn(0x7f8bb40fd0a0 msgr2=0x7f8bb40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.512+0000 7f8bbb400640 1 -- 172.21.15.59:0/2301468334 shutdown_connections 2024-04-02T17:12:46.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.513+0000 7f8bbb400640 1 -- 172.21.15.59:0/2301468334 wait complete. 2024-04-02T17:12:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 Processor -- start 2024-04-02T17:12:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 -- start start 2024-04-02T17:12:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 0x7f8bb419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 0x7f8bb419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410fa10 con 0x7f8bb41099e0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410f890 con 0x7f8bb4075ac0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.514+0000 7f8bbb400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb410fb90 con 0x7f8bb40744e0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38604/0 (socket says 172.21.15.59:38604) 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 -- 172.21.15.59:0/3746283281 learned_addr learned my addr 172.21.15.59:0/3746283281 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 0x7f8bb419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 0x7f8bb419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 -- 172.21.15.59:0/3746283281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 msgr2=0x7f8bb419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 0x7f8bb419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 -- 172.21.15.59:0/3746283281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 msgr2=0x7f8bb419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 0x7f8bb419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb9400640 1 -- 172.21.15.59:0/3746283281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb41a1c40 con 0x7f8bb41099e0 2024-04-02T17:12:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 0x7f8bb419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.515+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 0x7f8bb419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036040 tx=0x7f8bac02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bb1600640 1 -- 172.21.15.59:0/3746283281 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac043020 con 0x7f8bb41099e0 2024-04-02T17:12:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bb1600640 1 -- 172.21.15.59:0/3746283281 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bac0054e0 con 0x7f8bb41099e0 2024-04-02T17:12:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bb1600640 1 -- 172.21.15.59:0/3746283281 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac003b90 con 0x7f8bb41099e0 2024-04-02T17:12:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb41a1ed0 con 0x7f8bb41099e0 2024-04-02T17:12:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.516+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb41a23e0 con 0x7f8bb41099e0 2024-04-02T17:12:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.520+0000 7f8bbe22e740 1 Processor -- start 2024-04-02T17:12:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.520+0000 7f8bb1600640 1 -- 172.21.15.59:0/3746283281 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bac046020 con 0x7f8bb41099e0 2024-04-02T17:12:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.520+0000 7f8bbe22e740 1 -- start start 2024-04-02T17:12:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b78bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b7938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f4b6f1060 con 0x560f4b78c2e0 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f4b699560 con 0x560f4b78b8f0 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f4b6ed010 con 0x560f4b791450 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bb8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bb3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b78bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bb8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38610/0 (socket says 172.21.15.59:38610) 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bb8a00640 1 -- 172.21.15.59:0/2637220408 learned_addr learned my addr 172.21.15.59:0/2637220408 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.521+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b7938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 -- 172.21.15.59:0/2637220408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 msgr2=0x560f4b7938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b7938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 -- 172.21.15.59:0/2637220408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 msgr2=0x560f4b78bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b78bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 -- 172.21.15.59:0/2637220408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4b6bbfd0 con 0x560f4b78c2e0 2024-04-02T17:12:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b78bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40115c0 tx=0x7f8ba4011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d9fe3b801c44b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb1600640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 0x7f8b940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8ba9c00640 1 -- 172.21.15.59:0/2637220408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400f640 con 0x560f4b78c2e0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb1600640 1 -- 172.21.15.59:0/3746283281 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bac0f2420 con 0x7f8bb41099e0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8ba9c00640 1 -- 172.21.15.59:0/2637220408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba400fcc0 con 0x560f4b78c2e0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 0x7f8b940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.522+0000 7f8ba9c00640 1 -- 172.21.15.59:0/2637220408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4019cd0 con 0x560f4b78c2e0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 -- 172.21.15.59:0/2637220408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 msgr2=0x560f4b790de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 0x7f8b940a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x560f4b78cb40 tx=0x7f8b9c009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40115c0 tx=0x7f8ba4011a80 comp rx=0 tx=0).stop 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 -- 172.21.15.59:0/2637220408 shutdown_connections 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b790de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b7938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/2637220408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b78bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 -- 172.21.15.59:0/2637220408 >> 172.21.15.59:0/2637220408 conn(0x560f4b786e60 msgr2=0x560f4b787260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 -- 172.21.15.59:0/2637220408 shutdown_connections 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.523+0000 7f8bbe22e740 1 -- 172.21.15.59:0/2637220408 wait complete. 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 -- start start 2024-04-02T17:12:46.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 Processor -- start 2024-04-02T17:12:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b795e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78c2e0 0x560f4b796340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.524+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f4b794400 con 0x560f4b791450 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f4b6ac920 con 0x560f4b78b8f0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f4b794580 con 0x560f4b78c2e0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bbe22e740 10 client.0 _ll_register_callbacks cb 0x560f4b6ac310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38620/0 (socket says 172.21.15.59:38620) 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 -- 172.21.15.59:0/484569410 learned_addr learned my addr 172.21.15.59:0/484569410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b795e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78c2e0 0x560f4b796340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 -- 172.21.15.59:0/484569410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78c2e0 msgr2=0x560f4b796340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78c2e0 0x560f4b796340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 -- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 msgr2=0x560f4b795e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b795e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 -- 172.21.15.59:0/484569410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4b4052d0 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b795e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8bb9400640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0bb010 tx=0x7f8bac009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544c6557f5f2b081 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.525+0000 7f8ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0dc060 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bac0bacb0 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba400fcc0 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bac004c80 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8bbe22e740 10 client.15144 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f4b4cf6b0 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba400fcc0 con 0x560f4b791450 2024-04-02T17:12:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.526+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bac0c6e20 con 0x560f4b791450 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 20 client.15144 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b7a2c10 0x560f4b7a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 4 client.15144 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4b6fe5a0 con 0x560f4b7a2c10 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.527+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b7a2c10 0x560f4b7a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.528+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b7a2c10 0x560f4b7a4080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265668ecccac047c server_cookie=6a5bab16270d6bb3 in_seq=0 out_seq=0 2024-04-02T17:12:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.528+0000 7f8ba2000640 10 client.15144 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.528+0000 7f8ba2000640 1 -- 172.21.15.59:0/484569410 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f4b6fe5a0 con 0x560f4b7a2c10 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.528+0000 7f8ba2000640 10 client.15144 handle_command_reply: tid=0 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.528+0000 7f8bbe22e740 1 client.15144 shutdown 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 msgr2=0x560f4b79d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0bb010 tx=0x7f8bac009650 comp rx=0 tx=0).stop 2024-04-02T17:12:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 shutdown_connections 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b7a2c10 0x560f4b7a4080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b791450 0x560f4b79d380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78c2e0 0x560f4b796340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/484569410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b795e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8ba2000640 0 client.15144 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 >> 172.21.15.59:0/484569410 conn(0x560f4b786e60 msgr2=0x560f4b786080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 shutdown_connections 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 1 -- 172.21.15.59:0/484569410 wait complete. 2024-04-02T17:12:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.529+0000 7f8bbe22e740 20 client.15144 trim_cache size 0 max 16384 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.549+0000 7f8bbe22e740 1 Processor -- start 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.549+0000 7f8bbe22e740 1 -- start start 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 0x560f4b787240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b791450 0x560f4b795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a23c0 con 0x560f4b78c2e0 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a2a90 con 0x560f4b791450 2024-04-02T17:12:46.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a3160 con 0x560f4b78b8f0 2024-04-02T17:12:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bb3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bb3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38624/0 (socket says 172.21.15.59:38624) 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bb3e00640 1 -- 172.21.15.59:0/887558936 learned_addr learned my addr 172.21.15.59:0/887558936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bb9400640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b791450 0x560f4b795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.550+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 0x560f4b787240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 -- 172.21.15.59:0/887558936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 msgr2=0x560f4b787240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 0x560f4b787240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 -- 172.21.15.59:0/887558936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b791450 msgr2=0x560f4b795a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b791450 0x560f4b795a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 -- 172.21.15.59:0/887558936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4b7946f0 con 0x560f4b78c2e0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 0x560f4b787240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8bb3e00640 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c003f00 tx=0x7f8b9c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67927d5fe9f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8ba9200640 1 -- 172.21.15.59:0/887558936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c01ee40 con 0x560f4b78c2e0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8ba9200640 1 -- 172.21.15.59:0/887558936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c005020 con 0x560f4b78c2e0 2024-04-02T17:12:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.551+0000 7f8ba9200640 1 -- 172.21.15.59:0/887558936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c02a460 con 0x560f4b78c2e0 2024-04-02T17:12:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 -- 172.21.15.59:0/887558936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 msgr2=0x560f4b79d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c003f00 tx=0x7f8b9c009500 comp rx=0 tx=0).stop 2024-04-02T17:12:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 -- 172.21.15.59:0/887558936 shutdown_connections 2024-04-02T17:12:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b78c2e0 0x560f4b79d340 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b791450 0x560f4b795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/887558936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b78b8f0 0x560f4b787240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 -- 172.21.15.59:0/887558936 >> 172.21.15.59:0/887558936 conn(0x560f4b786e60 msgr2=0x560f4b794980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 -- 172.21.15.59:0/887558936 shutdown_connections 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.552+0000 7f8bbe22e740 1 -- 172.21.15.59:0/887558936 wait complete. 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 -- start start 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 Processor -- start 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b7994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b799a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a3f00 con 0x560f4b79c620 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a3830 con 0x560f4b78b8f0 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7a45d0 con 0x560f4b791450 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.553+0000 7f8bbe22e740 10 client.0 _ll_register_callbacks cb 0x7f8bac0ba8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:12:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b799a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38626/0 (socket says 172.21.15.59:38626) 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 -- 172.21.15.59:0/3251555717 learned_addr learned my addr 172.21.15.59:0/3251555717 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b7994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 -- 172.21.15.59:0/3251555717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 msgr2=0x560f4b799a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b799a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 -- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 msgr2=0x560f4b7994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b7994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 -- 172.21.15.59:0/3251555717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4b79d010 con 0x560f4b79c620 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8bb9400640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0c29f0 tx=0x7f8bac0bcb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04b88c93cd7bad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac005690 con 0x560f4b79c620 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.554+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bac0bacb0 con 0x560f4b79c620 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9c005020 con 0x560f4b79c620 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bac0bed40 con 0x560f4b79c620 2024-04-02T17:12:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 10 client.15150 fetch_fsmap learned FSMap version 27 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f4b809560 con 0x560f4b79c620 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c005020 con 0x560f4b79c620 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8bac0dd370 con 0x560f4b79c620 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 20 client.15150 populate_metadata read hostname 'smithi059' 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b80de90 0x560f4b80cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 4 client.15150 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T17:12:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.555+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "status"}) -- 0x560f4b6f88f0 con 0x560f4b80de90 2024-04-02T17:12:46.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.556+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b80de90 0x560f4b80cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:46.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.556+0000 7f8bb8a00640 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b80de90 0x560f4b80cba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc55adc0e8eeea8 server_cookie=d895cc5342def7b in_seq=0 out_seq=0 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.556+0000 7f8ba2000640 10 client.15150 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8ba2000640 1 -- 172.21.15.59:0/3251555717 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560f4b6f88f0 con 0x560f4b80de90 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8ba2000640 10 client.15150 handle_command_reply: tid=0 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 client.15150 shutdown 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 msgr2=0x560f4b79caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0c29f0 tx=0x7f8bac0bcb50 comp rx=0 tx=0).stop 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 shutdown_connections 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x560f4b80de90 0x560f4b80cba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f4b79c620 0x560f4b79caa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f4b791450 0x560f4b799a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 --2- 172.21.15.59:0/3251555717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560f4b78b8f0 0x560f4b7994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 >> 172.21.15.59:0/3251555717 conn(0x560f4b786e60 msgr2=0x560f4b787240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8ba2000640 0 client.15150 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:12:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 shutdown_connections 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 1 -- 172.21.15.59:0/3251555717 wait complete. 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.557+0000 7f8bbe22e740 20 client.15150 trim_cache size 0 max 16384 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "8286be62-f112-11ee-b647-cb9ed24678a4", 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24584, 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 131.074473549, 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 27, 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-04-02T17:12:46.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 136.73715497000001 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 msgr2=0x7f8b940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 0x7f8b940a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x560f4b78cb40 tx=0x7f8b9c009f90 comp rx=0 tx=0).stop 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 msgr2=0x7f8bb41a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036040 tx=0x7f8bac02c670 comp rx=0 tx=0).stop 2024-04-02T17:12:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 shutdown_connections 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8b940a3c30 0x7f8b940a60f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb41099e0 0x7f8bb41a15d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb4075ac0 0x7f8bb419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 --2- 172.21.15.59:0/3746283281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bb40744e0 0x7f8bb419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 >> 172.21.15.59:0/3746283281 conn(0x7f8bb40fd0a0 msgr2=0x7f8bb4105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 shutdown_connections 2024-04-02T17:12:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:46.558+0000 7f8bbb400640 1 -- 172.21.15.59:0/3746283281 wait complete. 2024-04-02T17:12:46.575 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T17:12:46.583 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T17:12:46.583 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T17:12:46.584 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T17:12:46.584 INFO:teuthology.run_tasks:Running task kclient... 2024-04-02T17:12:46.604 INFO:tasks.kclient:Mounting kernel clients... 2024-04-02T17:12:46.604 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-02T17:12:46.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:46.604 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-02T17:12:46.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:46.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-02T17:12:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2024-04-02T17:12:46.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T17:12:46.766 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-02T17:12:46.766 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T17:12:46.766 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T17:12:46.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T17:12:47.232 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:12:47.524 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:47 smithi059 ceph-mon[31664]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:47 smithi164 ceph-mon[36499]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:47 smithi154 ceph-mon[36479]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 -- 172.21.15.59:0/375373073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e8010a7e0 msgr2=0x7f4e8010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 --2- 172.21.15.59:0/375373073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e8010a7e0 0x7f4e8010abc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009990 tx=0x7f4e70009ca0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 -- 172.21.15.59:0/375373073 shutdown_connections 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 --2- 172.21.15.59:0/375373073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e801051a0 0x7f4e801116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 --2- 172.21.15.59:0/375373073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e8010a7e0 0x7f4e8010abc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 --2- 172.21.15.59:0/375373073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e80104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 -- 172.21.15.59:0/375373073 >> 172.21.15.59:0/375373073 conn(0x7f4e801002d0 msgr2=0x7f4e801026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 -- 172.21.15.59:0/375373073 shutdown_connections 2024-04-02T17:12:49.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.211+0000 7f4e85000700 1 -- 172.21.15.59:0/375373073 wait complete. 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.212+0000 7f4e85000700 1 Processor -- start 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 -- start start 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 0x7f4e8019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8010a7e0 0x7f4e8019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114b10 con 0x7f4e801051a0 2024-04-02T17:12:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114990 con 0x7f4e8010a7e0 2024-04-02T17:12:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e85000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114c90 con 0x7f4e801047e0 2024-04-02T17:12:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e7d400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 0x7f4e8019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.213+0000 7f4e7de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40388/0 (socket says 172.21.15.59:40388) 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7d400700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 0x7f4e8019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38644/0 (socket says 172.21.15.59:38644) 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 -- 172.21.15.59:0/4089932224 learned_addr learned my addr 172.21.15.59:0/4089932224 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7e800700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8010a7e0 0x7f4e8019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 -- 172.21.15.59:0/4089932224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8010a7e0 msgr2=0x7f4e8019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8010a7e0 0x7f4e8019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 -- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 msgr2=0x7f4e8019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 0x7f4e8019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 -- 172.21.15.59:0/4089932224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8019ff30 con 0x7f4e801047e0 2024-04-02T17:12:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.214+0000 7f4e7de00700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e7000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.215+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70027070 con 0x7f4e801047e0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.215+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a01c0 con 0x7f4e801047e0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.215+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a7aa0 con 0x7f4e801047e0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.215+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e700043d0 con 0x7f4e801047e0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.215+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70031060 con 0x7f4e801047e0 2024-04-02T17:12:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.216+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e60005330 con 0x7f4e801047e0 2024-04-02T17:12:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.218+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e7002a070 con 0x7f4e801047e0 2024-04-02T17:12:49.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.220+0000 7f4e76a00700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 0x7f4e640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.220+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4e700d0ff0 con 0x7f4e801047e0 2024-04-02T17:12:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.220+0000 7f4e7d400700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 0x7f4e640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.221+0000 7f4e7d400700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 0x7f4e640a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e80109d10 tx=0x7f4e6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:49.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.222+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4e70013070 con 0x7f4e801047e0 2024-04-02T17:12:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:49 smithi059 ceph-mon[31664]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:49.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.503+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e600058f0 con 0x7f4e801047e0 2024-04-02T17:12:49.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.504+0000 7f4e76a00700 1 -- 172.21.15.59:0/4089932224 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f4e7009d9c0 con 0x7f4e801047e0 2024-04-02T17:12:49.504 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-02T17:12:49.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 msgr2=0x7f4e640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:49.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 0x7f4e640a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e80109d10 tx=0x7f4e6c009260 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 msgr2=0x7f4e8019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e70000c00 tx=0x7f4e7000bab0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 shutdown_connections 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8010a7e0 0x7f4e8019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e801047e0 0x7f4e8019af20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e640a3c40 0x7f4e640a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 --2- 172.21.15.59:0/4089932224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e801051a0 0x7f4e8019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 >> 172.21.15.59:0/4089932224 conn(0x7f4e801002d0 msgr2=0x7f4e80100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 shutdown_connections 2024-04-02T17:12:49.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:49.507+0000 7f4e85000700 1 -- 172.21.15.59:0/4089932224 wait complete. 2024-04-02T17:12:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:49 smithi164 ceph-mon[36499]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:49 smithi154 ceph-mon[36479]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T17:12:50.241 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T17:12:50.241 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T17:12:50.241 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T17:12:50.242 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T17:12:50.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:50.242 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-02T17:12:50.302 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41947sec preferred_lft 41947sec 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-02T17:12:50.303 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-02T17:12:50.304 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T17:12:50.304 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T17:12:50.304 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-02T17:12:50.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-04-02T17:12:50.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-04-02T17:12:50.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-04-02T17:12:50.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T17:12:50.305 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-02T17:12:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:50.455 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:50 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4089932224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T17:12:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:50 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4089932224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T17:12:50.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:50 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4089932224' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T17:12:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:50.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:50.868 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T17:12:50.949 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-02T17:12:50.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:50.953 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-02T17:12:51.011 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-02T17:12:51.011 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T17:12:51.011 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T17:12:51.011 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T17:12:51.012 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T17:12:51.012 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-02T17:12:51.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:51.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:51 smithi059 ceph-mon[31664]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:51 smithi164 ceph-mon[36499]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:51 smithi154 ceph-mon[36479]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:51.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:51.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:51.832 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-02T17:12:51.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:51.892 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-04-02T17:12:51.952 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T17:12:51.952 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-02T17:12:51.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T17:12:51.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T17:12:51.953 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-02T17:12:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:52.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:52.173 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T17:12:52.174 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-02T17:12:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:52.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:52.551 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T17:12:52.551 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-02T17:12:52.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-02T17:12:52.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T17:12:52.552 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-02T17:12:52.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:52.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:52.794 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T17:12:52.795 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T17:12:52.844 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T17:12:52.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T17:12:52.844 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T17:12:52.892 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-02T17:12:52.892 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-02T17:12:52.893 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 2024-04-02T17:12:52.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:52.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T17:12:53.038 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T17:12:53.038 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-02T17:12:53.038 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-04-02T17:12:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:53 smithi059 ceph-mon[31664]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:53.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:53 smithi154 ceph-mon[36479]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:53 smithi164 ceph-mon[36499]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.154:3300,172.21.15.164:3300" 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:3300,172.21.15.154:3300,172.21.15.164:3300:/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 2024-04-02T17:12:53.736 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=8286be62-f112-11ee-b647-cb9ed24678a4" will pass to kernel 2024-04-02T17:12:53.740 INFO:teuthology.orchestra.run.smithi059.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T17:12:53.740 INFO:teuthology.orchestra.run.smithi059.stdout: You just mounted a file system that supports labels which does not 2024-04-02T17:12:53.740 INFO:teuthology.orchestra.run.smithi059.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T17:12:53.740 INFO:teuthology.orchestra.run.smithi059.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T17:12:53.740 INFO:teuthology.orchestra.run.smithi059.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T17:12:53.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T17:12:53.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:12:53.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T17:12:53.824 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T17:12:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T17:12:53.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:53.939 INFO:teuthology.orchestra.run.smithi059.stdout:client15159 2024-04-02T17:12:53.939 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:12:53.939 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T17:12:53.989 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T17:12:54.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T17:12:54.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:54.103 INFO:teuthology.orchestra.run.smithi059.stdout:client15159 2024-04-02T17:12:54.103 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T17:12:54.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T17:12:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:54.212 INFO:teuthology.orchestra.run.smithi059.stdout:client15159 2024-04-02T17:12:54.212 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8286be62-f112-11ee-b647-cb9ed24678a4.client15159/status') 2024-04-02T17:12:54.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T17:12:54.351 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.15159 (3)192.168.0.1:0/1146901436 2024-04-02T17:12:54.352 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2024-04-02T17:12:54.352 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2024-04-02T17:12:54.352 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2024-04-02T17:12:54.352 INFO:teuthology.orchestra.run.smithi059.stderr:70 bytes copied, 6.2137e-05 s, 1.1 MB/s 2024-04-02T17:12:54.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T17:12:54.365 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T17:12:54.375 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T17:12:54.375 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:12:54.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-02T17:12:54.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.583+0000 7fab3d600640 1 -- 172.21.15.59:0/624366082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 msgr2=0x7fab38101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.583+0000 7fab3d600640 1 --2- 172.21.15.59:0/624366082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab38101150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab24009900 tx=0x7fab2402bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 -- 172.21.15.59:0/624366082 shutdown_connections 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 --2- 172.21.15.59:0/624366082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab38101150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 --2- 172.21.15.59:0/624366082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38101690 0x7fab38101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 --2- 172.21.15.59:0/624366082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38100390 0x7fab381007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 -- 172.21.15.59:0/624366082 >> 172.21.15.59:0/624366082 conn(0x7fab380fbf90 msgr2=0x7fab380fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 -- 172.21.15.59:0/624366082 shutdown_connections 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.584+0000 7fab3d600640 1 -- 172.21.15.59:0/624366082 wait complete. 2024-04-02T17:12:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 Processor -- start 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 -- start start 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38100390 0x7fab38199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38101690 0x7fab381a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab3810f9e0 con 0x7fab38100cf0 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab3810f860 con 0x7fab38101690 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab3d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab3810fb60 con 0x7fab38100390 2024-04-02T17:12:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59992/0 (socket says 172.21.15.59:59992) 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.585+0000 7fab36000640 1 -- 172.21.15.59:0/527426275 learned_addr learned my addr 172.21.15.59:0/527426275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36a00640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38100390 0x7fab38199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab37400640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38101690 0x7fab381a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 -- 172.21.15.59:0/527426275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38100390 msgr2=0x7fab38199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38100390 0x7fab38199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 -- 172.21.15.59:0/527426275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38101690 msgr2=0x7fab381a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38101690 0x7fab381a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 -- 172.21.15.59:0/527426275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab381a1b40 con 0x7fab38100cf0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab36000640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab2402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.586+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24043020 con 0x7fab38100cf0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.587+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab240054e0 con 0x7fab38100cf0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.587+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab381a1dd0 con 0x7fab38100cf0 2024-04-02T17:12:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.587+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24003c40 con 0x7fab38100cf0 2024-04-02T17:12:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.587+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab381a22e0 con 0x7fab38100cf0 2024-04-02T17:12:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.588+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3810b150 con 0x7fab38100cf0 2024-04-02T17:12:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.590+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab24046020 con 0x7fab38100cf0 2024-04-02T17:12:54.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.592+0000 7fab2f400640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 0x7fab1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:54.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.592+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fab240f2440 con 0x7fab38100cf0 2024-04-02T17:12:54.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.592+0000 7fab36a00640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 0x7fab1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:54.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.593+0000 7fab36a00640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 0x7fab1c0a6190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fab28009bd0 tx=0x7fab28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.597+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab2404d020 con 0x7fab38100cf0 2024-04-02T17:12:54.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:54.844+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fab3802d370 con 0x7fab38100cf0 2024-04-02T17:12:55.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.215+0000 7fab2f400640 1 -- 172.21.15.59:0/527426275 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fab240bed60 con 0x7fab38100cf0 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.219+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 msgr2=0x7fab1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.219+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 0x7fab1c0a6190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fab28009bd0 tx=0x7fab28005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.219+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 msgr2=0x7fab3819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.219+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab2402c650 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 shutdown_connections 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fab1c0a3cd0 0x7fab1c0a6190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab38100cf0 0x7fab3819a4d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab38101690 0x7fab381a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 --2- 172.21.15.59:0/527426275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab38100390 0x7fab38199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 >> 172.21.15.59:0/527426275 conn(0x7fab380fbf90 msgr2=0x7fab380fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:55.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 shutdown_connections 2024-04-02T17:12:55.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.220+0000 7fab3d600640 1 -- 172.21.15.59:0/527426275 wait complete. 2024-04-02T17:12:55.241 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-02T17:12:55.252 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-02T17:12:55.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:55.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.490+0000 7f5e47200640 1 -- 172.21.15.59:0/1386953005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48109800 msgr2=0x7f5e4810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.490+0000 7f5e47200640 1 --2- 172.21.15.59:0/1386953005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48109800 0x7f5e4810bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009be0 tx=0x7f5e4002bef0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 -- 172.21.15.59:0/1386953005 shutdown_connections 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 --2- 172.21.15.59:0/1386953005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48109800 0x7f5e4810bc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 --2- 172.21.15.59:0/1386953005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48074560 0x7f5e480749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 --2- 172.21.15.59:0/1386953005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 0x7f5e48073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 -- 172.21.15.59:0/1386953005 >> 172.21.15.59:0/1386953005 conn(0x7f5e480fceb0 msgr2=0x7f5e480ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 -- 172.21.15.59:0/1386953005 shutdown_connections 2024-04-02T17:12:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.491+0000 7f5e47200640 1 -- 172.21.15.59:0/1386953005 wait complete. 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 Processor -- start 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 -- start start 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 0x7f5e4819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48109800 0x7f5e481a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810f7f0 con 0x7f5e48075b40 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810f670 con 0x7f5e48074560 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e47200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4810f970 con 0x7f5e48109800 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e45e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e45e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55638/0 (socket says 172.21.15.59:55638) 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.492+0000 7f5e45e00640 1 -- 172.21.15.59:0/932798010 learned_addr learned my addr 172.21.15.59:0/932798010 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e46800640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48109800 0x7f5e481a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45400640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 0x7f5e4819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 -- 172.21.15.59:0/932798010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48109800 msgr2=0x7f5e481a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48109800 0x7f5e481a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 -- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 msgr2=0x7f5e4819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 0x7f5e4819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 -- 172.21.15.59:0/932798010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e481a1a20 con 0x7f5e48074560 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.493+0000 7f5e45e00640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800d950 tx=0x7f5e3800dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.494+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38017070 con 0x7f5e48074560 2024-04-02T17:12:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.494+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3800de80 con 0x7f5e48074560 2024-04-02T17:12:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.494+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800f620 con 0x7f5e48074560 2024-04-02T17:12:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.494+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e481a1d10 con 0x7f5e48074560 2024-04-02T17:12:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.494+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481a21b0 con 0x7f5e48074560 2024-04-02T17:12:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.495+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e48108530 con 0x7f5e48074560 2024-04-02T17:12:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.497+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5e38009c30 con 0x7f5e48074560 2024-04-02T17:12:55.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.498+0000 7f5e3ea00640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 0x7f5e2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.498+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5e380cfff0 con 0x7f5e48074560 2024-04-02T17:12:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.499+0000 7f5e45400640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 0x7f5e2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.499+0000 7f5e45400640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 0x7f5e2c0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e4819b310 tx=0x7f5e34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:55.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.503+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5e3809c970 con 0x7f5e48074560 2024-04-02T17:12:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:55 smithi059 ceph-mon[31664]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:55 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T17:12:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:55 smithi164 ceph-mon[36499]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:55 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T17:12:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:55 smithi154 ceph-mon[36479]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-02T17:12:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:55 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T17:12:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.702+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e48062840 con 0x7f5e48074560 2024-04-02T17:12:55.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.704+0000 7f5e3ea00640 1 -- 172.21.15.59:0/932798010 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f5e3809c270 con 0x7f5e48074560 2024-04-02T17:12:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:55.705 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.706+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 msgr2=0x7f5e2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.706+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 0x7f5e2c0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e4819b310 tx=0x7f5e34009260 comp rx=0 tx=0).stop 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.706+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 msgr2=0x7f5e48199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.706+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800d950 tx=0x7f5e3800dd50 comp rx=0 tx=0).stop 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 shutdown_connections 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5e2c0a3c30 0x7f5e2c0a60f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e48075b40 0x7f5e4819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e48109800 0x7f5e481a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 --2- 172.21.15.59:0/932798010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e48074560 0x7f5e48199e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 >> 172.21.15.59:0/932798010 conn(0x7f5e480fceb0 msgr2=0x7f5e48105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:55.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 shutdown_connections 2024-04-02T17:12:55.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.707+0000 7f5e47200640 1 -- 172.21.15.59:0/932798010 wait complete. 2024-04-02T17:12:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:55.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.949+0000 7f09a9e00640 1 -- 172.21.15.59:0/851424625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 msgr2=0x7f09a41102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.949+0000 7f09a9e00640 1 --2- 172.21.15.59:0/851424625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41102c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f099c009900 tx=0x7f099c02bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 -- 172.21.15.59:0/851424625 shutdown_connections 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 --2- 172.21.15.59:0/851424625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41102c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 --2- 172.21.15.59:0/851424625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4074560 0x7f09a40749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 --2- 172.21.15.59:0/851424625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4075b40 0x7f09a4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 -- 172.21.15.59:0/851424625 >> 172.21.15.59:0/851424625 conn(0x7f09a40fce90 msgr2=0x7f09a40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 -- 172.21.15.59:0/851424625 shutdown_connections 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.950+0000 7f09a9e00640 1 -- 172.21.15.59:0/851424625 wait complete. 2024-04-02T17:12:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.951+0000 7f09a9e00640 1 Processor -- start 2024-04-02T17:12:55.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.951+0000 7f09a9e00640 1 -- start start 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 0x7f09a419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4113ba0 con 0x7f09a4074560 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4113a20 con 0x7f09a4075b40 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4113d20 con 0x7f09a410ded0 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60026/0 (socket says 172.21.15.59:60026) 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a8a00640 1 -- 172.21.15.59:0/590646389 learned_addr learned my addr 172.21.15.59:0/590646389 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a3e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 0x7f09a419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.952+0000 7f09a9400640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 -- 172.21.15.59:0/590646389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 msgr2=0x7f09a41a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 -- 172.21.15.59:0/590646389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 msgr2=0x7f09a419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 0x7f09a419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 -- 172.21.15.59:0/590646389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41a5d60 con 0x7f09a4074560 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a3e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 0x7f09a419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a8a00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0994009d50 tx=0x7f099400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994003b70 con 0x7f09a4074560 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994003d10 con 0x7f09a4074560 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41a5ff0 con 0x7f09a4074560 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.953+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994010a20 con 0x7f09a4074560 2024-04-02T17:12:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.954+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a6500 con 0x7f09a4074560 2024-04-02T17:12:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.955+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a41040a0 con 0x7f09a4074560 2024-04-02T17:12:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.957+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0994003ec0 con 0x7f09a4074560 2024-04-02T17:12:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.958+0000 7f09a1600640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 0x7f09880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.958+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f09940d0ba0 con 0x7f09a4074560 2024-04-02T17:12:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.958+0000 7f09a3e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 0x7f09880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:55.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.959+0000 7f09a3e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 0x7f09880a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09a419f650 tx=0x7f0990005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:55.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:55.962+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f099409d520 con 0x7f09a4074560 2024-04-02T17:12:56.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.169+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09a40627a0 con 0x7f09a4074560 2024-04-02T17:12:56.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.170+0000 7f09a1600640 1 -- 172.21.15.59:0/590646389 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f099409ce20 con 0x7f09a4074560 2024-04-02T17:12:56.170 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 msgr2=0x7f09880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 0x7f09880a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09a419f650 tx=0x7f0990005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 msgr2=0x7f09a419e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0994009d50 tx=0x7f099400c6a0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 shutdown_connections 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09a410ded0 0x7f09a41a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a4075b40 0x7f09a419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.172+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09880a3c30 0x7f09880a60f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.173+0000 7f09a9e00640 1 --2- 172.21.15.59:0/590646389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a4074560 0x7f09a419e140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.173+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 >> 172.21.15.59:0/590646389 conn(0x7f09a40fce90 msgr2=0x7f09a4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:56.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.173+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 shutdown_connections 2024-04-02T17:12:56.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.173+0000 7f09a9e00640 1 -- 172.21.15.59:0/590646389 wait complete. 2024-04-02T17:12:56.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:56.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 -- 172.21.15.59:0/3991255736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 msgr2=0x7efe3c104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 --2- 172.21.15.59:0/3991255736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c104140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009ca0 tx=0x7efe2c02c100 comp rx=0 tx=0).stop 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 -- 172.21.15.59:0/3991255736 shutdown_connections 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 --2- 172.21.15.59:0/3991255736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c1058d0 0x7efe3c10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 --2- 172.21.15.59:0/3991255736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c104f10 0x7efe3c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 --2- 172.21.15.59:0/3991255736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c104140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 -- 172.21.15.59:0/3991255736 >> 172.21.15.59:0/3991255736 conn(0x7efe3c0ff2b0 msgr2=0x7efe3c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 -- 172.21.15.59:0/3991255736 shutdown_connections 2024-04-02T17:12:56.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.408+0000 7efe42200640 1 -- 172.21.15.59:0/3991255736 wait complete. 2024-04-02T17:12:56.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.409+0000 7efe42200640 1 Processor -- start 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.409+0000 7efe42200640 1 -- start start 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.409+0000 7efe42200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe42200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c104f10 0x7efe3c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe42200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c1058d0 0x7efe3c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe42200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10f970 con 0x7efe3c104f10 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe42200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10f7f0 con 0x7efe3c1058d0 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe42200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10faf0 con 0x7efe3c103d20 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c104f10 0x7efe3c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50426/0 (socket says 172.21.15.59:50426) 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 -- 172.21.15.59:0/587056166 learned_addr learned my addr 172.21.15.59:0/587056166 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3be00640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c1058d0 0x7efe3c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 -- 172.21.15.59:0/587056166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c1058d0 msgr2=0x7efe3c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c1058d0 0x7efe3c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 -- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c104f10 msgr2=0x7efe3c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c104f10 0x7efe3c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.410+0000 7efe3b400640 1 -- 172.21.15.59:0/587056166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c1a1c30 con 0x7efe3c103d20 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe3b400640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f80 tx=0x7efe2c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c039040 con 0x7efe3c103d20 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c044070 con 0x7efe3c103d20 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c1a1ec0 con 0x7efe3c103d20 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c004430 con 0x7efe3c103d20 2024-04-02T17:12:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.411+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1a22c0 con 0x7efe3c103d20 2024-04-02T17:12:56.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.412+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c10b110 con 0x7efe3c103d20 2024-04-02T17:12:56.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.414+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efe2c0045d0 con 0x7efe3c103d20 2024-04-02T17:12:56.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.416+0000 7efe33e00640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 0x7efe200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.416+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7efe2c0f2cc0 con 0x7efe3c103d20 2024-04-02T17:12:56.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.416+0000 7efe3aa00640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 0x7efe200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.417+0000 7efe3aa00640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 0x7efe200a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe3c19b520 tx=0x7efe28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.418+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efe2c0bf700 con 0x7efe3c103d20 2024-04-02T17:12:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T17:12:56.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:56 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:12:56.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/932798010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:56.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:56 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/590646389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:56 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T17:12:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:56 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:12:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:56 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/932798010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:56 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/590646389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:56.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.628+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe3c0627a0 con 0x7efe3c103d20 2024-04-02T17:12:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.629+0000 7efe33e00640 1 -- 172.21.15.59:0/587056166 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7efe2c0bf000 con 0x7efe3c103d20 2024-04-02T17:12:56.629 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.631+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 msgr2=0x7efe200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.631+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 0x7efe200a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe3c19b520 tx=0x7efe28005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.631+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 msgr2=0x7efe3c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.631+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f80 tx=0x7efe2c035d10 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 shutdown_connections 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe200a3bb0 0x7efe200a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe3c104f10 0x7efe3c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c1058d0 0x7efe3c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 --2- 172.21.15.59:0/587056166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe3c103d20 0x7efe3c19a010 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 >> 172.21.15.59:0/587056166 conn(0x7efe3c0ff2b0 msgr2=0x7efe3c1082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:56.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 shutdown_connections 2024-04-02T17:12:56.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.632+0000 7efe42200640 1 -- 172.21.15.59:0/587056166 wait complete. 2024-04-02T17:12:56.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:56 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/527426275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T17:12:56.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:56 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:12:56.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:56 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/932798010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:56.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:56 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/590646389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:56.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:56.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 -- 172.21.15.59:0/3923895538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99010dfd0 msgr2=0x7fa9901103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 --2- 172.21.15.59:0/3923895538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99010dfd0 0x7fa9901103c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa988009c10 tx=0x7fa98802bef0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 -- 172.21.15.59:0/3923895538 shutdown_connections 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 --2- 172.21.15.59:0/3923895538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990074560 0x7fa9900749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 --2- 172.21.15.59:0/3923895538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99010dfd0 0x7fa9901103c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 --2- 172.21.15.59:0/3923895538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa990073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 -- 172.21.15.59:0/3923895538 >> 172.21.15.59:0/3923895538 conn(0x7fa9900fcf70 msgr2=0x7fa9900ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 -- 172.21.15.59:0/3923895538 shutdown_connections 2024-04-02T17:12:56.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.877+0000 7fa996400640 1 -- 172.21.15.59:0/3923895538 wait complete. 2024-04-02T17:12:56.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.878+0000 7fa996400640 1 Processor -- start 2024-04-02T17:12:56.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.878+0000 7fa996400640 1 -- start start 2024-04-02T17:12:56.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.878+0000 7fa996400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990074560 0x7fa99019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.878+0000 7fa996400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa99019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa996400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 0x7fa9901a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa996400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa990113c90 con 0x7fa99010dfd0 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa996400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa990113b10 con 0x7fa990074560 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa996400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa990113e10 con 0x7fa990075b40 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa99019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 0x7fa9901a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa99019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50450/0 (socket says 172.21.15.59:50450) 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa98fe00640 1 -- 172.21.15.59:0/1967231210 learned_addr learned my addr 172.21.15.59:0/1967231210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995000640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990074560 0x7fa99019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 -- 172.21.15.59:0/1967231210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 msgr2=0x7fa99019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa99019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 -- 172.21.15.59:0/1967231210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990074560 msgr2=0x7fa99019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990074560 0x7fa99019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:56.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.879+0000 7fa995a00640 1 -- 172.21.15.59:0/1967231210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9901a5ec0 con 0x7fa99010dfd0 2024-04-02T17:12:56.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa995a00640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 0x7fa9901a5850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa98802c430 tx=0x7fa98802cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:56.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988043020 con 0x7fa99010dfd0 2024-04-02T17:12:56.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa988003e50 con 0x7fa99010dfd0 2024-04-02T17:12:56.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9901a6150 con 0x7fa99010dfd0 2024-04-02T17:12:56.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988034690 con 0x7fa99010dfd0 2024-04-02T17:12:56.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.880+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901a66f0 con 0x7fa99010dfd0 2024-04-02T17:12:56.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.881+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa990104090 con 0x7fa99010dfd0 2024-04-02T17:12:56.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.883+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa988046020 con 0x7fa99010dfd0 2024-04-02T17:12:56.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.885+0000 7fa98d600640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 0x7fa9740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:56.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.885+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa9880f2950 con 0x7fa99010dfd0 2024-04-02T17:12:56.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.886+0000 7fa995000640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 0x7fa9740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:56.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.886+0000 7fa995000640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 0x7fa9740a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa980009bd0 tx=0x7fa980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:56.889+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa98804d020 con 0x7fa99010dfd0 2024-04-02T17:12:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.096+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa990062700 con 0x7fa99010dfd0 2024-04-02T17:12:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.097+0000 7fa98d600640 1 -- 172.21.15.59:0/1967231210 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fa9880bf270 con 0x7fa99010dfd0 2024-04-02T17:12:57.097 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.099+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 msgr2=0x7fa9740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.099+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 0x7fa9740a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa980009bd0 tx=0x7fa980005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.099+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 msgr2=0x7fa9901a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.099+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 0x7fa9901a5850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa98802c430 tx=0x7fa98802cb90 comp rx=0 tx=0).stop 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 shutdown_connections 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa9740a3c30 0x7fa9740a60f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa99010dfd0 0x7fa9901a5850 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990075b40 0x7fa99019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 --2- 172.21.15.59:0/1967231210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990074560 0x7fa99019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 >> 172.21.15.59:0/1967231210 conn(0x7fa9900fcf70 msgr2=0x7fa990101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 shutdown_connections 2024-04-02T17:12:57.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.100+0000 7fa996400640 1 -- 172.21.15.59:0/1967231210 wait complete. 2024-04-02T17:12:57.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 -- 172.21.15.59:0/644928914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4074500 msgr2=0x7f63b4074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 --2- 172.21.15.59:0/644928914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4074500 0x7f63b4074960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f639c009900 tx=0x7f639c02bcc0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 -- 172.21.15.59:0/644928914 shutdown_connections 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 --2- 172.21.15.59:0/644928914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b41099b0 0x7f63b410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 --2- 172.21.15.59:0/644928914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4074500 0x7f63b4074960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 --2- 172.21.15.59:0/644928914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b4075ae0 0x7f63b4073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 -- 172.21.15.59:0/644928914 >> 172.21.15.59:0/644928914 conn(0x7f63b40fd0b0 msgr2=0x7f63b40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:57.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 -- 172.21.15.59:0/644928914 shutdown_connections 2024-04-02T17:12:57.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.349+0000 7f63b8a00640 1 -- 172.21.15.59:0/644928914 wait complete. 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.350+0000 7f63b8a00640 1 Processor -- start 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.350+0000 7f63b8a00640 1 -- start start 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4074500 0x7f63b419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b41099b0 0x7f63b41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410f9d0 con 0x7f63b4074500 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410f850 con 0x7f63b41099b0 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410fb50 con 0x7f63b4075ae0 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4074500 0x7f63b419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50472/0 (socket says 172.21.15.59:50472) 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 -- 172.21.15.59:0/2218337052 learned_addr learned my addr 172.21.15.59:0/2218337052 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b2800640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b41099b0 0x7f63b41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 -- 172.21.15.59:0/2218337052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b41099b0 msgr2=0x7f63b41a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b41099b0 0x7f63b41a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 -- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4074500 msgr2=0x7f63b419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4074500 0x7f63b419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.351+0000 7f63b1400640 1 -- 172.21.15.59:0/2218337052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b41a1c40 con 0x7f63b4075ae0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63b1400640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c02c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c043020 con 0x7f63b4075ae0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f639c0054e0 con 0x7f63b4075ae0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41a1ed0 con 0x7f63b4075ae0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c003c20 con 0x7f63b4075ae0 2024-04-02T17:12:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.352+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41a2390 con 0x7f63b4075ae0 2024-04-02T17:12:57.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.353+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b4108770 con 0x7f63b4075ae0 2024-04-02T17:12:57.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.355+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f639c046020 con 0x7f63b4075ae0 2024-04-02T17:12:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.357+0000 7f63aaa00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 0x7f63980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.357+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f639c0f2b10 con 0x7f63b4075ae0 2024-04-02T17:12:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.357+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 0x7f63980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.358+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 0x7f63980a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009bd0 tx=0x7f63ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:57.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.359+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f639c04d020 con 0x7f63b4075ae0 2024-04-02T17:12:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:57 smithi059 ceph-mon[31664]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:57 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/587056166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:57 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1967231210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.567+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b402d4f0 con 0x7f63b4075ae0 2024-04-02T17:12:57.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.570+0000 7f63aaa00640 1 -- 172.21.15.59:0/2218337052 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f639c0bf430 con 0x7f63b4075ae0 2024-04-02T17:12:57.570 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:57.570 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:12:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:57 smithi164 ceph-mon[36499]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:57 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/587056166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:57 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1967231210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 msgr2=0x7f63980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 0x7f63980a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009bd0 tx=0x7f63ac005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 msgr2=0x7f63b419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c02c6e0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 shutdown_connections 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63b41099b0 0x7f63b41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b4075ae0 0x7f63b419a560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63980a3bb0 0x7f63980a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2218337052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63b4074500 0x7f63b419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 >> 172.21.15.59:0/2218337052 conn(0x7f63b40fd0b0 msgr2=0x7f63b4105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:57.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 shutdown_connections 2024-04-02T17:12:57.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.573+0000 7f63b8a00640 1 -- 172.21.15.59:0/2218337052 wait complete. 2024-04-02T17:12:57.592 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:12:57.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:12:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:57 smithi154 ceph-mon[36479]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:12:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:57 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/587056166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:57 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1967231210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.822+0000 7f55db800640 1 -- 172.21.15.59:0/2358447612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 msgr2=0x7f55d41072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.822+0000 7f55db800640 1 --2- 172.21.15.59:0/2358447612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d41072a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009900 tx=0x7f55c002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 -- 172.21.15.59:0/2358447612 shutdown_connections 2024-04-02T17:12:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 --2- 172.21.15.59:0/2358447612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 --2- 172.21.15.59:0/2358447612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d41072a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 --2- 172.21.15.59:0/2358447612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 -- 172.21.15.59:0/2358447612 >> 172.21.15.59:0/2358447612 conn(0x7f55d40fbf90 msgr2=0x7f55d40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 -- 172.21.15.59:0/2358447612 shutdown_connections 2024-04-02T17:12:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.823+0000 7f55db800640 1 -- 172.21.15.59:0/2358447612 wait complete. 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 Processor -- start 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 -- start start 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410d780 con 0x7f55d4102580 2024-04-02T17:12:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410d600 con 0x7f55d41077e0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55db800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410d900 con 0x7f55d4104eb0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60112/0 (socket says 172.21.15.59:60112) 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d8e00640 1 -- 172.21.15.59:0/454696987 learned_addr learned my addr 172.21.15.59:0/454696987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.824+0000 7f55d3e00640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50488/0 (socket says 172.21.15.59:50488) 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 -- 172.21.15.59:0/454696987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 msgr2=0x7f55d419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 -- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 msgr2=0x7f55d419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 -- 172.21.15.59:0/454696987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41a1c30 con 0x7f55d41077e0 2024-04-02T17:12:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d9800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d41a15c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e970 tx=0x7f55cc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.825+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00ccc0 con 0x7f55d41077e0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.826+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d41a1f20 con 0x7f55d41077e0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.826+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55cc005020 con 0x7f55d41077e0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.826+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc004a00 con 0x7f55d41077e0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.826+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41a23e0 con 0x7f55d41077e0 2024-04-02T17:12:57.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.827+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d4109330 con 0x7f55d41077e0 2024-04-02T17:12:57.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.830+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f55cc0051d0 con 0x7f55d41077e0 2024-04-02T17:12:57.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.832+0000 7f55d1600640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 0x7f55bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:57.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.832+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f55cc0d0410 con 0x7f55d41077e0 2024-04-02T17:12:57.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.832+0000 7f55d8e00640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 0x7f55bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:57.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.833+0000 7f55d8e00640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 0x7f55bc0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009fd0 tx=0x7f55c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:57.836+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f55cc09cd30 con 0x7f55d41077e0 2024-04-02T17:12:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.046+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55d40627a0 con 0x7f55d41077e0 2024-04-02T17:12:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.047+0000 7f55d1600640 1 -- 172.21.15.59:0/454696987 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f55cc09c630 con 0x7f55d41077e0 2024-04-02T17:12:58.047 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 msgr2=0x7f55bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 0x7f55bc0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009fd0 tx=0x7f55c8009500 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 msgr2=0x7f55d41a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d41a15c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e970 tx=0x7f55cc00ee30 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 shutdown_connections 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55d41077e0 0x7f55d41a15c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55d4104eb0 0x7f55d419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55bc0a3bb0 0x7f55bc0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 --2- 172.21.15.59:0/454696987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4102580 0x7f55d419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 >> 172.21.15.59:0/454696987 conn(0x7f55d40fbf90 msgr2=0x7f55d4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 shutdown_connections 2024-04-02T17:12:58.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.050+0000 7f55db800640 1 -- 172.21.15.59:0/454696987 wait complete. 2024-04-02T17:12:58.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 -- 172.21.15.59:0/2169442022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 msgr2=0x7f43b80749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 --2- 172.21.15.59:0/2169442022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b80749e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009900 tx=0x7f43a002bc30 comp rx=0 tx=0).stop 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 -- 172.21.15.59:0/2169442022 shutdown_connections 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 --2- 172.21.15.59:0/2169442022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b81098c0 0x7f43b810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 --2- 172.21.15.59:0/2169442022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b80749e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 --2- 172.21.15.59:0/2169442022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b8075b40 0x7f43b8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 -- 172.21.15.59:0/2169442022 >> 172.21.15.59:0/2169442022 conn(0x7f43b80fcf70 msgr2=0x7f43b80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 -- 172.21.15.59:0/2169442022 shutdown_connections 2024-04-02T17:12:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.277+0000 7f43bd400640 1 -- 172.21.15.59:0/2169442022 wait complete. 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.278+0000 7f43bd400640 1 Processor -- start 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.278+0000 7f43bd400640 1 -- start start 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b8075b40 0x7f43b8105260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b81098c0 0x7f43b8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810f840 con 0x7f43b81098c0 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810f6c0 con 0x7f43b8074560 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bd400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810f9c0 con 0x7f43b8075b40 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43b7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43b7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55700/0 (socket says 172.21.15.59:55700) 2024-04-02T17:12:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43b7e00640 1 -- 172.21.15.59:0/1562331101 learned_addr learned my addr 172.21.15.59:0/1562331101 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43bca00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b81098c0 0x7f43b8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.279+0000 7f43afe00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b8075b40 0x7f43b8105260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 -- 172.21.15.59:0/1562331101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b8075b40 msgr2=0x7f43b8105260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b8075b40 0x7f43b8105260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 -- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b81098c0 msgr2=0x7f43b8101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b81098c0 0x7f43b8101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 -- 172.21.15.59:0/1562331101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b8102150 con 0x7f43b8074560 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b7e00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d950 tx=0x7f43a800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8017070 con 0x7f43b8074560 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8009c30 con 0x7f43b8074560 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a800f6b0 con 0x7f43b8074560 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81a6350 con 0x7f43b8074560 2024-04-02T17:12:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.280+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a6700 con 0x7f43b8074560 2024-04-02T17:12:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.282+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b8068810 con 0x7f43b8074560 2024-04-02T17:12:58.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.284+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f43a80109d0 con 0x7f43b8074560 2024-04-02T17:12:58.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.285+0000 7f43b5600640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 0x7f439c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.285+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f43a80cffb0 con 0x7f43b8074560 2024-04-02T17:12:58.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.285+0000 7f43afe00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 0x7f439c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.286+0000 7f43afe00640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 0x7f439c0a60f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c7b0 tx=0x7f43a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:58.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.289+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f43a8009de0 con 0x7f43b8074560 2024-04-02T17:12:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.501+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43b8062700 con 0x7f43b8074560 2024-04-02T17:12:58.502 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2218337052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.502 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.503 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:58 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/454696987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.508+0000 7f43b5600640 1 -- 172.21.15.59:0/1562331101 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f43a809c280 con 0x7f43b8074560 2024-04-02T17:12:58.508 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:12:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 msgr2=0x7f439c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 0x7f439c0a60f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c7b0 tx=0x7f43a0005fd0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 msgr2=0x7f43b8104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d950 tx=0x7f43a800de10 comp rx=0 tx=0).stop 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 shutdown_connections 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f439c0a3c30 0x7f439c0a60f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43b81098c0 0x7f43b8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43b8075b40 0x7f43b8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 --2- 172.21.15.59:0/1562331101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43b8074560 0x7f43b8104d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 >> 172.21.15.59:0/1562331101 conn(0x7f43b80fcf70 msgr2=0x7f43b81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 shutdown_connections 2024-04-02T17:12:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.511+0000 7f43bd400640 1 -- 172.21.15.59:0/1562331101 wait complete. 2024-04-02T17:12:58.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2218337052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.528 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:58 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/454696987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:58.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:12:58.570 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2218337052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:12:58.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:58 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/454696987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 -- 172.21.15.59:0/3505356004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014074500 msgr2=0x7f1014074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 --2- 172.21.15.59:0/3505356004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014074500 0x7f1014074960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1000009860 tx=0x7f100002bc10 comp rx=0 tx=0).stop 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 -- 172.21.15.59:0/3505356004 shutdown_connections 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 --2- 172.21.15.59:0/3505356004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014109890 0x7f101410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 --2- 172.21.15.59:0/3505356004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014074500 0x7f1014074960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 --2- 172.21.15.59:0/3505356004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014075ae0 0x7f1014073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 -- 172.21.15.59:0/3505356004 >> 172.21.15.59:0/3505356004 conn(0x7f10140fcf70 msgr2=0x7f10140ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 -- 172.21.15.59:0/3505356004 shutdown_connections 2024-04-02T17:12:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.727+0000 7f1019e00640 1 -- 172.21.15.59:0/3505356004 wait complete. 2024-04-02T17:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.728+0000 7f1019e00640 1 Processor -- start 2024-04-02T17:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 -- start start 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 0x7f1014199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 0x7f101419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014109890 0x7f10141a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f101410f860 con 0x7f1014074500 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f101410f6e0 con 0x7f1014075ae0 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1019e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f101410f9e0 con 0x7f1014109890 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1018a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 0x7f1014199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.729+0000 7f1013e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 0x7f101419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1018a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 0x7f1014199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60134/0 (socket says 172.21.15.59:60134) 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1018a00640 1 -- 172.21.15.59:0/643178950 learned_addr learned my addr 172.21.15.59:0/643178950 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1019400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014109890 0x7f10141a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 -- 172.21.15.59:0/643178950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014109890 msgr2=0x7f10141a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014109890 0x7f10141a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 -- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 msgr2=0x7f1014199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 0x7f1014199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 -- 172.21.15.59:0/643178950 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141a1ad0 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.730+0000 7f1013e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 0x7f101419a3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f100002c150 tx=0x7f10000042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.731+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000041020 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.731+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141a1d60 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.731+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10000035f0 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.731+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000039040 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.731+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141a21f0 con 0x7f1014075ae0 2024-04-02T17:12:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.732+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1014108650 con 0x7f1014075ae0 2024-04-02T17:12:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.734+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1000044020 con 0x7f1014075ae0 2024-04-02T17:12:58.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.736+0000 7f1011600640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 0x7f0ffc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:58.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.736+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f10000f1c30 con 0x7f1014075ae0 2024-04-02T17:12:58.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.736+0000 7f1018a00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 0x7f0ffc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:58.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.737+0000 7f1018a00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 0x7f0ffc0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10080058c0 tx=0x7f1008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.739+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f100004b020 con 0x7f1014075ae0 2024-04-02T17:12:58.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.938+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1014062730 con 0x7f1014075ae0 2024-04-02T17:12:58.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.939+0000 7f1011600640 1 -- 172.21.15.59:0/643178950 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f10000be550 con 0x7f1014075ae0 2024-04-02T17:12:58.939 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:12:58.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 msgr2=0x7f0ffc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 0x7f0ffc0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10080058c0 tx=0x7f1008009260 comp rx=0 tx=0).stop 2024-04-02T17:12:58.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 msgr2=0x7f101419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 0x7f101419a3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f100002c150 tx=0x7f10000042f0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 shutdown_connections 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1014109890 0x7f10141a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1014075ae0 0x7f101419a3f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0ffc0a3bb0 0x7f0ffc0a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 --2- 172.21.15.59:0/643178950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014074500 0x7f1014199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.941+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 >> 172.21.15.59:0/643178950 conn(0x7f10140fcf70 msgr2=0x7f1014105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:58.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.942+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 shutdown_connections 2024-04-02T17:12:58.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:58.942+0000 7f1019e00640 1 -- 172.21.15.59:0/643178950 wait complete. 2024-04-02T17:12:58.960 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-02T17:12:58.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:12:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 -- 172.21.15.59:0/801144799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 msgr2=0x7fb9d01049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 --2- 172.21.15.59:0/801144799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d01049b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009900 tx=0x7fb9c402bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 -- 172.21.15.59:0/801144799 shutdown_connections 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 --2- 172.21.15.59:0/801144799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 0x7fb9d01072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 --2- 172.21.15.59:0/801144799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 0x7fb9d0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 --2- 172.21.15.59:0/801144799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d01049b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 -- 172.21.15.59:0/801144799 >> 172.21.15.59:0/801144799 conn(0x7fb9d00fbfb0 msgr2=0x7fb9d00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 -- 172.21.15.59:0/801144799 shutdown_connections 2024-04-02T17:12:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.168+0000 7fb9d7800640 1 -- 172.21.15.59:0/801144799 wait complete. 2024-04-02T17:12:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.169+0000 7fb9d7800640 1 Processor -- start 2024-04-02T17:12:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.169+0000 7fb9d7800640 1 -- start start 2024-04-02T17:12:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 0x7fb9d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 0x7fb9d01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d010d7c0 con 0x7fb9d0104ef0 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d010d640 con 0x7fb9d01025c0 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d7800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d010d940 con 0x7fb9d0107820 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 0x7fb9d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55736/0 (socket says 172.21.15.59:55736) 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 -- 172.21.15.59:0/944115860 learned_addr learned my addr 172.21.15.59:0/944115860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d5800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 0x7fb9d01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 -- 172.21.15.59:0/944115860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 msgr2=0x7fb9d01a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 0x7fb9d01a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 -- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 msgr2=0x7fb9d019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 0x7fb9d019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.170+0000 7fb9d4e00640 1 -- 172.21.15.59:0/944115860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d01a1d00 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9d4e00640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009230 tx=0x7fb9c402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4042020 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c4004450 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4003db0 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d01a1f90 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.171+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d01a23a0 con 0x7fb9d01025c0 2024-04-02T17:12:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.172+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d0109370 con 0x7fb9d01025c0 2024-04-02T17:12:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.174+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9c4045020 con 0x7fb9d01025c0 2024-04-02T17:12:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.175+0000 7fb9cd600640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 0x7fb9b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.175+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb9c40f24c0 con 0x7fb9d01025c0 2024-04-02T17:12:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.176+0000 7fb9cfe00640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 0x7fb9b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.176+0000 7fb9cfe00640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 0x7fb9b80a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0095f0 tx=0x7fb9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.179+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9c404c020 con 0x7fb9d01025c0 2024-04-02T17:12:59.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.481+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9d002d370 con 0x7fb9d01025c0 2024-04-02T17:12:59.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.482+0000 7fb9cd600640 1 -- 172.21.15.59:0/944115860 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7fb9c40bede0 con 0x7fb9d01025c0 2024-04-02T17:12:59.483 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:59.483 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:12:54.846637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24529,24530,24547,24560,24584]},"id":1}]} 2024-04-02T17:12:59.483 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 msgr2=0x7fb9b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 0x7fb9b80a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0095f0 tx=0x7fb9bc005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 msgr2=0x7fb9d019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009230 tx=0x7fb9c402c670 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 shutdown_connections 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9b80a3bb0 0x7fb9b80a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9d0104ef0 0x7fb9d019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9d0107820 0x7fb9d01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 --2- 172.21.15.59:0/944115860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9d01025c0 0x7fb9d019a0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 >> 172.21.15.59:0/944115860 conn(0x7fb9d00fbfb0 msgr2=0x7fb9d0101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 shutdown_connections 2024-04-02T17:12:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.486+0000 7fb9d7800640 1 -- 172.21.15.59:0/944115860 wait complete. 2024-04-02T17:12:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:59 smithi059 ceph-mon[31664]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:12:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:59 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1562331101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:12:59 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/643178950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.503 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-02T17:12:59.503 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-02T17:12:59.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T17:12:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:59 smithi164 ceph-mon[36499]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:12:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:59 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1562331101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:12:59 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/643178950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:59 smithi154 ceph-mon[36479]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T17:12:59.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:59 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1562331101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:12:59.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:12:59 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/643178950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:12:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 -- 172.21.15.59:0/178862309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c1025c0 msgr2=0x7f982c1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 --2- 172.21.15.59:0/178862309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c1025c0 0x7f982c1049b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f981c009900 tx=0x7f981c02bc50 comp rx=0 tx=0).stop 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 -- 172.21.15.59:0/178862309 shutdown_connections 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 --2- 172.21.15.59:0/178862309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 --2- 172.21.15.59:0/178862309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c104ef0 0x7f982c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 --2- 172.21.15.59:0/178862309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c1025c0 0x7f982c1049b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 -- 172.21.15.59:0/178862309 >> 172.21.15.59:0/178862309 conn(0x7f982c0fbfb0 msgr2=0x7f982c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 -- 172.21.15.59:0/178862309 shutdown_connections 2024-04-02T17:12:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.696+0000 7f9832e00640 1 -- 172.21.15.59:0/178862309 wait complete. 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.697+0000 7f9832e00640 1 Processor -- start 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.697+0000 7f9832e00640 1 -- start start 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c104ef0 0x7f982c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d7c0 con 0x7f982c107820 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d640 con 0x7f982c104ef0 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9832e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f982c10d940 con 0x7f982c1025c0 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f982be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f982be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50566/0 (socket says 172.21.15.59:50566) 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f982be00640 1 -- 172.21.15.59:0/1809181918 learned_addr learned my addr 172.21.15.59:0/1809181918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f982b400640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c104ef0 0x7f982c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.698+0000 7f9830e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 -- 172.21.15.59:0/1809181918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c104ef0 msgr2=0x7f982c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c104ef0 0x7f982c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 -- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 msgr2=0x7f982c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 -- 172.21.15.59:0/1809181918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c19fb50 con 0x7f982c1025c0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f9830e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f982be00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f981c009230 tx=0x7f981c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c042020 con 0x7f982c1025c0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c004450 con 0x7f982c1025c0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c003e20 con 0x7f982c1025c0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.699+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c19fde0 con 0x7f982c1025c0 2024-04-02T17:12:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.700+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1a0190 con 0x7f982c1025c0 2024-04-02T17:12:59.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.701+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0fdaa0 con 0x7f982c1025c0 2024-04-02T17:12:59.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.702+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f981c045020 con 0x7f982c1025c0 2024-04-02T17:12:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.704+0000 7f9828c00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 0x7f98100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:12:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.704+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f981c0f24d0 con 0x7f982c1025c0 2024-04-02T17:12:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.704+0000 7f982b400640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 0x7f98100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:12:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.705+0000 7f982b400640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 0x7f98100a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98180095f0 tx=0x7f9818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:12:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.708+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f981c04c020 con 0x7f982c1025c0 2024-04-02T17:12:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.923+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f982c061f40 con 0x7f98100a3c30 2024-04-02T17:12:59.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.948+0000 7f9828c00640 1 -- 172.21.15.59:0/1809181918 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7f982c061f40 con 0x7f98100a3c30 2024-04-02T17:12:59.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:12:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:12:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 msgr2=0x7f98100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 0x7f98100a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98180095f0 tx=0x7f9818005f90 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 msgr2=0x7f982c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f981c009230 tx=0x7f981c02c670 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 shutdown_connections 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f98100a3c30 0x7f98100a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f982c107820 0x7f982c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f982c104ef0 0x7f982c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 --2- 172.21.15.59:0/1809181918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c1025c0 0x7f982c197f30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 >> 172.21.15.59:0/1809181918 conn(0x7f982c0fbfb0 msgr2=0x7f982c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 shutdown_connections 2024-04-02T17:12:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:12:59.955+0000 7f9832e00640 1 -- 172.21.15.59:0/1809181918 wait complete. 2024-04-02T17:12:59.975 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-02T17:12:59.549779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666750,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1509,"num_read_kb":886,"num_write":441,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1180943,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388248,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553576,"statfs":{"total":1151940427776,"available":1151542861824,"internally_reserved":0,"allocated":14434304,"data_stored":7803741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004618"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818152+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:11:52.818152+0000","last_peered":"2024-04-02T17:11:52.818152+0000","last_clean":"2024-04-02T17:11:52.818152+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:11:52.818152+0000","last_undegraded":"2024-04-02T17:11:52.818152+0000","last_fullsized":"2024-04-02T17:11:52.818152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818026+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:11:52.818026+0000","last_peered":"2024-04-02T17:11:52.818026+0000","last_clean":"2024-04-02T17:11:52.818026+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:11:52.818026+0000","last_undegraded":"2024-04-02T17:11:52.818026+0000","last_fullsized":"2024-04-02T17:11:52.818026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222628+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:11:52.222628+0000","last_peered":"2024-04-02T17:11:52.222628+0000","last_clean":"2024-04-02T17:11:52.222628+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:11:52.222628+0000","last_undegraded":"2024-04-02T17:11:52.222628+0000","last_fullsized":"2024-04-02T17:11:52.222628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430624+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:11:52.430624+0000","last_peered":"2024-04-02T17:11:52.430624+0000","last_clean":"2024-04-02T17:11:52.430624+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:11:52.430624+0000","last_undegraded":"2024-04-02T17:11:52.430624+0000","last_fullsized":"2024-04-02T17:11:52.430624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225467+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:11:52.225467+0000","last_peered":"2024-04-02T17:11:52.225467+0000","last_clean":"2024-04-02T17:11:52.225467+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:11:52.225467+0000","last_undegraded":"2024-04-02T17:11:52.225467+0000","last_fullsized":"2024-04-02T17:11:52.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.432056+0000","last_change":"2024-04-02T17:11:52.432056+0000","last_active":"2024-04-02T17:11:52.432056+0000","last_peered":"2024-04-02T17:11:52.432056+0000","last_clean":"2024-04-02T17:11:52.432056+0000","last_became_active":"2024-04-02T17:11:52.431692+0000","last_became_peered":"2024-04-02T17:11:52.431692+0000","last_unstale":"2024-04-02T17:11:52.432056+0000","last_undegraded":"2024-04-02T17:11:52.432056+0000","last_fullsized":"2024-04-02T17:11:52.432056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:51.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230343+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:11:52.230343+0000","last_peered":"2024-04-02T17:11:52.230343+0000","last_clean":"2024-04-02T17:11:52.230343+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:11:52.230343+0000","last_undegraded":"2024-04-02T17:11:52.230343+0000","last_fullsized":"2024-04-02T17:11:52.230343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430135+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:11:52.430135+0000","last_peered":"2024-04-02T17:11:52.430135+0000","last_clean":"2024-04-02T17:11:52.430135+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:11:52.430135+0000","last_undegraded":"2024-04-02T17:11:52.430135+0000","last_fullsized":"2024-04-02T17:11:52.430135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225843+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:11:52.225843+0000","last_peered":"2024-04-02T17:11:52.225843+0000","last_clean":"2024-04-02T17:11:52.225843+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:11:52.225843+0000","last_undegraded":"2024-04-02T17:11:52.225843+0000","last_fullsized":"2024-04-02T17:11:52.225843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224298+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:11:52.224298+0000","last_peered":"2024-04-02T17:11:52.224298+0000","last_clean":"2024-04-02T17:11:52.224298+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:11:52.224298+0000","last_undegraded":"2024-04-02T17:11:52.224298+0000","last_fullsized":"2024-04-02T17:11:52.224298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222208+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:11:52.222208+0000","last_peered":"2024-04-02T17:11:52.222208+0000","last_clean":"2024-04-02T17:11:52.222208+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:11:52.222208+0000","last_undegraded":"2024-04-02T17:11:52.222208+0000","last_fullsized":"2024-04-02T17:11:52.222208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236269+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:11:52.236269+0000","last_peered":"2024-04-02T17:11:52.236269+0000","last_clean":"2024-04-02T17:11:52.236269+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:11:52.236269+0000","last_undegraded":"2024-04-02T17:11:52.236269+0000","last_fullsized":"2024-04-02T17:11:52.236269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818685+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:11:52.818685+0000","last_peered":"2024-04-02T17:11:52.818685+0000","last_clean":"2024-04-02T17:11:52.818685+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:11:52.818685+0000","last_undegraded":"2024-04-02T17:11:52.818685+0000","last_fullsized":"2024-04-02T17:11:52.818685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818544+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:11:52.818544+0000","last_peered":"2024-04-02T17:11:52.818544+0000","last_clean":"2024-04-02T17:11:52.818544+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:11:52.818544+0000","last_undegraded":"2024-04-02T17:11:52.818544+0000","last_fullsized":"2024-04-02T17:11:52.818544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429577+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:11:52.429577+0000","last_peered":"2024-04-02T17:11:52.429577+0000","last_clean":"2024-04-02T17:11:52.429577+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:11:52.429577+0000","last_undegraded":"2024-04-02T17:11:52.429577+0000","last_fullsized":"2024-04-02T17:11:52.429577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430790+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:11:52.430790+0000","last_peered":"2024-04-02T17:11:52.430790+0000","last_clean":"2024-04-02T17:11:52.430790+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:11:52.430790+0000","last_undegraded":"2024-04-02T17:11:52.430790+0000","last_fullsized":"2024-04-02T17:11:52.430790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222683+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:11:52.222683+0000","last_peered":"2024-04-02T17:11:52.222683+0000","last_clean":"2024-04-02T17:11:52.222683+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:11:52.222683+0000","last_undegraded":"2024-04-02T17:11:52.222683+0000","last_fullsized":"2024-04-02T17:11:52.222683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219697+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:11:52.219697+0000","last_peered":"2024-04-02T17:11:52.219697+0000","last_clean":"2024-04-02T17:11:52.219697+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:11:52.219697+0000","last_undegraded":"2024-04-02T17:11:52.219697+0000","last_fullsized":"2024-04-02T17:11:52.219697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.383706+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:12:55.383706+0000","last_peered":"2024-04-02T17:12:55.383706+0000","last_clean":"2024-04-02T17:12:55.383706+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:12:55.383706+0000","last_undegraded":"2024-04-02T17:12:55.383706+0000","last_fullsized":"2024-04-02T17:12:55.383706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232279+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:11:52.232279+0000","last_peered":"2024-04-02T17:11:52.232279+0000","last_clean":"2024-04-02T17:11:52.232279+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:11:52.232279+0000","last_undegraded":"2024-04-02T17:11:52.232279+0000","last_fullsized":"2024-04-02T17:11:52.232279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232788+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:11:52.232788+0000","last_peered":"2024-04-02T17:11:52.232788+0000","last_clean":"2024-04-02T17:11:52.232788+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:11:52.232788+0000","last_undegraded":"2024-04-02T17:11:52.232788+0000","last_fullsized":"2024-04-02T17:11:52.232788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818995+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:11:52.818995+0000","last_peered":"2024-04-02T17:11:52.818995+0000","last_clean":"2024-04-02T17:11:52.818995+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:11:52.818995+0000","last_undegraded":"2024-04-02T17:11:52.818995+0000","last_fullsized":"2024-04-02T17:11:52.818995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430291+0000","last_change":"2024-04-02T17:10:51.330798+0000","last_active":"2024-04-02T17:11:52.430291+0000","last_peered":"2024-04-02T17:11:52.430291+0000","last_clean":"2024-04-02T17:11:52.430291+0000","last_became_active":"2024-04-02T17:10:51.318708+0000","last_became_peered":"2024-04-02T17:10:51.318708+0000","last_unstale":"2024-04-02T17:11:52.430291+0000","last_undegraded":"2024-04-02T17:11:52.430291+0000","last_fullsized":"2024-04-02T17:11:52.430291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:08:25.035375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429776+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:11:52.429776+0000","last_peered":"2024-04-02T17:11:52.429776+0000","last_clean":"2024-04-02T17:11:52.429776+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:11:52.429776+0000","last_undegraded":"2024-04-02T17:11:52.429776+0000","last_fullsized":"2024-04-02T17:11:52.429776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430934+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:11:52.430934+0000","last_peered":"2024-04-02T17:11:52.430934+0000","last_clean":"2024-04-02T17:11:52.430934+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:11:52.430934+0000","last_undegraded":"2024-04-02T17:11:52.430934+0000","last_fullsized":"2024-04-02T17:11:52.430934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221978+0000","last_change":"2024-04-02T17:10:51.634458+0000","last_active":"2024-04-02T17:11:52.221978+0000","last_peered":"2024-04-02T17:11:52.221978+0000","last_clean":"2024-04-02T17:11:52.221978+0000","last_became_active":"2024-04-02T17:10:51.634064+0000","last_became_peered":"2024-04-02T17:10:51.634064+0000","last_unstale":"2024-04-02T17:11:52.221978+0000","last_undegraded":"2024-04-02T17:11:52.221978+0000","last_fullsized":"2024-04-02T17:11:52.221978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:10.010493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:54.296307+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:12:54.296307+0000","last_peered":"2024-04-02T17:12:54.296307+0000","last_clean":"2024-04-02T17:12:54.296307+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:12:54.296307+0000","last_undegraded":"2024-04-02T17:12:54.296307+0000","last_fullsized":"2024-04-02T17:12:54.296307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430521+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:11:52.430521+0000","last_peered":"2024-04-02T17:11:52.430521+0000","last_clean":"2024-04-02T17:11:52.430521+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:11:52.430521+0000","last_undegraded":"2024-04-02T17:11:52.430521+0000","last_fullsized":"2024-04-02T17:11:52.430521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225449+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:11:52.225449+0000","last_peered":"2024-04-02T17:11:52.225449+0000","last_clean":"2024-04-02T17:11:52.225449+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:11:52.225449+0000","last_undegraded":"2024-04-02T17:11:52.225449+0000","last_fullsized":"2024-04-02T17:11:52.225449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221833+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:11:52.221833+0000","last_peered":"2024-04-02T17:11:52.221833+0000","last_clean":"2024-04-02T17:11:52.221833+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:11:52.221833+0000","last_undegraded":"2024-04-02T17:11:52.221833+0000","last_fullsized":"2024-04-02T17:11:52.221833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235557+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:11:52.235557+0000","last_peered":"2024-04-02T17:11:52.235557+0000","last_clean":"2024-04-02T17:11:52.235557+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:11:52.235557+0000","last_undegraded":"2024-04-02T17:11:52.235557+0000","last_fullsized":"2024-04-02T17:11:52.235557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230277+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:11:52.230277+0000","last_peered":"2024-04-02T17:11:52.230277+0000","last_clean":"2024-04-02T17:11:52.230277+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:11:52.230277+0000","last_undegraded":"2024-04-02T17:11:52.230277+0000","last_fullsized":"2024-04-02T17:11:52.230277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221584+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:11:52.221584+0000","last_peered":"2024-04-02T17:11:52.221584+0000","last_clean":"2024-04-02T17:11:52.221584+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:11:52.221584+0000","last_undegraded":"2024-04-02T17:11:52.221584+0000","last_fullsized":"2024-04-02T17:11:52.221584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221849+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:11:52.221849+0000","last_peered":"2024-04-02T17:11:52.221849+0000","last_clean":"2024-04-02T17:11:52.221849+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:11:52.221849+0000","last_undegraded":"2024-04-02T17:11:52.221849+0000","last_fullsized":"2024-04-02T17:11:52.221849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.299066+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:12:57.299066+0000","last_peered":"2024-04-02T17:12:57.299066+0000","last_clean":"2024-04-02T17:12:57.299066+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:12:57.299066+0000","last_undegraded":"2024-04-02T17:12:57.299066+0000","last_fullsized":"2024-04-02T17:12:57.299066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430074+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:11:52.430074+0000","last_peered":"2024-04-02T17:11:52.430074+0000","last_clean":"2024-04-02T17:11:52.430074+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:11:52.430074+0000","last_undegraded":"2024-04-02T17:11:52.430074+0000","last_fullsized":"2024-04-02T17:11:52.430074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224087+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:11:52.224087+0000","last_peered":"2024-04-02T17:11:52.224087+0000","last_clean":"2024-04-02T17:11:52.224087+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:11:52.224087+0000","last_undegraded":"2024-04-02T17:11:52.224087+0000","last_fullsized":"2024-04-02T17:11:52.224087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819045+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:11:52.819045+0000","last_peered":"2024-04-02T17:11:52.819045+0000","last_clean":"2024-04-02T17:11:52.819045+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:11:52.819045+0000","last_undegraded":"2024-04-02T17:11:52.819045+0000","last_fullsized":"2024-04-02T17:11:52.819045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222125+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:11:52.222125+0000","last_peered":"2024-04-02T17:11:52.222125+0000","last_clean":"2024-04-02T17:11:52.222125+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:11:52.222125+0000","last_undegraded":"2024-04-02T17:11:52.222125+0000","last_fullsized":"2024-04-02T17:11:52.222125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429946+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:11:52.429946+0000","last_peered":"2024-04-02T17:11:52.429946+0000","last_clean":"2024-04-02T17:11:52.429946+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:11:52.429946+0000","last_undegraded":"2024-04-02T17:11:52.429946+0000","last_fullsized":"2024-04-02T17:11:52.429946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":340,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.300323+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:12:57.300323+0000","last_peered":"2024-04-02T17:12:57.300323+0000","last_clean":"2024-04-02T17:12:57.300323+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:12:57.300323+0000","last_undegraded":"2024-04-02T17:12:57.300323+0000","last_fullsized":"2024-04-02T17:12:57.300323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67824,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":207,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430928+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:11:52.430928+0000","last_peered":"2024-04-02T17:11:52.430928+0000","last_clean":"2024-04-02T17:11:52.430928+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:11:52.430928+0000","last_undegraded":"2024-04-02T17:11:52.430928+0000","last_fullsized":"2024-04-02T17:11:52.430928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219640+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:11:52.219640+0000","last_peered":"2024-04-02T17:11:52.219640+0000","last_clean":"2024-04-02T17:11:52.219640+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:11:52.219640+0000","last_undegraded":"2024-04-02T17:11:52.219640+0000","last_fullsized":"2024-04-02T17:11:52.219640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235862+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:11:52.235862+0000","last_peered":"2024-04-02T17:11:52.235862+0000","last_clean":"2024-04-02T17:11:52.235862+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:11:52.235862+0000","last_undegraded":"2024-04-02T17:11:52.235862+0000","last_fullsized":"2024-04-02T17:11:52.235862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430181+0000","last_change":"2024-04-02T17:10:51.336678+0000","last_active":"2024-04-02T17:11:52.430181+0000","last_peered":"2024-04-02T17:11:52.430181+0000","last_clean":"2024-04-02T17:11:52.430181+0000","last_became_active":"2024-04-02T17:10:51.325028+0000","last_became_peered":"2024-04-02T17:10:51.325028+0000","last_unstale":"2024-04-02T17:11:52.430181+0000","last_undegraded":"2024-04-02T17:11:52.430181+0000","last_fullsized":"2024-04-02T17:11:52.430181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:54:29.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224346+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:11:52.224346+0000","last_peered":"2024-04-02T17:11:52.224346+0000","last_clean":"2024-04-02T17:11:52.224346+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:11:52.224346+0000","last_undegraded":"2024-04-02T17:11:52.224346+0000","last_fullsized":"2024-04-02T17:11:52.224346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.399180+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:12:55.399180+0000","last_peered":"2024-04-02T17:12:55.399180+0000","last_clean":"2024-04-02T17:12:55.399180+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:12:55.399180+0000","last_undegraded":"2024-04-02T17:12:55.399180+0000","last_fullsized":"2024-04-02T17:12:55.399180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223641+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:11:52.223641+0000","last_peered":"2024-04-02T17:11:52.223641+0000","last_clean":"2024-04-02T17:11:52.223641+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:11:52.223641+0000","last_undegraded":"2024-04-02T17:11:52.223641+0000","last_fullsized":"2024-04-02T17:11:52.223641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430477+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:11:52.430477+0000","last_peered":"2024-04-02T17:11:52.430477+0000","last_clean":"2024-04-02T17:11:52.430477+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:11:52.430477+0000","last_undegraded":"2024-04-02T17:11:52.430477+0000","last_fullsized":"2024-04-02T17:11:52.430477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222202+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:11:52.222202+0000","last_peered":"2024-04-02T17:11:52.222202+0000","last_clean":"2024-04-02T17:11:52.222202+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:11:52.222202+0000","last_undegraded":"2024-04-02T17:11:52.222202+0000","last_fullsized":"2024-04-02T17:11:52.222202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431210+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:11:52.431210+0000","last_peered":"2024-04-02T17:11:52.431210+0000","last_clean":"2024-04-02T17:11:52.431210+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:11:52.431210+0000","last_undegraded":"2024-04-02T17:11:52.431210+0000","last_fullsized":"2024-04-02T17:11:52.431210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430525+0000","last_change":"2024-04-02T17:10:51.321193+0000","last_active":"2024-04-02T17:11:52.430525+0000","last_peered":"2024-04-02T17:11:52.430525+0000","last_clean":"2024-04-02T17:11:52.430525+0000","last_became_active":"2024-04-02T17:10:51.320791+0000","last_became_peered":"2024-04-02T17:10:51.320791+0000","last_unstale":"2024-04-02T17:11:52.430525+0000","last_undegraded":"2024-04-02T17:11:52.430525+0000","last_fullsized":"2024-04-02T17:11:52.430525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:22:18.979764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222639+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:11:52.222639+0000","last_peered":"2024-04-02T17:11:52.222639+0000","last_clean":"2024-04-02T17:11:52.222639+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:11:52.222639+0000","last_undegraded":"2024-04-02T17:11:52.222639+0000","last_fullsized":"2024-04-02T17:11:52.222639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.243589+0000","last_change":"2024-04-02T17:11:52.243152+0000","last_active":"2024-04-02T17:11:52.243589+0000","last_peered":"2024-04-02T17:11:52.243589+0000","last_clean":"2024-04-02T17:11:52.243589+0000","last_became_active":"2024-04-02T17:11:52.232947+0000","last_became_peered":"2024-04-02T17:11:52.232947+0000","last_unstale":"2024-04-02T17:11:52.243589+0000","last_undegraded":"2024-04-02T17:11:52.243589+0000","last_fullsized":"2024-04-02T17:11:52.243589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:49:58.858519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:56.386744+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:12:56.386744+0000","last_peered":"2024-04-02T17:12:56.386744+0000","last_clean":"2024-04-02T17:12:56.386744+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:12:56.386744+0000","last_undegraded":"2024-04-02T17:12:56.386744+0000","last_fullsized":"2024-04-02T17:12:56.386744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225702+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:11:52.225702+0000","last_peered":"2024-04-02T17:11:52.225702+0000","last_clean":"2024-04-02T17:11:52.225702+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:11:52.225702+0000","last_undegraded":"2024-04-02T17:11:52.225702+0000","last_fullsized":"2024-04-02T17:11:52.225702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221789+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:11:52.221789+0000","last_peered":"2024-04-02T17:11:52.221789+0000","last_clean":"2024-04-02T17:11:52.221789+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:11:52.221789+0000","last_undegraded":"2024-04-02T17:11:52.221789+0000","last_fullsized":"2024-04-02T17:11:52.221789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.402458+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:12:57.402458+0000","last_peered":"2024-04-02T17:12:57.402458+0000","last_clean":"2024-04-02T17:12:57.402458+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:12:57.402458+0000","last_undegraded":"2024-04-02T17:12:57.402458+0000","last_fullsized":"2024-04-02T17:12:57.402458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222174+0000","last_change":"2024-04-02T17:10:51.633808+0000","last_active":"2024-04-02T17:11:52.222174+0000","last_peered":"2024-04-02T17:11:52.222174+0000","last_clean":"2024-04-02T17:11:52.222174+0000","last_became_active":"2024-04-02T17:10:51.633095+0000","last_became_peered":"2024-04-02T17:10:51.633095+0000","last_unstale":"2024-04-02T17:11:52.222174+0000","last_undegraded":"2024-04-02T17:11:52.222174+0000","last_fullsized":"2024-04-02T17:11:52.222174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:36:16.308694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222251+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:11:52.222251+0000","last_peered":"2024-04-02T17:11:52.222251+0000","last_clean":"2024-04-02T17:11:52.222251+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:11:52.222251+0000","last_undegraded":"2024-04-02T17:11:52.222251+0000","last_fullsized":"2024-04-02T17:11:52.222251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223736+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:11:52.223736+0000","last_peered":"2024-04-02T17:11:52.223736+0000","last_clean":"2024-04-02T17:11:52.223736+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:11:52.223736+0000","last_undegraded":"2024-04-02T17:11:52.223736+0000","last_fullsized":"2024-04-02T17:11:52.223736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235203+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:11:52.235203+0000","last_peered":"2024-04-02T17:11:52.235203+0000","last_clean":"2024-04-02T17:11:52.235203+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:11:52.235203+0000","last_undegraded":"2024-04-02T17:11:52.235203+0000","last_fullsized":"2024-04-02T17:11:52.235203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223856+0000","last_change":"2024-04-02T17:10:51.637846+0000","last_active":"2024-04-02T17:11:52.223856+0000","last_peered":"2024-04-02T17:11:52.223856+0000","last_clean":"2024-04-02T17:11:52.223856+0000","last_became_active":"2024-04-02T17:10:51.633829+0000","last_became_peered":"2024-04-02T17:10:51.633829+0000","last_unstale":"2024-04-02T17:11:52.223856+0000","last_undegraded":"2024-04-02T17:11:52.223856+0000","last_fullsized":"2024-04-02T17:11:52.223856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:10:45.177246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431026+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:11:52.431026+0000","last_peered":"2024-04-02T17:11:52.431026+0000","last_clean":"2024-04-02T17:11:52.431026+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:11:52.431026+0000","last_undegraded":"2024-04-02T17:11:52.431026+0000","last_fullsized":"2024-04-02T17:11:52.431026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237144+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:11:52.237144+0000","last_peered":"2024-04-02T17:11:52.237144+0000","last_clean":"2024-04-02T17:11:52.237144+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:11:52.237144+0000","last_undegraded":"2024-04-02T17:11:52.237144+0000","last_fullsized":"2024-04-02T17:11:52.237144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:47.323048+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:12:47.323048+0000","last_peered":"2024-04-02T17:12:47.323048+0000","last_clean":"2024-04-02T17:12:47.323048+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:12:47.323048+0000","last_undegraded":"2024-04-02T17:12:47.323048+0000","last_fullsized":"2024-04-02T17:12:47.323048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.233095+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:11:52.233095+0000","last_peered":"2024-04-02T17:11:52.233095+0000","last_clean":"2024-04-02T17:11:52.233095+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:11:52.233095+0000","last_undegraded":"2024-04-02T17:11:52.233095+0000","last_fullsized":"2024-04-02T17:11:52.233095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222419+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:11:52.222419+0000","last_peered":"2024-04-02T17:11:52.222419+0000","last_clean":"2024-04-02T17:11:52.222419+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:11:52.222419+0000","last_undegraded":"2024-04-02T17:11:52.222419+0000","last_fullsized":"2024-04-02T17:11:52.222419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818278+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:11:52.818278+0000","last_peered":"2024-04-02T17:11:52.818278+0000","last_clean":"2024-04-02T17:11:52.818278+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:11:52.818278+0000","last_undegraded":"2024-04-02T17:11:52.818278+0000","last_fullsized":"2024-04-02T17:11:52.818278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429640+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:11:52.429640+0000","last_peered":"2024-04-02T17:11:52.429640+0000","last_clean":"2024-04-02T17:11:52.429640+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:11:52.429640+0000","last_undegraded":"2024-04-02T17:11:52.429640+0000","last_fullsized":"2024-04-02T17:11:52.429640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430857+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:11:52.430857+0000","last_peered":"2024-04-02T17:11:52.430857+0000","last_clean":"2024-04-02T17:11:52.430857+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:11:52.430857+0000","last_undegraded":"2024-04-02T17:11:52.430857+0000","last_fullsized":"2024-04-02T17:11:52.430857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230823+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:11:52.230823+0000","last_peered":"2024-04-02T17:11:52.230823+0000","last_clean":"2024-04-02T17:11:52.230823+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:11:52.230823+0000","last_undegraded":"2024-04-02T17:11:52.230823+0000","last_fullsized":"2024-04-02T17:11:52.230823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430110+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:11:52.430110+0000","last_peered":"2024-04-02T17:11:52.430110+0000","last_clean":"2024-04-02T17:11:52.430110+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:11:52.430110+0000","last_undegraded":"2024-04-02T17:11:52.430110+0000","last_fullsized":"2024-04-02T17:11:52.430110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219529+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:11:52.219529+0000","last_peered":"2024-04-02T17:11:52.219529+0000","last_clean":"2024-04-02T17:11:52.219529+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:11:52.219529+0000","last_undegraded":"2024-04-02T17:11:52.219529+0000","last_fullsized":"2024-04-02T17:11:52.219529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232232+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:11:52.232232+0000","last_peered":"2024-04-02T17:11:52.232232+0000","last_clean":"2024-04-02T17:11:52.232232+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:11:52.232232+0000","last_undegraded":"2024-04-02T17:11:52.232232+0000","last_fullsized":"2024-04-02T17:11:52.232232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:51.641235+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:51.634715+0000","last_became_peered":"2024-04-02T17:10:51.634715+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:56:38.764506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224004+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:11:52.224004+0000","last_peered":"2024-04-02T17:11:52.224004+0000","last_clean":"2024-04-02T17:11:52.224004+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:11:52.224004+0000","last_undegraded":"2024-04-02T17:11:52.224004+0000","last_fullsized":"2024-04-02T17:11:52.224004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:53.293909+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:12:53.293909+0000","last_peered":"2024-04-02T17:12:53.293909+0000","last_clean":"2024-04-02T17:12:53.293909+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:12:53.293909+0000","last_undegraded":"2024-04-02T17:12:53.293909+0000","last_fullsized":"2024-04-02T17:12:53.293909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224498+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:11:52.224498+0000","last_peered":"2024-04-02T17:11:52.224498+0000","last_clean":"2024-04-02T17:11:52.224498+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:11:52.224498+0000","last_undegraded":"2024-04-02T17:11:52.224498+0000","last_fullsized":"2024-04-02T17:11:52.224498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221897+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:11:52.221897+0000","last_peered":"2024-04-02T17:11:52.221897+0000","last_clean":"2024-04-02T17:11:52.221897+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:11:52.221897+0000","last_undegraded":"2024-04-02T17:11:52.221897+0000","last_fullsized":"2024-04-02T17:11:52.221897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224105+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:11:52.224105+0000","last_peered":"2024-04-02T17:11:52.224105+0000","last_clean":"2024-04-02T17:11:52.224105+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:11:52.224105+0000","last_undegraded":"2024-04-02T17:11:52.224105+0000","last_fullsized":"2024-04-02T17:11:52.224105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236455+0000","last_change":"2024-04-02T17:10:51.642752+0000","last_active":"2024-04-02T17:11:52.236455+0000","last_peered":"2024-04-02T17:11:52.236455+0000","last_clean":"2024-04-02T17:11:52.236455+0000","last_became_active":"2024-04-02T17:10:51.635734+0000","last_became_peered":"2024-04-02T17:10:51.635734+0000","last_unstale":"2024-04-02T17:11:52.236455+0000","last_undegraded":"2024-04-02T17:11:52.236455+0000","last_fullsized":"2024-04-02T17:11:52.236455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.358529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224156+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:11:52.224156+0000","last_peered":"2024-04-02T17:11:52.224156+0000","last_clean":"2024-04-02T17:11:52.224156+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:11:52.224156+0000","last_undegraded":"2024-04-02T17:11:52.224156+0000","last_fullsized":"2024-04-02T17:11:52.224156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430173+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:11:52.430173+0000","last_peered":"2024-04-02T17:11:52.430173+0000","last_clean":"2024-04-02T17:11:52.430173+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:11:52.430173+0000","last_undegraded":"2024-04-02T17:11:52.430173+0000","last_fullsized":"2024-04-02T17:11:52.430173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237295+0000","last_change":"2024-04-02T17:11:52.236694+0000","last_active":"2024-04-02T17:11:52.237295+0000","last_peered":"2024-04-02T17:11:52.237295+0000","last_clean":"2024-04-02T17:11:52.237295+0000","last_became_active":"2024-04-02T17:11:52.236128+0000","last_became_peered":"2024-04-02T17:11:52.236128+0000","last_unstale":"2024-04-02T17:11:52.237295+0000","last_undegraded":"2024-04-02T17:11:52.237295+0000","last_fullsized":"2024-04-02T17:11:52.237295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:40.601150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219390+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:11:52.219390+0000","last_peered":"2024-04-02T17:11:52.219390+0000","last_clean":"2024-04-02T17:11:52.219390+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:11:52.219390+0000","last_undegraded":"2024-04-02T17:11:52.219390+0000","last_fullsized":"2024-04-02T17:11:52.219390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819694+0000","last_change":"2024-04-02T17:10:51.332185+0000","last_active":"2024-04-02T17:11:52.819694+0000","last_peered":"2024-04-02T17:11:52.819694+0000","last_clean":"2024-04-02T17:11:52.819694+0000","last_became_active":"2024-04-02T17:10:51.331547+0000","last_became_peered":"2024-04-02T17:10:51.331547+0000","last_unstale":"2024-04-02T17:11:52.819694+0000","last_undegraded":"2024-04-02T17:11:52.819694+0000","last_fullsized":"2024-04-02T17:11:52.819694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:43.766907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236916+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:11:52.236916+0000","last_peered":"2024-04-02T17:11:52.236916+0000","last_clean":"2024-04-02T17:11:52.236916+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:11:52.236916+0000","last_undegraded":"2024-04-02T17:11:52.236916+0000","last_fullsized":"2024-04-02T17:11:52.236916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430480+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:11:52.430480+0000","last_peered":"2024-04-02T17:11:52.430480+0000","last_clean":"2024-04-02T17:11:52.430480+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:11:52.430480+0000","last_undegraded":"2024-04-02T17:11:52.430480+0000","last_fullsized":"2024-04-02T17:11:52.430480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.220006+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:11:52.220006+0000","last_peered":"2024-04-02T17:11:52.220006+0000","last_clean":"2024-04-02T17:11:52.220006+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:11:52.220006+0000","last_undegraded":"2024-04-02T17:11:52.220006+0000","last_fullsized":"2024-04-02T17:11:52.220006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232637+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:11:52.232637+0000","last_peered":"2024-04-02T17:11:52.232637+0000","last_clean":"2024-04-02T17:11:52.232637+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:11:52.232637+0000","last_undegraded":"2024-04-02T17:11:52.232637+0000","last_fullsized":"2024-04-02T17:11:52.232637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221764+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:11:52.221764+0000","last_peered":"2024-04-02T17:11:52.221764+0000","last_clean":"2024-04-02T17:11:52.221764+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:11:52.221764+0000","last_undegraded":"2024-04-02T17:11:52.221764+0000","last_fullsized":"2024-04-02T17:11:52.221764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230748+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:11:52.230748+0000","last_peered":"2024-04-02T17:11:52.230748+0000","last_clean":"2024-04-02T17:11:52.230748+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:11:52.230748+0000","last_undegraded":"2024-04-02T17:11:52.230748+0000","last_fullsized":"2024-04-02T17:11:52.230748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429720+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:11:52.429720+0000","last_peered":"2024-04-02T17:11:52.429720+0000","last_clean":"2024-04-02T17:11:52.429720+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:11:52.429720+0000","last_undegraded":"2024-04-02T17:11:52.429720+0000","last_fullsized":"2024-04-02T17:11:52.429720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236708+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:11:52.236708+0000","last_peered":"2024-04-02T17:11:52.236708+0000","last_clean":"2024-04-02T17:11:52.236708+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:11:52.236708+0000","last_undegraded":"2024-04-02T17:11:52.236708+0000","last_fullsized":"2024-04-02T17:11:52.236708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430273+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:11:52.430273+0000","last_peered":"2024-04-02T17:11:52.430273+0000","last_clean":"2024-04-02T17:11:52.430273+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:11:52.430273+0000","last_undegraded":"2024-04-02T17:11:52.430273+0000","last_fullsized":"2024-04-02T17:11:52.430273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76148,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1383,"num_read_kb":777,"num_write":258,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":228444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":485886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1144649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":485311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":485572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1079069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":484911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":553648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":486782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":549865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":484619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":485304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1642496,"data_stored":1078125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:12:59.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T17:13:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.246+0000 7f9b66400640 1 -- 172.21.15.59:0/3637402713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 msgr2=0x7f9b60101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.246+0000 7f9b66400640 1 --2- 172.21.15.59:0/3637402713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b60101170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009860 tx=0x7f9b4c02bb70 comp rx=0 tx=0).stop 2024-04-02T17:13:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 -- 172.21.15.59:0/3637402713 shutdown_connections 2024-04-02T17:13:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 --2- 172.21.15.59:0/3637402713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 0x7f9b60101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 --2- 172.21.15.59:0/3637402713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b60101170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 --2- 172.21.15.59:0/3637402713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 0x7f9b601007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 -- 172.21.15.59:0/3637402713 >> 172.21.15.59:0/3637402713 conn(0x7f9b600fbfb0 msgr2=0x7f9b600fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 -- 172.21.15.59:0/3637402713 shutdown_connections 2024-04-02T17:13:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.247+0000 7f9b66400640 1 -- 172.21.15.59:0/3637402713 wait complete. 2024-04-02T17:13:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 Processor -- start 2024-04-02T17:13:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 -- start start 2024-04-02T17:13:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 0x7f9b6019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 0x7f9b601a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010f980 con 0x7f9b601003b0 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010f800 con 0x7f9b601016b0 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.248+0000 7f9b66400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010fb00 con 0x7f9b60100d10 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50576/0 (socket says 172.21.15.59:50576) 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 -- 172.21.15.59:0/4052369704 learned_addr learned my addr 172.21.15.59:0/4052369704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5fe00640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 0x7f9b601a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5f400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 0x7f9b6019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 -- 172.21.15.59:0/4052369704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 msgr2=0x7f9b601a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 0x7f9b601a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 -- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 msgr2=0x7f9b6019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 0x7f9b6019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 -- 172.21.15.59:0/4052369704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b601a1ca0 con 0x7f9b60100d10 2024-04-02T17:13:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.249+0000 7f9b5ea00640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c037040 tx=0x7f9b4c0323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.250+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c041030 con 0x7f9b60100d10 2024-04-02T17:13:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.250+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c02c510 con 0x7f9b60100d10 2024-04-02T17:13:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.250+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b601a1f30 con 0x7f9b60100d10 2024-04-02T17:13:00.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.250+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c039040 con 0x7f9b60100d10 2024-04-02T17:13:00.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.250+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b601a23a0 con 0x7f9b60100d10 2024-04-02T17:13:00.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.252+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6010b2d0 con 0x7f9b60100d10 2024-04-02T17:13:00.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.253+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b4c044070 con 0x7f9b60100d10 2024-04-02T17:13:00.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.254+0000 7f9b57e00640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 0x7f9b440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.254+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9b4c0f1970 con 0x7f9b60100d10 2024-04-02T17:13:00.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.255+0000 7f9b5f400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 0x7f9b440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.255+0000 7f9b5f400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 0x7f9b440a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b50005fd0 tx=0x7f9b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:00.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.258+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b4c0be290 con 0x7f9b60100d10 2024-04-02T17:13:00.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.479+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b60061f40 con 0x7f9b440a3c30 2024-04-02T17:13:00.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.497+0000 7f9b57e00640 1 -- 172.21.15.59:0/4052369704 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7f9b60061f40 con 0x7f9b440a3c30 2024-04-02T17:13:00.501 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:00.503 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:13:00.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 msgr2=0x7f9b440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 0x7f9b440a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b50005fd0 tx=0x7f9b50009260 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 msgr2=0x7f9b6019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c037040 tx=0x7f9b4c0323d0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 shutdown_connections 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b601016b0 0x7f9b601a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b60100d10 0x7f9b6019a5c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b440a3c30 0x7f9b440a60f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 --2- 172.21.15.59:0/4052369704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b601003b0 0x7f9b6019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 >> 172.21.15.59:0/4052369704 conn(0x7f9b600fbfb0 msgr2=0x7f9b600fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 shutdown_connections 2024-04-02T17:13:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.502+0000 7f9b66400640 1 -- 172.21.15.59:0/4052369704 wait complete. 2024-04-02T17:13:00.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:00 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/944115860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:00.522 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-02T17:12:59.549779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666750,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1509,"num_read_kb":886,"num_write":441,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1180943,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388248,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553576,"statfs":{"total":1151940427776,"available":1151542861824,"internally_reserved":0,"allocated":14434304,"data_stored":7803741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004618"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818152+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:11:52.818152+0000","last_peered":"2024-04-02T17:11:52.818152+0000","last_clean":"2024-04-02T17:11:52.818152+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:11:52.818152+0000","last_undegraded":"2024-04-02T17:11:52.818152+0000","last_fullsized":"2024-04-02T17:11:52.818152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818026+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:11:52.818026+0000","last_peered":"2024-04-02T17:11:52.818026+0000","last_clean":"2024-04-02T17:11:52.818026+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:11:52.818026+0000","last_undegraded":"2024-04-02T17:11:52.818026+0000","last_fullsized":"2024-04-02T17:11:52.818026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222628+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:11:52.222628+0000","last_peered":"2024-04-02T17:11:52.222628+0000","last_clean":"2024-04-02T17:11:52.222628+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:11:52.222628+0000","last_undegraded":"2024-04-02T17:11:52.222628+0000","last_fullsized":"2024-04-02T17:11:52.222628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430624+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:11:52.430624+0000","last_peered":"2024-04-02T17:11:52.430624+0000","last_clean":"2024-04-02T17:11:52.430624+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:11:52.430624+0000","last_undegraded":"2024-04-02T17:11:52.430624+0000","last_fullsized":"2024-04-02T17:11:52.430624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225467+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:11:52.225467+0000","last_peered":"2024-04-02T17:11:52.225467+0000","last_clean":"2024-04-02T17:11:52.225467+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:11:52.225467+0000","last_undegraded":"2024-04-02T17:11:52.225467+0000","last_fullsized":"2024-04-02T17:11:52.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.432056+0000","last_change":"2024-04-02T17:11:52.432056+0000","last_active":"2024-04-02T17:11:52.432056+0000","last_peered":"2024-04-02T17:11:52.432056+0000","last_clean":"2024-04-02T17:11:52.432056+0000","last_became_active":"2024-04-02T17:11:52.431692+0000","last_became_peered":"2024-04-02T17:11:52.431692+0000","last_unstale":"2024-04-02T17:11:52.432056+0000","last_undegraded":"2024-04-02T17:11:52.432056+0000","last_fullsized":"2024-04-02T17:11:52.432056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:51.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230343+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:11:52.230343+0000","last_peered":"2024-04-02T17:11:52.230343+0000","last_clean":"2024-04-02T17:11:52.230343+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:11:52.230343+0000","last_undegraded":"2024-04-02T17:11:52.230343+0000","last_fullsized":"2024-04-02T17:11:52.230343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430135+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:11:52.430135+0000","last_peered":"2024-04-02T17:11:52.430135+0000","last_clean":"2024-04-02T17:11:52.430135+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:11:52.430135+0000","last_undegraded":"2024-04-02T17:11:52.430135+0000","last_fullsized":"2024-04-02T17:11:52.430135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225843+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:11:52.225843+0000","last_peered":"2024-04-02T17:11:52.225843+0000","last_clean":"2024-04-02T17:11:52.225843+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:11:52.225843+0000","last_undegraded":"2024-04-02T17:11:52.225843+0000","last_fullsized":"2024-04-02T17:11:52.225843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224298+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:11:52.224298+0000","last_peered":"2024-04-02T17:11:52.224298+0000","last_clean":"2024-04-02T17:11:52.224298+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:11:52.224298+0000","last_undegraded":"2024-04-02T17:11:52.224298+0000","last_fullsized":"2024-04-02T17:11:52.224298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222208+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:11:52.222208+0000","last_peered":"2024-04-02T17:11:52.222208+0000","last_clean":"2024-04-02T17:11:52.222208+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:11:52.222208+0000","last_undegraded":"2024-04-02T17:11:52.222208+0000","last_fullsized":"2024-04-02T17:11:52.222208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236269+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:11:52.236269+0000","last_peered":"2024-04-02T17:11:52.236269+0000","last_clean":"2024-04-02T17:11:52.236269+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:11:52.236269+0000","last_undegraded":"2024-04-02T17:11:52.236269+0000","last_fullsized":"2024-04-02T17:11:52.236269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818685+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:11:52.818685+0000","last_peered":"2024-04-02T17:11:52.818685+0000","last_clean":"2024-04-02T17:11:52.818685+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:11:52.818685+0000","last_undegraded":"2024-04-02T17:11:52.818685+0000","last_fullsized":"2024-04-02T17:11:52.818685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818544+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:11:52.818544+0000","last_peered":"2024-04-02T17:11:52.818544+0000","last_clean":"2024-04-02T17:11:52.818544+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:11:52.818544+0000","last_undegraded":"2024-04-02T17:11:52.818544+0000","last_fullsized":"2024-04-02T17:11:52.818544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429577+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:11:52.429577+0000","last_peered":"2024-04-02T17:11:52.429577+0000","last_clean":"2024-04-02T17:11:52.429577+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:11:52.429577+0000","last_undegraded":"2024-04-02T17:11:52.429577+0000","last_fullsized":"2024-04-02T17:11:52.429577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430790+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:11:52.430790+0000","last_peered":"2024-04-02T17:11:52.430790+0000","last_clean":"2024-04-02T17:11:52.430790+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:11:52.430790+0000","last_undegraded":"2024-04-02T17:11:52.430790+0000","last_fullsized":"2024-04-02T17:11:52.430790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222683+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:11:52.222683+0000","last_peered":"2024-04-02T17:11:52.222683+0000","last_clean":"2024-04-02T17:11:52.222683+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:11:52.222683+0000","last_undegraded":"2024-04-02T17:11:52.222683+0000","last_fullsized":"2024-04-02T17:11:52.222683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219697+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:11:52.219697+0000","last_peered":"2024-04-02T17:11:52.219697+0000","last_clean":"2024-04-02T17:11:52.219697+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:11:52.219697+0000","last_undegraded":"2024-04-02T17:11:52.219697+0000","last_fullsized":"2024-04-02T17:11:52.219697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.383706+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:12:55.383706+0000","last_peered":"2024-04-02T17:12:55.383706+0000","last_clean":"2024-04-02T17:12:55.383706+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:12:55.383706+0000","last_undegraded":"2024-04-02T17:12:55.383706+0000","last_fullsized":"2024-04-02T17:12:55.383706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232279+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:11:52.232279+0000","last_peered":"2024-04-02T17:11:52.232279+0000","last_clean":"2024-04-02T17:11:52.232279+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:11:52.232279+0000","last_undegraded":"2024-04-02T17:11:52.232279+0000","last_fullsized":"2024-04-02T17:11:52.232279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232788+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:11:52.232788+0000","last_peered":"2024-04-02T17:11:52.232788+0000","last_clean":"2024-04-02T17:11:52.232788+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:11:52.232788+0000","last_undegraded":"2024-04-02T17:11:52.232788+0000","last_fullsized":"2024-04-02T17:11:52.232788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818995+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:11:52.818995+0000","last_peered":"2024-04-02T17:11:52.818995+0000","last_clean":"2024-04-02T17:11:52.818995+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:11:52.818995+0000","last_undegraded":"2024-04-02T17:11:52.818995+0000","last_fullsized":"2024-04-02T17:11:52.818995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430291+0000","last_change":"2024-04-02T17:10:51.330798+0000","last_active":"2024-04-02T17:11:52.430291+0000","last_peered":"2024-04-02T17:11:52.430291+0000","last_clean":"2024-04-02T17:11:52.430291+0000","last_became_active":"2024-04-02T17:10:51.318708+0000","last_became_peered":"2024-04-02T17:10:51.318708+0000","last_unstale":"2024-04-02T17:11:52.430291+0000","last_undegraded":"2024-04-02T17:11:52.430291+0000","last_fullsized":"2024-04-02T17:11:52.430291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:08:25.035375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429776+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:11:52.429776+0000","last_peered":"2024-04-02T17:11:52.429776+0000","last_clean":"2024-04-02T17:11:52.429776+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:11:52.429776+0000","last_undegraded":"2024-04-02T17:11:52.429776+0000","last_fullsized":"2024-04-02T17:11:52.429776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430934+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:11:52.430934+0000","last_peered":"2024-04-02T17:11:52.430934+0000","last_clean":"2024-04-02T17:11:52.430934+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:11:52.430934+0000","last_undegraded":"2024-04-02T17:11:52.430934+0000","last_fullsized":"2024-04-02T17:11:52.430934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221978+0000","last_change":"2024-04-02T17:10:51.634458+0000","last_active":"2024-04-02T17:11:52.221978+0000","last_peered":"2024-04-02T17:11:52.221978+0000","last_clean":"2024-04-02T17:11:52.221978+0000","last_became_active":"2024-04-02T17:10:51.634064+0000","last_became_peered":"2024-04-02T17:10:51.634064+0000","last_unstale":"2024-04-02T17:11:52.221978+0000","last_undegraded":"2024-04-02T17:11:52.221978+0000","last_fullsized":"2024-04-02T17:11:52.221978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:10.010493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:54.296307+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:12:54.296307+0000","last_peered":"2024-04-02T17:12:54.296307+0000","last_clean":"2024-04-02T17:12:54.296307+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:12:54.296307+0000","last_undegraded":"2024-04-02T17:12:54.296307+0000","last_fullsized":"2024-04-02T17:12:54.296307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430521+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:11:52.430521+0000","last_peered":"2024-04-02T17:11:52.430521+0000","last_clean":"2024-04-02T17:11:52.430521+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:11:52.430521+0000","last_undegraded":"2024-04-02T17:11:52.430521+0000","last_fullsized":"2024-04-02T17:11:52.430521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225449+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:11:52.225449+0000","last_peered":"2024-04-02T17:11:52.225449+0000","last_clean":"2024-04-02T17:11:52.225449+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:11:52.225449+0000","last_undegraded":"2024-04-02T17:11:52.225449+0000","last_fullsized":"2024-04-02T17:11:52.225449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221833+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:11:52.221833+0000","last_peered":"2024-04-02T17:11:52.221833+0000","last_clean":"2024-04-02T17:11:52.221833+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:11:52.221833+0000","last_undegraded":"2024-04-02T17:11:52.221833+0000","last_fullsized":"2024-04-02T17:11:52.221833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235557+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:11:52.235557+0000","last_peered":"2024-04-02T17:11:52.235557+0000","last_clean":"2024-04-02T17:11:52.235557+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:11:52.235557+0000","last_undegraded":"2024-04-02T17:11:52.235557+0000","last_fullsized":"2024-04-02T17:11:52.235557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230277+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:11:52.230277+0000","last_peered":"2024-04-02T17:11:52.230277+0000","last_clean":"2024-04-02T17:11:52.230277+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:11:52.230277+0000","last_undegraded":"2024-04-02T17:11:52.230277+0000","last_fullsized":"2024-04-02T17:11:52.230277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221584+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:11:52.221584+0000","last_peered":"2024-04-02T17:11:52.221584+0000","last_clean":"2024-04-02T17:11:52.221584+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:11:52.221584+0000","last_undegraded":"2024-04-02T17:11:52.221584+0000","last_fullsized":"2024-04-02T17:11:52.221584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221849+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:11:52.221849+0000","last_peered":"2024-04-02T17:11:52.221849+0000","last_clean":"2024-04-02T17:11:52.221849+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:11:52.221849+0000","last_undegraded":"2024-04-02T17:11:52.221849+0000","last_fullsized":"2024-04-02T17:11:52.221849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.299066+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:12:57.299066+0000","last_peered":"2024-04-02T17:12:57.299066+0000","last_clean":"2024-04-02T17:12:57.299066+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:12:57.299066+0000","last_undegraded":"2024-04-02T17:12:57.299066+0000","last_fullsized":"2024-04-02T17:12:57.299066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430074+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:11:52.430074+0000","last_peered":"2024-04-02T17:11:52.430074+0000","last_clean":"2024-04-02T17:11:52.430074+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:11:52.430074+0000","last_undegraded":"2024-04-02T17:11:52.430074+0000","last_fullsized":"2024-04-02T17:11:52.430074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224087+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:11:52.224087+0000","last_peered":"2024-04-02T17:11:52.224087+0000","last_clean":"2024-04-02T17:11:52.224087+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:11:52.224087+0000","last_undegraded":"2024-04-02T17:11:52.224087+0000","last_fullsized":"2024-04-02T17:11:52.224087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819045+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:11:52.819045+0000","last_peered":"2024-04-02T17:11:52.819045+0000","last_clean":"2024-04-02T17:11:52.819045+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:11:52.819045+0000","last_undegraded":"2024-04-02T17:11:52.819045+0000","last_fullsized":"2024-04-02T17:11:52.819045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222125+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:11:52.222125+0000","last_peered":"2024-04-02T17:11:52.222125+0000","last_clean":"2024-04-02T17:11:52.222125+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:11:52.222125+0000","last_undegraded":"2024-04-02T17:11:52.222125+0000","last_fullsized":"2024-04-02T17:11:52.222125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429946+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:11:52.429946+0000","last_peered":"2024-04-02T17:11:52.429946+0000","last_clean":"2024-04-02T17:11:52.429946+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:11:52.429946+0000","last_undegraded":"2024-04-02T17:11:52.429946+0000","last_fullsized":"2024-04-02T17:11:52.429946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":340,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.300323+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:12:57.300323+0000","last_peered":"2024-04-02T17:12:57.300323+0000","last_clean":"2024-04-02T17:12:57.300323+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:12:57.300323+0000","last_undegraded":"2024-04-02T17:12:57.300323+0000","last_fullsized":"2024-04-02T17:12:57.300323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67824,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":207,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430928+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:11:52.430928+0000","last_peered":"2024-04-02T17:11:52.430928+0000","last_clean":"2024-04-02T17:11:52.430928+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:11:52.430928+0000","last_undegraded":"2024-04-02T17:11:52.430928+0000","last_fullsized":"2024-04-02T17:11:52.430928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219640+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:11:52.219640+0000","last_peered":"2024-04-02T17:11:52.219640+0000","last_clean":"2024-04-02T17:11:52.219640+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:11:52.219640+0000","last_undegraded":"2024-04-02T17:11:52.219640+0000","last_fullsized":"2024-04-02T17:11:52.219640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235862+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:11:52.235862+0000","last_peered":"2024-04-02T17:11:52.235862+0000","last_clean":"2024-04-02T17:11:52.235862+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:11:52.235862+0000","last_undegraded":"2024-04-02T17:11:52.235862+0000","last_fullsized":"2024-04-02T17:11:52.235862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430181+0000","last_change":"2024-04-02T17:10:51.336678+0000","last_active":"2024-04-02T17:11:52.430181+0000","last_peered":"2024-04-02T17:11:52.430181+0000","last_clean":"2024-04-02T17:11:52.430181+0000","last_became_active":"2024-04-02T17:10:51.325028+0000","last_became_peered":"2024-04-02T17:10:51.325028+0000","last_unstale":"2024-04-02T17:11:52.430181+0000","last_undegraded":"2024-04-02T17:11:52.430181+0000","last_fullsized":"2024-04-02T17:11:52.430181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:54:29.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224346+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:11:52.224346+0000","last_peered":"2024-04-02T17:11:52.224346+0000","last_clean":"2024-04-02T17:11:52.224346+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:11:52.224346+0000","last_undegraded":"2024-04-02T17:11:52.224346+0000","last_fullsized":"2024-04-02T17:11:52.224346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.399180+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:12:55.399180+0000","last_peered":"2024-04-02T17:12:55.399180+0000","last_clean":"2024-04-02T17:12:55.399180+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:12:55.399180+0000","last_undegraded":"2024-04-02T17:12:55.399180+0000","last_fullsized":"2024-04-02T17:12:55.399180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223641+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:11:52.223641+0000","last_peered":"2024-04-02T17:11:52.223641+0000","last_clean":"2024-04-02T17:11:52.223641+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:11:52.223641+0000","last_undegraded":"2024-04-02T17:11:52.223641+0000","last_fullsized":"2024-04-02T17:11:52.223641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430477+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:11:52.430477+0000","last_peered":"2024-04-02T17:11:52.430477+0000","last_clean":"2024-04-02T17:11:52.430477+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:11:52.430477+0000","last_undegraded":"2024-04-02T17:11:52.430477+0000","last_fullsized":"2024-04-02T17:11:52.430477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222202+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:11:52.222202+0000","last_peered":"2024-04-02T17:11:52.222202+0000","last_clean":"2024-04-02T17:11:52.222202+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:11:52.222202+0000","last_undegraded":"2024-04-02T17:11:52.222202+0000","last_fullsized":"2024-04-02T17:11:52.222202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431210+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:11:52.431210+0000","last_peered":"2024-04-02T17:11:52.431210+0000","last_clean":"2024-04-02T17:11:52.431210+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:11:52.431210+0000","last_undegraded":"2024-04-02T17:11:52.431210+0000","last_fullsized":"2024-04-02T17:11:52.431210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430525+0000","last_change":"2024-04-02T17:10:51.321193+0000","last_active":"2024-04-02T17:11:52.430525+0000","last_peered":"2024-04-02T17:11:52.430525+0000","last_clean":"2024-04-02T17:11:52.430525+0000","last_became_active":"2024-04-02T17:10:51.320791+0000","last_became_peered":"2024-04-02T17:10:51.320791+0000","last_unstale":"2024-04-02T17:11:52.430525+0000","last_undegraded":"2024-04-02T17:11:52.430525+0000","last_fullsized":"2024-04-02T17:11:52.430525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:22:18.979764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222639+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:11:52.222639+0000","last_peered":"2024-04-02T17:11:52.222639+0000","last_clean":"2024-04-02T17:11:52.222639+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:11:52.222639+0000","last_undegraded":"2024-04-02T17:11:52.222639+0000","last_fullsized":"2024-04-02T17:11:52.222639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.243589+0000","last_change":"2024-04-02T17:11:52.243152+0000","last_active":"2024-04-02T17:11:52.243589+0000","last_peered":"2024-04-02T17:11:52.243589+0000","last_clean":"2024-04-02T17:11:52.243589+0000","last_became_active":"2024-04-02T17:11:52.232947+0000","last_became_peered":"2024-04-02T17:11:52.232947+0000","last_unstale":"2024-04-02T17:11:52.243589+0000","last_undegraded":"2024-04-02T17:11:52.243589+0000","last_fullsized":"2024-04-02T17:11:52.243589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:49:58.858519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:56.386744+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:12:56.386744+0000","last_peered":"2024-04-02T17:12:56.386744+0000","last_clean":"2024-04-02T17:12:56.386744+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:12:56.386744+0000","last_undegraded":"2024-04-02T17:12:56.386744+0000","last_fullsized":"2024-04-02T17:12:56.386744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225702+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:11:52.225702+0000","last_peered":"2024-04-02T17:11:52.225702+0000","last_clean":"2024-04-02T17:11:52.225702+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:11:52.225702+0000","last_undegraded":"2024-04-02T17:11:52.225702+0000","last_fullsized":"2024-04-02T17:11:52.225702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221789+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:11:52.221789+0000","last_peered":"2024-04-02T17:11:52.221789+0000","last_clean":"2024-04-02T17:11:52.221789+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:11:52.221789+0000","last_undegraded":"2024-04-02T17:11:52.221789+0000","last_fullsized":"2024-04-02T17:11:52.221789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.402458+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:12:57.402458+0000","last_peered":"2024-04-02T17:12:57.402458+0000","last_clean":"2024-04-02T17:12:57.402458+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:12:57.402458+0000","last_undegraded":"2024-04-02T17:12:57.402458+0000","last_fullsized":"2024-04-02T17:12:57.402458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222174+0000","last_change":"2024-04-02T17:10:51.633808+0000","last_active":"2024-04-02T17:11:52.222174+0000","last_peered":"2024-04-02T17:11:52.222174+0000","last_clean":"2024-04-02T17:11:52.222174+0000","last_became_active":"2024-04-02T17:10:51.633095+0000","last_became_peered":"2024-04-02T17:10:51.633095+0000","last_unstale":"2024-04-02T17:11:52.222174+0000","last_undegraded":"2024-04-02T17:11:52.222174+0000","last_fullsized":"2024-04-02T17:11:52.222174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:36:16.308694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222251+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:11:52.222251+0000","last_peered":"2024-04-02T17:11:52.222251+0000","last_clean":"2024-04-02T17:11:52.222251+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:11:52.222251+0000","last_undegraded":"2024-04-02T17:11:52.222251+0000","last_fullsized":"2024-04-02T17:11:52.222251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223736+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:11:52.223736+0000","last_peered":"2024-04-02T17:11:52.223736+0000","last_clean":"2024-04-02T17:11:52.223736+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:11:52.223736+0000","last_undegraded":"2024-04-02T17:11:52.223736+0000","last_fullsized":"2024-04-02T17:11:52.223736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235203+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:11:52.235203+0000","last_peered":"2024-04-02T17:11:52.235203+0000","last_clean":"2024-04-02T17:11:52.235203+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:11:52.235203+0000","last_undegraded":"2024-04-02T17:11:52.235203+0000","last_fullsized":"2024-04-02T17:11:52.235203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223856+0000","last_change":"2024-04-02T17:10:51.637846+0000","last_active":"2024-04-02T17:11:52.223856+0000","last_peered":"2024-04-02T17:11:52.223856+0000","last_clean":"2024-04-02T17:11:52.223856+0000","last_became_active":"2024-04-02T17:10:51.633829+0000","last_became_peered":"2024-04-02T17:10:51.633829+0000","last_unstale":"2024-04-02T17:11:52.223856+0000","last_undegraded":"2024-04-02T17:11:52.223856+0000","last_fullsized":"2024-04-02T17:11:52.223856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:10:45.177246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431026+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:11:52.431026+0000","last_peered":"2024-04-02T17:11:52.431026+0000","last_clean":"2024-04-02T17:11:52.431026+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:11:52.431026+0000","last_undegraded":"2024-04-02T17:11:52.431026+0000","last_fullsized":"2024-04-02T17:11:52.431026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237144+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:11:52.237144+0000","last_peered":"2024-04-02T17:11:52.237144+0000","last_clean":"2024-04-02T17:11:52.237144+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:11:52.237144+0000","last_undegraded":"2024-04-02T17:11:52.237144+0000","last_fullsized":"2024-04-02T17:11:52.237144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:47.323048+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:12:47.323048+0000","last_peered":"2024-04-02T17:12:47.323048+0000","last_clean":"2024-04-02T17:12:47.323048+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:12:47.323048+0000","last_undegraded":"2024-04-02T17:12:47.323048+0000","last_fullsized":"2024-04-02T17:12:47.323048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.233095+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:11:52.233095+0000","last_peered":"2024-04-02T17:11:52.233095+0000","last_clean":"2024-04-02T17:11:52.233095+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:11:52.233095+0000","last_undegraded":"2024-04-02T17:11:52.233095+0000","last_fullsized":"2024-04-02T17:11:52.233095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222419+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:11:52.222419+0000","last_peered":"2024-04-02T17:11:52.222419+0000","last_clean":"2024-04-02T17:11:52.222419+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:11:52.222419+0000","last_undegraded":"2024-04-02T17:11:52.222419+0000","last_fullsized":"2024-04-02T17:11:52.222419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818278+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:11:52.818278+0000","last_peered":"2024-04-02T17:11:52.818278+0000","last_clean":"2024-04-02T17:11:52.818278+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:11:52.818278+0000","last_undegraded":"2024-04-02T17:11:52.818278+0000","last_fullsized":"2024-04-02T17:11:52.818278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429640+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:11:52.429640+0000","last_peered":"2024-04-02T17:11:52.429640+0000","last_clean":"2024-04-02T17:11:52.429640+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:11:52.429640+0000","last_undegraded":"2024-04-02T17:11:52.429640+0000","last_fullsized":"2024-04-02T17:11:52.429640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430857+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:11:52.430857+0000","last_peered":"2024-04-02T17:11:52.430857+0000","last_clean":"2024-04-02T17:11:52.430857+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:11:52.430857+0000","last_undegraded":"2024-04-02T17:11:52.430857+0000","last_fullsized":"2024-04-02T17:11:52.430857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230823+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:11:52.230823+0000","last_peered":"2024-04-02T17:11:52.230823+0000","last_clean":"2024-04-02T17:11:52.230823+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:11:52.230823+0000","last_undegraded":"2024-04-02T17:11:52.230823+0000","last_fullsized":"2024-04-02T17:11:52.230823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430110+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:11:52.430110+0000","last_peered":"2024-04-02T17:11:52.430110+0000","last_clean":"2024-04-02T17:11:52.430110+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:11:52.430110+0000","last_undegraded":"2024-04-02T17:11:52.430110+0000","last_fullsized":"2024-04-02T17:11:52.430110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219529+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:11:52.219529+0000","last_peered":"2024-04-02T17:11:52.219529+0000","last_clean":"2024-04-02T17:11:52.219529+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:11:52.219529+0000","last_undegraded":"2024-04-02T17:11:52.219529+0000","last_fullsized":"2024-04-02T17:11:52.219529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232232+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:11:52.232232+0000","last_peered":"2024-04-02T17:11:52.232232+0000","last_clean":"2024-04-02T17:11:52.232232+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:11:52.232232+0000","last_undegraded":"2024-04-02T17:11:52.232232+0000","last_fullsized":"2024-04-02T17:11:52.232232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:51.641235+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:51.634715+0000","last_became_peered":"2024-04-02T17:10:51.634715+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:56:38.764506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224004+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:11:52.224004+0000","last_peered":"2024-04-02T17:11:52.224004+0000","last_clean":"2024-04-02T17:11:52.224004+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:11:52.224004+0000","last_undegraded":"2024-04-02T17:11:52.224004+0000","last_fullsized":"2024-04-02T17:11:52.224004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:53.293909+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:12:53.293909+0000","last_peered":"2024-04-02T17:12:53.293909+0000","last_clean":"2024-04-02T17:12:53.293909+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:12:53.293909+0000","last_undegraded":"2024-04-02T17:12:53.293909+0000","last_fullsized":"2024-04-02T17:12:53.293909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224498+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:11:52.224498+0000","last_peered":"2024-04-02T17:11:52.224498+0000","last_clean":"2024-04-02T17:11:52.224498+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:11:52.224498+0000","last_undegraded":"2024-04-02T17:11:52.224498+0000","last_fullsized":"2024-04-02T17:11:52.224498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221897+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:11:52.221897+0000","last_peered":"2024-04-02T17:11:52.221897+0000","last_clean":"2024-04-02T17:11:52.221897+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:11:52.221897+0000","last_undegraded":"2024-04-02T17:11:52.221897+0000","last_fullsized":"2024-04-02T17:11:52.221897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224105+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:11:52.224105+0000","last_peered":"2024-04-02T17:11:52.224105+0000","last_clean":"2024-04-02T17:11:52.224105+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:11:52.224105+0000","last_undegraded":"2024-04-02T17:11:52.224105+0000","last_fullsized":"2024-04-02T17:11:52.224105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236455+0000","last_change":"2024-04-02T17:10:51.642752+0000","last_active":"2024-04-02T17:11:52.236455+0000","last_peered":"2024-04-02T17:11:52.236455+0000","last_clean":"2024-04-02T17:11:52.236455+0000","last_became_active":"2024-04-02T17:10:51.635734+0000","last_became_peered":"2024-04-02T17:10:51.635734+0000","last_unstale":"2024-04-02T17:11:52.236455+0000","last_undegraded":"2024-04-02T17:11:52.236455+0000","last_fullsized":"2024-04-02T17:11:52.236455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.358529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224156+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:11:52.224156+0000","last_peered":"2024-04-02T17:11:52.224156+0000","last_clean":"2024-04-02T17:11:52.224156+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:11:52.224156+0000","last_undegraded":"2024-04-02T17:11:52.224156+0000","last_fullsized":"2024-04-02T17:11:52.224156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430173+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:11:52.430173+0000","last_peered":"2024-04-02T17:11:52.430173+0000","last_clean":"2024-04-02T17:11:52.430173+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:11:52.430173+0000","last_undegraded":"2024-04-02T17:11:52.430173+0000","last_fullsized":"2024-04-02T17:11:52.430173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237295+0000","last_change":"2024-04-02T17:11:52.236694+0000","last_active":"2024-04-02T17:11:52.237295+0000","last_peered":"2024-04-02T17:11:52.237295+0000","last_clean":"2024-04-02T17:11:52.237295+0000","last_became_active":"2024-04-02T17:11:52.236128+0000","last_became_peered":"2024-04-02T17:11:52.236128+0000","last_unstale":"2024-04-02T17:11:52.237295+0000","last_undegraded":"2024-04-02T17:11:52.237295+0000","last_fullsized":"2024-04-02T17:11:52.237295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:40.601150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219390+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:11:52.219390+0000","last_peered":"2024-04-02T17:11:52.219390+0000","last_clean":"2024-04-02T17:11:52.219390+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:11:52.219390+0000","last_undegraded":"2024-04-02T17:11:52.219390+0000","last_fullsized":"2024-04-02T17:11:52.219390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819694+0000","last_change":"2024-04-02T17:10:51.332185+0000","last_active":"2024-04-02T17:11:52.819694+0000","last_peered":"2024-04-02T17:11:52.819694+0000","last_clean":"2024-04-02T17:11:52.819694+0000","last_became_active":"2024-04-02T17:10:51.331547+0000","last_became_peered":"2024-04-02T17:10:51.331547+0000","last_unstale":"2024-04-02T17:11:52.819694+0000","last_undegraded":"2024-04-02T17:11:52.819694+0000","last_fullsized":"2024-04-02T17:11:52.819694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:43.766907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236916+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:11:52.236916+0000","last_peered":"2024-04-02T17:11:52.236916+0000","last_clean":"2024-04-02T17:11:52.236916+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:11:52.236916+0000","last_undegraded":"2024-04-02T17:11:52.236916+0000","last_fullsized":"2024-04-02T17:11:52.236916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430480+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:11:52.430480+0000","last_peered":"2024-04-02T17:11:52.430480+0000","last_clean":"2024-04-02T17:11:52.430480+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:11:52.430480+0000","last_undegraded":"2024-04-02T17:11:52.430480+0000","last_fullsized":"2024-04-02T17:11:52.430480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.220006+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:11:52.220006+0000","last_peered":"2024-04-02T17:11:52.220006+0000","last_clean":"2024-04-02T17:11:52.220006+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:11:52.220006+0000","last_undegraded":"2024-04-02T17:11:52.220006+0000","last_fullsized":"2024-04-02T17:11:52.220006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232637+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:11:52.232637+0000","last_peered":"2024-04-02T17:11:52.232637+0000","last_clean":"2024-04-02T17:11:52.232637+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:11:52.232637+0000","last_undegraded":"2024-04-02T17:11:52.232637+0000","last_fullsized":"2024-04-02T17:11:52.232637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221764+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:11:52.221764+0000","last_peered":"2024-04-02T17:11:52.221764+0000","last_clean":"2024-04-02T17:11:52.221764+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:11:52.221764+0000","last_undegraded":"2024-04-02T17:11:52.221764+0000","last_fullsized":"2024-04-02T17:11:52.221764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230748+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:11:52.230748+0000","last_peered":"2024-04-02T17:11:52.230748+0000","last_clean":"2024-04-02T17:11:52.230748+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:11:52.230748+0000","last_undegraded":"2024-04-02T17:11:52.230748+0000","last_fullsized":"2024-04-02T17:11:52.230748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429720+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:11:52.429720+0000","last_peered":"2024-04-02T17:11:52.429720+0000","last_clean":"2024-04-02T17:11:52.429720+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:11:52.429720+0000","last_undegraded":"2024-04-02T17:11:52.429720+0000","last_fullsized":"2024-04-02T17:11:52.429720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236708+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:11:52.236708+0000","last_peered":"2024-04-02T17:11:52.236708+0000","last_clean":"2024-04-02T17:11:52.236708+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:11:52.236708+0000","last_undegraded":"2024-04-02T17:11:52.236708+0000","last_fullsized":"2024-04-02T17:11:52.236708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430273+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:11:52.430273+0000","last_peered":"2024-04-02T17:11:52.430273+0000","last_clean":"2024-04-02T17:11:52.430273+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:11:52.430273+0000","last_undegraded":"2024-04-02T17:11:52.430273+0000","last_fullsized":"2024-04-02T17:11:52.430273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76148,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1383,"num_read_kb":777,"num_write":258,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":228444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":485886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1144649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":485311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":485572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1079069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":484911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":553648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":486782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":549865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":484619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":485304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1642496,"data_stored":1078125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:13:00.526 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-02T17:13:00.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T17:13:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:00 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/944115860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:00.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:00 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/944115860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:00.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.776+0000 7feecb400640 1 -- 172.21.15.59:0/437367229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 msgr2=0x7feec40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 --2- 172.21.15.59:0/437367229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec40768e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009900 tx=0x7feeb802bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:00.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 -- 172.21.15.59:0/437367229 shutdown_connections 2024-04-02T17:13:00.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 --2- 172.21.15.59:0/437367229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec4077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 --2- 172.21.15.59:0/437367229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 0x7feec4078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 --2- 172.21.15.59:0/437367229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec40768e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 -- 172.21.15.59:0/437367229 >> 172.21.15.59:0/437367229 conn(0x7feec40ff2f0 msgr2=0x7feec4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:00.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 -- 172.21.15.59:0/437367229 shutdown_connections 2024-04-02T17:13:00.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.777+0000 7feecb400640 1 -- 172.21.15.59:0/437367229 wait complete. 2024-04-02T17:13:00.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 Processor -- start 2024-04-02T17:13:00.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 -- start start 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 0x7feec41a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113dd0 con 0x7feec4077700 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113c50 con 0x7feec40780a0 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.778+0000 7feecb400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113f50 con 0x7feec40764c0 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60202/0 (socket says 172.21.15.59:60202) 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec3e00640 1 -- 172.21.15.59:0/565671049 learned_addr learned my addr 172.21.15.59:0/565671049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec9400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 0x7feec41a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 -- 172.21.15.59:0/565671049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 msgr2=0x7feec41a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 0x7feec41a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 -- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 msgr2=0x7feec419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:00.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:00.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec8a00640 1 -- 172.21.15.59:0/565671049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec41a6040 con 0x7feec40764c0 2024-04-02T17:13:00.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.779+0000 7feec3e00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:00.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feec8a00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec419e420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009230 tx=0x7feeb802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:00.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8041070 con 0x7feec40764c0 2024-04-02T17:13:00.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb8004450 con 0x7feec40764c0 2024-04-02T17:13:00.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8003df0 con 0x7feec40764c0 2024-04-02T17:13:00.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec41a62d0 con 0x7feec40764c0 2024-04-02T17:13:00.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.780+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41a6770 con 0x7feec40764c0 2024-04-02T17:13:00.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.781+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec41107e0 con 0x7feec40764c0 2024-04-02T17:13:00.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.784+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feeb8044070 con 0x7feec40764c0 2024-04-02T17:13:00.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.785+0000 7feec1600640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 0x7feeac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:00.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.785+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7feeb80f1e50 con 0x7feec40764c0 2024-04-02T17:13:00.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.785+0000 7feec3e00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 0x7feeac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:00.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.786+0000 7feec3e00640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 0x7feeac0a60f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feec419f930 tx=0x7feeb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:00.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:00.789+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feeb80be120 con 0x7feec40764c0 2024-04-02T17:13:01.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.007+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 --> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feec4061f40 con 0x7feeac0a3c30 2024-04-02T17:13:01.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.020+0000 7feec1600640 1 -- 172.21.15.59:0/565671049 <== mgr.14152 v2:172.21.15.59:6800/1634390325 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7feec4061f40 con 0x7feeac0a3c30 2024-04-02T17:13:01.023 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:01.025 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-02T17:13:01.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.025+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 msgr2=0x7feeac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.025+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 0x7feeac0a60f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feec419f930 tx=0x7feeb0005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:01.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.025+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 msgr2=0x7feec419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.025+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec419e420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009230 tx=0x7feeb802c650 comp rx=0 tx=0).stop 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 shutdown_connections 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7feeac0a3c30 0x7feeac0a60f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feec4077700 0x7feec419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40780a0 0x7feec41a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 --2- 172.21.15.59:0/565671049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40764c0 0x7feec419e420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 >> 172.21.15.59:0/565671049 conn(0x7feec40ff2f0 msgr2=0x7feec41009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 shutdown_connections 2024-04-02T17:13:01.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.026+0000 7feecb400640 1 -- 172.21.15.59:0/565671049 wait complete. 2024-04-02T17:13:01.052 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-02T17:12:59.549779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666750,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1509,"num_read_kb":886,"num_write":441,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1180943,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388248,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553576,"statfs":{"total":1151940427776,"available":1151542861824,"internally_reserved":0,"allocated":14434304,"data_stored":7803741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004618"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:35.039578+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:35.038302+0000","last_became_peered":"2024-04-02T17:10:35.038302+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:59:05.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818152+0000","last_change":"2024-04-02T17:10:35.042216+0000","last_active":"2024-04-02T17:11:52.818152+0000","last_peered":"2024-04-02T17:11:52.818152+0000","last_clean":"2024-04-02T17:11:52.818152+0000","last_became_active":"2024-04-02T17:10:35.041713+0000","last_became_peered":"2024-04-02T17:10:35.041713+0000","last_unstale":"2024-04-02T17:11:52.818152+0000","last_undegraded":"2024-04-02T17:11:52.818152+0000","last_fullsized":"2024-04-02T17:11:52.818152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:51:10.245388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818026+0000","last_change":"2024-04-02T17:10:35.047377+0000","last_active":"2024-04-02T17:11:52.818026+0000","last_peered":"2024-04-02T17:11:52.818026+0000","last_clean":"2024-04-02T17:11:52.818026+0000","last_became_active":"2024-04-02T17:10:35.047078+0000","last_became_peered":"2024-04-02T17:10:35.047078+0000","last_unstale":"2024-04-02T17:11:52.818026+0000","last_undegraded":"2024-04-02T17:11:52.818026+0000","last_fullsized":"2024-04-02T17:11:52.818026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:31:43.966373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222628+0000","last_change":"2024-04-02T17:10:35.034324+0000","last_active":"2024-04-02T17:11:52.222628+0000","last_peered":"2024-04-02T17:11:52.222628+0000","last_clean":"2024-04-02T17:11:52.222628+0000","last_became_active":"2024-04-02T17:10:35.034010+0000","last_became_peered":"2024-04-02T17:10:35.034010+0000","last_unstale":"2024-04-02T17:11:52.222628+0000","last_undegraded":"2024-04-02T17:11:52.222628+0000","last_fullsized":"2024-04-02T17:11:52.222628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:13.805002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430624+0000","last_change":"2024-04-02T17:10:35.045286+0000","last_active":"2024-04-02T17:11:52.430624+0000","last_peered":"2024-04-02T17:11:52.430624+0000","last_clean":"2024-04-02T17:11:52.430624+0000","last_became_active":"2024-04-02T17:10:35.044863+0000","last_became_peered":"2024-04-02T17:10:35.044863+0000","last_unstale":"2024-04-02T17:11:52.430624+0000","last_undegraded":"2024-04-02T17:11:52.430624+0000","last_fullsized":"2024-04-02T17:11:52.430624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:36:14.788510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225467+0000","last_change":"2024-04-02T17:10:35.037881+0000","last_active":"2024-04-02T17:11:52.225467+0000","last_peered":"2024-04-02T17:11:52.225467+0000","last_clean":"2024-04-02T17:11:52.225467+0000","last_became_active":"2024-04-02T17:10:35.037561+0000","last_became_peered":"2024-04-02T17:10:35.037561+0000","last_unstale":"2024-04-02T17:11:52.225467+0000","last_undegraded":"2024-04-02T17:11:52.225467+0000","last_fullsized":"2024-04-02T17:11:52.225467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:04.591750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.432056+0000","last_change":"2024-04-02T17:11:52.432056+0000","last_active":"2024-04-02T17:11:52.432056+0000","last_peered":"2024-04-02T17:11:52.432056+0000","last_clean":"2024-04-02T17:11:52.432056+0000","last_became_active":"2024-04-02T17:11:52.431692+0000","last_became_peered":"2024-04-02T17:11:52.431692+0000","last_unstale":"2024-04-02T17:11:52.432056+0000","last_undegraded":"2024-04-02T17:11:52.432056+0000","last_fullsized":"2024-04-02T17:11:52.432056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:51.196280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230343+0000","last_change":"2024-04-02T17:10:35.034235+0000","last_active":"2024-04-02T17:11:52.230343+0000","last_peered":"2024-04-02T17:11:52.230343+0000","last_clean":"2024-04-02T17:11:52.230343+0000","last_became_active":"2024-04-02T17:10:35.033874+0000","last_became_peered":"2024-04-02T17:10:35.033874+0000","last_unstale":"2024-04-02T17:11:52.230343+0000","last_undegraded":"2024-04-02T17:11:52.230343+0000","last_fullsized":"2024-04-02T17:11:52.230343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:06:21.431541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430135+0000","last_change":"2024-04-02T17:10:35.036490+0000","last_active":"2024-04-02T17:11:52.430135+0000","last_peered":"2024-04-02T17:11:52.430135+0000","last_clean":"2024-04-02T17:11:52.430135+0000","last_became_active":"2024-04-02T17:10:35.034835+0000","last_became_peered":"2024-04-02T17:10:35.034835+0000","last_unstale":"2024-04-02T17:11:52.430135+0000","last_undegraded":"2024-04-02T17:11:52.430135+0000","last_fullsized":"2024-04-02T17:11:52.430135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:55.843019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225843+0000","last_change":"2024-04-02T17:10:35.038654+0000","last_active":"2024-04-02T17:11:52.225843+0000","last_peered":"2024-04-02T17:11:52.225843+0000","last_clean":"2024-04-02T17:11:52.225843+0000","last_became_active":"2024-04-02T17:10:35.038271+0000","last_became_peered":"2024-04-02T17:10:35.038271+0000","last_unstale":"2024-04-02T17:11:52.225843+0000","last_undegraded":"2024-04-02T17:11:52.225843+0000","last_fullsized":"2024-04-02T17:11:52.225843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:48:25.824627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224298+0000","last_change":"2024-04-02T17:10:35.036688+0000","last_active":"2024-04-02T17:11:52.224298+0000","last_peered":"2024-04-02T17:11:52.224298+0000","last_clean":"2024-04-02T17:11:52.224298+0000","last_became_active":"2024-04-02T17:10:35.036376+0000","last_became_peered":"2024-04-02T17:10:35.036376+0000","last_unstale":"2024-04-02T17:11:52.224298+0000","last_undegraded":"2024-04-02T17:11:52.224298+0000","last_fullsized":"2024-04-02T17:11:52.224298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:18.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222208+0000","last_change":"2024-04-02T17:10:35.030248+0000","last_active":"2024-04-02T17:11:52.222208+0000","last_peered":"2024-04-02T17:11:52.222208+0000","last_clean":"2024-04-02T17:11:52.222208+0000","last_became_active":"2024-04-02T17:10:35.029299+0000","last_became_peered":"2024-04-02T17:10:35.029299+0000","last_unstale":"2024-04-02T17:11:52.222208+0000","last_undegraded":"2024-04-02T17:11:52.222208+0000","last_fullsized":"2024-04-02T17:11:52.222208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:47:15.096506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236269+0000","last_change":"2024-04-02T17:10:35.040020+0000","last_active":"2024-04-02T17:11:52.236269+0000","last_peered":"2024-04-02T17:11:52.236269+0000","last_clean":"2024-04-02T17:11:52.236269+0000","last_became_active":"2024-04-02T17:10:35.039380+0000","last_became_peered":"2024-04-02T17:10:35.039380+0000","last_unstale":"2024-04-02T17:11:52.236269+0000","last_undegraded":"2024-04-02T17:11:52.236269+0000","last_fullsized":"2024-04-02T17:11:52.236269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:46:01.281621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818685+0000","last_change":"2024-04-02T17:10:35.047519+0000","last_active":"2024-04-02T17:11:52.818685+0000","last_peered":"2024-04-02T17:11:52.818685+0000","last_clean":"2024-04-02T17:11:52.818685+0000","last_became_active":"2024-04-02T17:10:35.040492+0000","last_became_peered":"2024-04-02T17:10:35.040492+0000","last_unstale":"2024-04-02T17:11:52.818685+0000","last_undegraded":"2024-04-02T17:11:52.818685+0000","last_fullsized":"2024-04-02T17:11:52.818685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:44.129162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818544+0000","last_change":"2024-04-02T17:10:35.051441+0000","last_active":"2024-04-02T17:11:52.818544+0000","last_peered":"2024-04-02T17:11:52.818544+0000","last_clean":"2024-04-02T17:11:52.818544+0000","last_became_active":"2024-04-02T17:10:35.046798+0000","last_became_peered":"2024-04-02T17:10:35.046798+0000","last_unstale":"2024-04-02T17:11:52.818544+0000","last_undegraded":"2024-04-02T17:11:52.818544+0000","last_fullsized":"2024-04-02T17:11:52.818544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:08:35.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429577+0000","last_change":"2024-04-02T17:10:35.051966+0000","last_active":"2024-04-02T17:11:52.429577+0000","last_peered":"2024-04-02T17:11:52.429577+0000","last_clean":"2024-04-02T17:11:52.429577+0000","last_became_active":"2024-04-02T17:10:35.051640+0000","last_became_peered":"2024-04-02T17:10:35.051640+0000","last_unstale":"2024-04-02T17:11:52.429577+0000","last_undegraded":"2024-04-02T17:11:52.429577+0000","last_fullsized":"2024-04-02T17:11:52.429577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:11.965483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430790+0000","last_change":"2024-04-02T17:10:35.042747+0000","last_active":"2024-04-02T17:11:52.430790+0000","last_peered":"2024-04-02T17:11:52.430790+0000","last_clean":"2024-04-02T17:11:52.430790+0000","last_became_active":"2024-04-02T17:10:35.042428+0000","last_became_peered":"2024-04-02T17:10:35.042428+0000","last_unstale":"2024-04-02T17:11:52.430790+0000","last_undegraded":"2024-04-02T17:11:52.430790+0000","last_fullsized":"2024-04-02T17:11:52.430790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:10.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222683+0000","last_change":"2024-04-02T17:10:35.039030+0000","last_active":"2024-04-02T17:11:52.222683+0000","last_peered":"2024-04-02T17:11:52.222683+0000","last_clean":"2024-04-02T17:11:52.222683+0000","last_became_active":"2024-04-02T17:10:35.038243+0000","last_became_peered":"2024-04-02T17:10:35.038243+0000","last_unstale":"2024-04-02T17:11:52.222683+0000","last_undegraded":"2024-04-02T17:11:52.222683+0000","last_fullsized":"2024-04-02T17:11:52.222683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:00.781761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219697+0000","last_change":"2024-04-02T17:10:35.043394+0000","last_active":"2024-04-02T17:11:52.219697+0000","last_peered":"2024-04-02T17:11:52.219697+0000","last_clean":"2024-04-02T17:11:52.219697+0000","last_became_active":"2024-04-02T17:10:35.043097+0000","last_became_peered":"2024-04-02T17:10:35.043097+0000","last_unstale":"2024-04-02T17:11:52.219697+0000","last_undegraded":"2024-04-02T17:11:52.219697+0000","last_fullsized":"2024-04-02T17:11:52.219697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:16.642697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.383706+0000","last_change":"2024-04-02T17:10:34.037213+0000","last_active":"2024-04-02T17:12:55.383706+0000","last_peered":"2024-04-02T17:12:55.383706+0000","last_clean":"2024-04-02T17:12:55.383706+0000","last_became_active":"2024-04-02T17:10:34.035353+0000","last_became_peered":"2024-04-02T17:10:34.035353+0000","last_unstale":"2024-04-02T17:12:55.383706+0000","last_undegraded":"2024-04-02T17:12:55.383706+0000","last_fullsized":"2024-04-02T17:12:55.383706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:03.748545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232279+0000","last_change":"2024-04-02T17:10:35.041293+0000","last_active":"2024-04-02T17:11:52.232279+0000","last_peered":"2024-04-02T17:11:52.232279+0000","last_clean":"2024-04-02T17:11:52.232279+0000","last_became_active":"2024-04-02T17:10:35.033700+0000","last_became_peered":"2024-04-02T17:10:35.033700+0000","last_unstale":"2024-04-02T17:11:52.232279+0000","last_undegraded":"2024-04-02T17:11:52.232279+0000","last_fullsized":"2024-04-02T17:11:52.232279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:15.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232788+0000","last_change":"2024-04-02T17:10:34.050134+0000","last_active":"2024-04-02T17:11:52.232788+0000","last_peered":"2024-04-02T17:11:52.232788+0000","last_clean":"2024-04-02T17:11:52.232788+0000","last_became_active":"2024-04-02T17:10:34.049790+0000","last_became_peered":"2024-04-02T17:10:34.049790+0000","last_unstale":"2024-04-02T17:11:52.232788+0000","last_undegraded":"2024-04-02T17:11:52.232788+0000","last_fullsized":"2024-04-02T17:11:52.232788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.941736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818995+0000","last_change":"2024-04-02T17:10:35.041149+0000","last_active":"2024-04-02T17:11:52.818995+0000","last_peered":"2024-04-02T17:11:52.818995+0000","last_clean":"2024-04-02T17:11:52.818995+0000","last_became_active":"2024-04-02T17:10:35.040146+0000","last_became_peered":"2024-04-02T17:10:35.040146+0000","last_unstale":"2024-04-02T17:11:52.818995+0000","last_undegraded":"2024-04-02T17:11:52.818995+0000","last_fullsized":"2024-04-02T17:11:52.818995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:13:44.061213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430291+0000","last_change":"2024-04-02T17:10:51.330798+0000","last_active":"2024-04-02T17:11:52.430291+0000","last_peered":"2024-04-02T17:11:52.430291+0000","last_clean":"2024-04-02T17:11:52.430291+0000","last_became_active":"2024-04-02T17:10:51.318708+0000","last_became_peered":"2024-04-02T17:10:51.318708+0000","last_unstale":"2024-04-02T17:11:52.430291+0000","last_undegraded":"2024-04-02T17:11:52.430291+0000","last_fullsized":"2024-04-02T17:11:52.430291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:08:25.035375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429776+0000","last_change":"2024-04-02T17:10:35.037742+0000","last_active":"2024-04-02T17:11:52.429776+0000","last_peered":"2024-04-02T17:11:52.429776+0000","last_clean":"2024-04-02T17:11:52.429776+0000","last_became_active":"2024-04-02T17:10:35.037117+0000","last_became_peered":"2024-04-02T17:10:35.037117+0000","last_unstale":"2024-04-02T17:11:52.429776+0000","last_undegraded":"2024-04-02T17:11:52.429776+0000","last_fullsized":"2024-04-02T17:11:52.429776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:01:04.828576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430934+0000","last_change":"2024-04-02T17:10:34.023628+0000","last_active":"2024-04-02T17:11:52.430934+0000","last_peered":"2024-04-02T17:11:52.430934+0000","last_clean":"2024-04-02T17:11:52.430934+0000","last_became_active":"2024-04-02T17:10:34.020803+0000","last_became_peered":"2024-04-02T17:10:34.020803+0000","last_unstale":"2024-04-02T17:11:52.430934+0000","last_undegraded":"2024-04-02T17:11:52.430934+0000","last_fullsized":"2024-04-02T17:11:52.430934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:48:59.338574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221978+0000","last_change":"2024-04-02T17:10:51.634458+0000","last_active":"2024-04-02T17:11:52.221978+0000","last_peered":"2024-04-02T17:11:52.221978+0000","last_clean":"2024-04-02T17:11:52.221978+0000","last_became_active":"2024-04-02T17:10:51.634064+0000","last_became_peered":"2024-04-02T17:10:51.634064+0000","last_unstale":"2024-04-02T17:11:52.221978+0000","last_undegraded":"2024-04-02T17:11:52.221978+0000","last_fullsized":"2024-04-02T17:11:52.221978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:10.010493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:54.296307+0000","last_change":"2024-04-02T17:10:34.021690+0000","last_active":"2024-04-02T17:12:54.296307+0000","last_peered":"2024-04-02T17:12:54.296307+0000","last_clean":"2024-04-02T17:12:54.296307+0000","last_became_active":"2024-04-02T17:10:34.021338+0000","last_became_peered":"2024-04-02T17:10:34.021338+0000","last_unstale":"2024-04-02T17:12:54.296307+0000","last_undegraded":"2024-04-02T17:12:54.296307+0000","last_fullsized":"2024-04-02T17:12:54.296307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:14.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430521+0000","last_change":"2024-04-02T17:10:35.037412+0000","last_active":"2024-04-02T17:11:52.430521+0000","last_peered":"2024-04-02T17:11:52.430521+0000","last_clean":"2024-04-02T17:11:52.430521+0000","last_became_active":"2024-04-02T17:10:35.037122+0000","last_became_peered":"2024-04-02T17:10:35.037122+0000","last_unstale":"2024-04-02T17:11:52.430521+0000","last_undegraded":"2024-04-02T17:11:52.430521+0000","last_fullsized":"2024-04-02T17:11:52.430521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:17:55.401725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225449+0000","last_change":"2024-04-02T17:10:34.022292+0000","last_active":"2024-04-02T17:11:52.225449+0000","last_peered":"2024-04-02T17:11:52.225449+0000","last_clean":"2024-04-02T17:11:52.225449+0000","last_became_active":"2024-04-02T17:10:34.021988+0000","last_became_peered":"2024-04-02T17:10:34.021988+0000","last_unstale":"2024-04-02T17:11:52.225449+0000","last_undegraded":"2024-04-02T17:11:52.225449+0000","last_fullsized":"2024-04-02T17:11:52.225449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:53:50.196150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221833+0000","last_change":"2024-04-02T17:10:35.030480+0000","last_active":"2024-04-02T17:11:52.221833+0000","last_peered":"2024-04-02T17:11:52.221833+0000","last_clean":"2024-04-02T17:11:52.221833+0000","last_became_active":"2024-04-02T17:10:35.029374+0000","last_became_peered":"2024-04-02T17:10:35.029374+0000","last_unstale":"2024-04-02T17:11:52.221833+0000","last_undegraded":"2024-04-02T17:11:52.221833+0000","last_fullsized":"2024-04-02T17:11:52.221833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:05.932393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235557+0000","last_change":"2024-04-02T17:10:34.035401+0000","last_active":"2024-04-02T17:11:52.235557+0000","last_peered":"2024-04-02T17:11:52.235557+0000","last_clean":"2024-04-02T17:11:52.235557+0000","last_became_active":"2024-04-02T17:10:34.031765+0000","last_became_peered":"2024-04-02T17:10:34.031765+0000","last_unstale":"2024-04-02T17:11:52.235557+0000","last_undegraded":"2024-04-02T17:11:52.235557+0000","last_fullsized":"2024-04-02T17:11:52.235557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.073927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230277+0000","last_change":"2024-04-02T17:10:35.038510+0000","last_active":"2024-04-02T17:11:52.230277+0000","last_peered":"2024-04-02T17:11:52.230277+0000","last_clean":"2024-04-02T17:11:52.230277+0000","last_became_active":"2024-04-02T17:10:35.038135+0000","last_became_peered":"2024-04-02T17:10:35.038135+0000","last_unstale":"2024-04-02T17:11:52.230277+0000","last_undegraded":"2024-04-02T17:11:52.230277+0000","last_fullsized":"2024-04-02T17:11:52.230277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:15:15.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221584+0000","last_change":"2024-04-02T17:10:34.025314+0000","last_active":"2024-04-02T17:11:52.221584+0000","last_peered":"2024-04-02T17:11:52.221584+0000","last_clean":"2024-04-02T17:11:52.221584+0000","last_became_active":"2024-04-02T17:10:34.024688+0000","last_became_peered":"2024-04-02T17:10:34.024688+0000","last_unstale":"2024-04-02T17:11:52.221584+0000","last_undegraded":"2024-04-02T17:11:52.221584+0000","last_fullsized":"2024-04-02T17:11:52.221584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:55:04.747881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221849+0000","last_change":"2024-04-02T17:10:35.046103+0000","last_active":"2024-04-02T17:11:52.221849+0000","last_peered":"2024-04-02T17:11:52.221849+0000","last_clean":"2024-04-02T17:11:52.221849+0000","last_became_active":"2024-04-02T17:10:35.045463+0000","last_became_peered":"2024-04-02T17:10:35.045463+0000","last_unstale":"2024-04-02T17:11:52.221849+0000","last_undegraded":"2024-04-02T17:11:52.221849+0000","last_fullsized":"2024-04-02T17:11:52.221849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:55:58.212302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.299066+0000","last_change":"2024-04-02T17:10:34.033128+0000","last_active":"2024-04-02T17:12:57.299066+0000","last_peered":"2024-04-02T17:12:57.299066+0000","last_clean":"2024-04-02T17:12:57.299066+0000","last_became_active":"2024-04-02T17:10:34.032839+0000","last_became_peered":"2024-04-02T17:10:34.032839+0000","last_unstale":"2024-04-02T17:12:57.299066+0000","last_undegraded":"2024-04-02T17:12:57.299066+0000","last_fullsized":"2024-04-02T17:12:57.299066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T05:08:55.698672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430074+0000","last_change":"2024-04-02T17:10:35.035883+0000","last_active":"2024-04-02T17:11:52.430074+0000","last_peered":"2024-04-02T17:11:52.430074+0000","last_clean":"2024-04-02T17:11:52.430074+0000","last_became_active":"2024-04-02T17:10:35.035555+0000","last_became_peered":"2024-04-02T17:10:35.035555+0000","last_unstale":"2024-04-02T17:11:52.430074+0000","last_undegraded":"2024-04-02T17:11:52.430074+0000","last_fullsized":"2024-04-02T17:11:52.430074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:57.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224087+0000","last_change":"2024-04-02T17:10:34.021567+0000","last_active":"2024-04-02T17:11:52.224087+0000","last_peered":"2024-04-02T17:11:52.224087+0000","last_clean":"2024-04-02T17:11:52.224087+0000","last_became_active":"2024-04-02T17:10:34.021216+0000","last_became_peered":"2024-04-02T17:10:34.021216+0000","last_unstale":"2024-04-02T17:11:52.224087+0000","last_undegraded":"2024-04-02T17:11:52.224087+0000","last_fullsized":"2024-04-02T17:11:52.224087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:38:53.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819045+0000","last_change":"2024-04-02T17:10:35.042246+0000","last_active":"2024-04-02T17:11:52.819045+0000","last_peered":"2024-04-02T17:11:52.819045+0000","last_clean":"2024-04-02T17:11:52.819045+0000","last_became_active":"2024-04-02T17:10:35.041610+0000","last_became_peered":"2024-04-02T17:10:35.041610+0000","last_unstale":"2024-04-02T17:11:52.819045+0000","last_undegraded":"2024-04-02T17:11:52.819045+0000","last_fullsized":"2024-04-02T17:11:52.819045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:04:56.800303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222125+0000","last_change":"2024-04-02T17:10:34.036697+0000","last_active":"2024-04-02T17:11:52.222125+0000","last_peered":"2024-04-02T17:11:52.222125+0000","last_clean":"2024-04-02T17:11:52.222125+0000","last_became_active":"2024-04-02T17:10:34.036368+0000","last_became_peered":"2024-04-02T17:10:34.036368+0000","last_unstale":"2024-04-02T17:11:52.222125+0000","last_undegraded":"2024-04-02T17:11:52.222125+0000","last_fullsized":"2024-04-02T17:11:52.222125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:45:31.965118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429946+0000","last_change":"2024-04-02T17:10:35.036308+0000","last_active":"2024-04-02T17:11:52.429946+0000","last_peered":"2024-04-02T17:11:52.429946+0000","last_clean":"2024-04-02T17:11:52.429946+0000","last_became_active":"2024-04-02T17:10:35.034922+0000","last_became_peered":"2024-04-02T17:10:35.034922+0000","last_unstale":"2024-04-02T17:11:52.429946+0000","last_undegraded":"2024-04-02T17:11:52.429946+0000","last_fullsized":"2024-04-02T17:11:52.429946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:24:01.677041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":340,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.300323+0000","last_change":"2024-04-02T17:10:34.021804+0000","last_active":"2024-04-02T17:12:57.300323+0000","last_peered":"2024-04-02T17:12:57.300323+0000","last_clean":"2024-04-02T17:12:57.300323+0000","last_became_active":"2024-04-02T17:10:34.021042+0000","last_became_peered":"2024-04-02T17:10:34.021042+0000","last_unstale":"2024-04-02T17:12:57.300323+0000","last_undegraded":"2024-04-02T17:12:57.300323+0000","last_fullsized":"2024-04-02T17:12:57.300323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:49:18.101830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67824,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":207,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430928+0000","last_change":"2024-04-02T17:10:35.029965+0000","last_active":"2024-04-02T17:11:52.430928+0000","last_peered":"2024-04-02T17:11:52.430928+0000","last_clean":"2024-04-02T17:11:52.430928+0000","last_became_active":"2024-04-02T17:10:35.029685+0000","last_became_peered":"2024-04-02T17:10:35.029685+0000","last_unstale":"2024-04-02T17:11:52.430928+0000","last_undegraded":"2024-04-02T17:11:52.430928+0000","last_fullsized":"2024-04-02T17:11:52.430928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:00.170714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219640+0000","last_change":"2024-04-02T17:10:34.042132+0000","last_active":"2024-04-02T17:11:52.219640+0000","last_peered":"2024-04-02T17:11:52.219640+0000","last_clean":"2024-04-02T17:11:52.219640+0000","last_became_active":"2024-04-02T17:10:34.041809+0000","last_became_peered":"2024-04-02T17:10:34.041809+0000","last_unstale":"2024-04-02T17:11:52.219640+0000","last_undegraded":"2024-04-02T17:11:52.219640+0000","last_fullsized":"2024-04-02T17:11:52.219640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:42:19.982301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235862+0000","last_change":"2024-04-02T17:10:35.036642+0000","last_active":"2024-04-02T17:11:52.235862+0000","last_peered":"2024-04-02T17:11:52.235862+0000","last_clean":"2024-04-02T17:11:52.235862+0000","last_became_active":"2024-04-02T17:10:35.035104+0000","last_became_peered":"2024-04-02T17:10:35.035104+0000","last_unstale":"2024-04-02T17:11:52.235862+0000","last_undegraded":"2024-04-02T17:11:52.235862+0000","last_fullsized":"2024-04-02T17:11:52.235862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:14.316380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430181+0000","last_change":"2024-04-02T17:10:51.336678+0000","last_active":"2024-04-02T17:11:52.430181+0000","last_peered":"2024-04-02T17:11:52.430181+0000","last_clean":"2024-04-02T17:11:52.430181+0000","last_became_active":"2024-04-02T17:10:51.325028+0000","last_became_peered":"2024-04-02T17:10:51.325028+0000","last_unstale":"2024-04-02T17:11:52.430181+0000","last_undegraded":"2024-04-02T17:11:52.430181+0000","last_fullsized":"2024-04-02T17:11:52.430181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:54:29.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224346+0000","last_change":"2024-04-02T17:10:35.045552+0000","last_active":"2024-04-02T17:11:52.224346+0000","last_peered":"2024-04-02T17:11:52.224346+0000","last_clean":"2024-04-02T17:11:52.224346+0000","last_became_active":"2024-04-02T17:10:35.045144+0000","last_became_peered":"2024-04-02T17:10:35.045144+0000","last_unstale":"2024-04-02T17:11:52.224346+0000","last_undegraded":"2024-04-02T17:11:52.224346+0000","last_fullsized":"2024-04-02T17:11:52.224346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:00.938943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:55.399180+0000","last_change":"2024-04-02T17:10:34.024699+0000","last_active":"2024-04-02T17:12:55.399180+0000","last_peered":"2024-04-02T17:12:55.399180+0000","last_clean":"2024-04-02T17:12:55.399180+0000","last_became_active":"2024-04-02T17:10:34.024314+0000","last_became_peered":"2024-04-02T17:10:34.024314+0000","last_unstale":"2024-04-02T17:12:55.399180+0000","last_undegraded":"2024-04-02T17:12:55.399180+0000","last_fullsized":"2024-04-02T17:12:55.399180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:28:42.836096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223641+0000","last_change":"2024-04-02T17:10:35.040982+0000","last_active":"2024-04-02T17:11:52.223641+0000","last_peered":"2024-04-02T17:11:52.223641+0000","last_clean":"2024-04-02T17:11:52.223641+0000","last_became_active":"2024-04-02T17:10:35.040466+0000","last_became_peered":"2024-04-02T17:10:35.040466+0000","last_unstale":"2024-04-02T17:11:52.223641+0000","last_undegraded":"2024-04-02T17:11:52.223641+0000","last_fullsized":"2024-04-02T17:11:52.223641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:49.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430477+0000","last_change":"2024-04-02T17:10:34.033167+0000","last_active":"2024-04-02T17:11:52.430477+0000","last_peered":"2024-04-02T17:11:52.430477+0000","last_clean":"2024-04-02T17:11:52.430477+0000","last_became_active":"2024-04-02T17:10:34.032668+0000","last_became_peered":"2024-04-02T17:10:34.032668+0000","last_unstale":"2024-04-02T17:11:52.430477+0000","last_undegraded":"2024-04-02T17:11:52.430477+0000","last_fullsized":"2024-04-02T17:11:52.430477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:48.681633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222202+0000","last_change":"2024-04-02T17:10:35.038828+0000","last_active":"2024-04-02T17:11:52.222202+0000","last_peered":"2024-04-02T17:11:52.222202+0000","last_clean":"2024-04-02T17:11:52.222202+0000","last_became_active":"2024-04-02T17:10:35.038521+0000","last_became_peered":"2024-04-02T17:10:35.038521+0000","last_unstale":"2024-04-02T17:11:52.222202+0000","last_undegraded":"2024-04-02T17:11:52.222202+0000","last_fullsized":"2024-04-02T17:11:52.222202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:05.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431210+0000","last_change":"2024-04-02T17:10:34.021522+0000","last_active":"2024-04-02T17:11:52.431210+0000","last_peered":"2024-04-02T17:11:52.431210+0000","last_clean":"2024-04-02T17:11:52.431210+0000","last_became_active":"2024-04-02T17:10:34.021203+0000","last_became_peered":"2024-04-02T17:10:34.021203+0000","last_unstale":"2024-04-02T17:11:52.431210+0000","last_undegraded":"2024-04-02T17:11:52.431210+0000","last_fullsized":"2024-04-02T17:11:52.431210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:00:02.427617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430525+0000","last_change":"2024-04-02T17:10:51.321193+0000","last_active":"2024-04-02T17:11:52.430525+0000","last_peered":"2024-04-02T17:11:52.430525+0000","last_clean":"2024-04-02T17:11:52.430525+0000","last_became_active":"2024-04-02T17:10:51.320791+0000","last_became_peered":"2024-04-02T17:10:51.320791+0000","last_unstale":"2024-04-02T17:11:52.430525+0000","last_undegraded":"2024-04-02T17:11:52.430525+0000","last_fullsized":"2024-04-02T17:11:52.430525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:22:18.979764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222639+0000","last_change":"2024-04-02T17:10:34.038883+0000","last_active":"2024-04-02T17:11:52.222639+0000","last_peered":"2024-04-02T17:11:52.222639+0000","last_clean":"2024-04-02T17:11:52.222639+0000","last_became_active":"2024-04-02T17:10:34.037427+0000","last_became_peered":"2024-04-02T17:10:34.037427+0000","last_unstale":"2024-04-02T17:11:52.222639+0000","last_undegraded":"2024-04-02T17:11:52.222639+0000","last_fullsized":"2024-04-02T17:11:52.222639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:06:28.903646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.243589+0000","last_change":"2024-04-02T17:11:52.243152+0000","last_active":"2024-04-02T17:11:52.243589+0000","last_peered":"2024-04-02T17:11:52.243589+0000","last_clean":"2024-04-02T17:11:52.243589+0000","last_became_active":"2024-04-02T17:11:52.232947+0000","last_became_peered":"2024-04-02T17:11:52.232947+0000","last_unstale":"2024-04-02T17:11:52.243589+0000","last_undegraded":"2024-04-02T17:11:52.243589+0000","last_fullsized":"2024-04-02T17:11:52.243589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:49:58.858519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:56.386744+0000","last_change":"2024-04-02T17:10:34.022770+0000","last_active":"2024-04-02T17:12:56.386744+0000","last_peered":"2024-04-02T17:12:56.386744+0000","last_clean":"2024-04-02T17:12:56.386744+0000","last_became_active":"2024-04-02T17:10:34.022234+0000","last_became_peered":"2024-04-02T17:10:34.022234+0000","last_unstale":"2024-04-02T17:12:56.386744+0000","last_undegraded":"2024-04-02T17:12:56.386744+0000","last_fullsized":"2024-04-02T17:12:56.386744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:13:25.183925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.225702+0000","last_change":"2024-04-02T17:10:35.042284+0000","last_active":"2024-04-02T17:11:52.225702+0000","last_peered":"2024-04-02T17:11:52.225702+0000","last_clean":"2024-04-02T17:11:52.225702+0000","last_became_active":"2024-04-02T17:10:35.041970+0000","last_became_peered":"2024-04-02T17:10:35.041970+0000","last_unstale":"2024-04-02T17:11:52.225702+0000","last_undegraded":"2024-04-02T17:11:52.225702+0000","last_fullsized":"2024-04-02T17:11:52.225702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:59:56.968359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221789+0000","last_change":"2024-04-02T17:10:35.031360+0000","last_active":"2024-04-02T17:11:52.221789+0000","last_peered":"2024-04-02T17:11:52.221789+0000","last_clean":"2024-04-02T17:11:52.221789+0000","last_became_active":"2024-04-02T17:10:35.030839+0000","last_became_peered":"2024-04-02T17:10:35.030839+0000","last_unstale":"2024-04-02T17:11:52.221789+0000","last_undegraded":"2024-04-02T17:11:52.221789+0000","last_fullsized":"2024-04-02T17:11:52.221789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:16.062169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:57.402458+0000","last_change":"2024-04-02T17:10:34.045047+0000","last_active":"2024-04-02T17:12:57.402458+0000","last_peered":"2024-04-02T17:12:57.402458+0000","last_clean":"2024-04-02T17:12:57.402458+0000","last_became_active":"2024-04-02T17:10:34.043176+0000","last_became_peered":"2024-04-02T17:10:34.043176+0000","last_unstale":"2024-04-02T17:12:57.402458+0000","last_undegraded":"2024-04-02T17:12:57.402458+0000","last_fullsized":"2024-04-02T17:12:57.402458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:32.579548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222174+0000","last_change":"2024-04-02T17:10:51.633808+0000","last_active":"2024-04-02T17:11:52.222174+0000","last_peered":"2024-04-02T17:11:52.222174+0000","last_clean":"2024-04-02T17:11:52.222174+0000","last_became_active":"2024-04-02T17:10:51.633095+0000","last_became_peered":"2024-04-02T17:10:51.633095+0000","last_unstale":"2024-04-02T17:11:52.222174+0000","last_undegraded":"2024-04-02T17:11:52.222174+0000","last_fullsized":"2024-04-02T17:11:52.222174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:36:16.308694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222251+0000","last_change":"2024-04-02T17:10:34.032408+0000","last_active":"2024-04-02T17:11:52.222251+0000","last_peered":"2024-04-02T17:11:52.222251+0000","last_clean":"2024-04-02T17:11:52.222251+0000","last_became_active":"2024-04-02T17:10:34.032048+0000","last_became_peered":"2024-04-02T17:10:34.032048+0000","last_unstale":"2024-04-02T17:11:52.222251+0000","last_undegraded":"2024-04-02T17:11:52.222251+0000","last_fullsized":"2024-04-02T17:11:52.222251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:54.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223736+0000","last_change":"2024-04-02T17:09:15.321740+0000","last_active":"2024-04-02T17:11:52.223736+0000","last_peered":"2024-04-02T17:11:52.223736+0000","last_clean":"2024-04-02T17:11:52.223736+0000","last_became_active":"2024-04-02T17:09:15.321148+0000","last_became_peered":"2024-04-02T17:09:15.321148+0000","last_unstale":"2024-04-02T17:11:52.223736+0000","last_undegraded":"2024-04-02T17:11:52.223736+0000","last_fullsized":"2024-04-02T17:11:52.223736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:05:30.096344+0000","last_clean_scrub_stamp":"2024-04-02T17:05:30.096344+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:39:47.927926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.235203+0000","last_change":"2024-04-02T17:10:35.044366+0000","last_active":"2024-04-02T17:11:52.235203+0000","last_peered":"2024-04-02T17:11:52.235203+0000","last_clean":"2024-04-02T17:11:52.235203+0000","last_became_active":"2024-04-02T17:10:35.043832+0000","last_became_peered":"2024-04-02T17:10:35.043832+0000","last_unstale":"2024-04-02T17:11:52.235203+0000","last_undegraded":"2024-04-02T17:11:52.235203+0000","last_fullsized":"2024-04-02T17:11:52.235203+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:38:04.843371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.223856+0000","last_change":"2024-04-02T17:10:51.637846+0000","last_active":"2024-04-02T17:11:52.223856+0000","last_peered":"2024-04-02T17:11:52.223856+0000","last_clean":"2024-04-02T17:11:52.223856+0000","last_became_active":"2024-04-02T17:10:51.633829+0000","last_became_peered":"2024-04-02T17:10:51.633829+0000","last_unstale":"2024-04-02T17:11:52.223856+0000","last_undegraded":"2024-04-02T17:11:52.223856+0000","last_fullsized":"2024-04-02T17:11:52.223856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:10:45.177246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.431026+0000","last_change":"2024-04-02T17:10:34.036754+0000","last_active":"2024-04-02T17:11:52.431026+0000","last_peered":"2024-04-02T17:11:52.431026+0000","last_clean":"2024-04-02T17:11:52.431026+0000","last_became_active":"2024-04-02T17:10:34.036388+0000","last_became_peered":"2024-04-02T17:10:34.036388+0000","last_unstale":"2024-04-02T17:11:52.431026+0000","last_undegraded":"2024-04-02T17:11:52.431026+0000","last_fullsized":"2024-04-02T17:11:52.431026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:45:03.881090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237144+0000","last_change":"2024-04-02T17:10:35.045872+0000","last_active":"2024-04-02T17:11:52.237144+0000","last_peered":"2024-04-02T17:11:52.237144+0000","last_clean":"2024-04-02T17:11:52.237144+0000","last_became_active":"2024-04-02T17:10:35.045211+0000","last_became_peered":"2024-04-02T17:10:35.045211+0000","last_unstale":"2024-04-02T17:11:52.237144+0000","last_undegraded":"2024-04-02T17:11:52.237144+0000","last_fullsized":"2024-04-02T17:11:52.237144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:20:14.290975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:47.323048+0000","last_change":"2024-04-02T17:10:34.048378+0000","last_active":"2024-04-02T17:12:47.323048+0000","last_peered":"2024-04-02T17:12:47.323048+0000","last_clean":"2024-04-02T17:12:47.323048+0000","last_became_active":"2024-04-02T17:10:34.048033+0000","last_became_peered":"2024-04-02T17:10:34.048033+0000","last_unstale":"2024-04-02T17:12:47.323048+0000","last_undegraded":"2024-04-02T17:12:47.323048+0000","last_fullsized":"2024-04-02T17:12:47.323048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:54.737747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.233095+0000","last_change":"2024-04-02T17:10:35.037835+0000","last_active":"2024-04-02T17:11:52.233095+0000","last_peered":"2024-04-02T17:11:52.233095+0000","last_clean":"2024-04-02T17:11:52.233095+0000","last_became_active":"2024-04-02T17:10:35.031263+0000","last_became_peered":"2024-04-02T17:10:35.031263+0000","last_unstale":"2024-04-02T17:11:52.233095+0000","last_undegraded":"2024-04-02T17:11:52.233095+0000","last_fullsized":"2024-04-02T17:11:52.233095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:48:49.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.222419+0000","last_change":"2024-04-02T17:10:34.018541+0000","last_active":"2024-04-02T17:11:52.222419+0000","last_peered":"2024-04-02T17:11:52.222419+0000","last_clean":"2024-04-02T17:11:52.222419+0000","last_became_active":"2024-04-02T17:10:34.018230+0000","last_became_peered":"2024-04-02T17:10:34.018230+0000","last_unstale":"2024-04-02T17:11:52.222419+0000","last_undegraded":"2024-04-02T17:11:52.222419+0000","last_fullsized":"2024-04-02T17:11:52.222419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:14.956634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.818278+0000","last_change":"2024-04-02T17:10:35.052806+0000","last_active":"2024-04-02T17:11:52.818278+0000","last_peered":"2024-04-02T17:11:52.818278+0000","last_clean":"2024-04-02T17:11:52.818278+0000","last_became_active":"2024-04-02T17:10:35.052456+0000","last_became_peered":"2024-04-02T17:10:35.052456+0000","last_unstale":"2024-04-02T17:11:52.818278+0000","last_undegraded":"2024-04-02T17:11:52.818278+0000","last_fullsized":"2024-04-02T17:11:52.818278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:55:52.977319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429640+0000","last_change":"2024-04-02T17:10:34.036602+0000","last_active":"2024-04-02T17:11:52.429640+0000","last_peered":"2024-04-02T17:11:52.429640+0000","last_clean":"2024-04-02T17:11:52.429640+0000","last_became_active":"2024-04-02T17:10:34.036240+0000","last_became_peered":"2024-04-02T17:10:34.036240+0000","last_unstale":"2024-04-02T17:11:52.429640+0000","last_undegraded":"2024-04-02T17:11:52.429640+0000","last_fullsized":"2024-04-02T17:11:52.429640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:55:36.357128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430857+0000","last_change":"2024-04-02T17:10:35.031099+0000","last_active":"2024-04-02T17:11:52.430857+0000","last_peered":"2024-04-02T17:11:52.430857+0000","last_clean":"2024-04-02T17:11:52.430857+0000","last_became_active":"2024-04-02T17:10:35.030764+0000","last_became_peered":"2024-04-02T17:10:35.030764+0000","last_unstale":"2024-04-02T17:11:52.430857+0000","last_undegraded":"2024-04-02T17:11:52.430857+0000","last_fullsized":"2024-04-02T17:11:52.430857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:34:32.907887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230823+0000","last_change":"2024-04-02T17:10:34.048851+0000","last_active":"2024-04-02T17:11:52.230823+0000","last_peered":"2024-04-02T17:11:52.230823+0000","last_clean":"2024-04-02T17:11:52.230823+0000","last_became_active":"2024-04-02T17:10:34.047621+0000","last_became_peered":"2024-04-02T17:10:34.047621+0000","last_unstale":"2024-04-02T17:11:52.230823+0000","last_undegraded":"2024-04-02T17:11:52.230823+0000","last_fullsized":"2024-04-02T17:11:52.230823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:39.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430110+0000","last_change":"2024-04-02T17:10:35.034161+0000","last_active":"2024-04-02T17:11:52.430110+0000","last_peered":"2024-04-02T17:11:52.430110+0000","last_clean":"2024-04-02T17:11:52.430110+0000","last_became_active":"2024-04-02T17:10:35.033867+0000","last_became_peered":"2024-04-02T17:10:35.033867+0000","last_unstale":"2024-04-02T17:11:52.430110+0000","last_undegraded":"2024-04-02T17:11:52.430110+0000","last_fullsized":"2024-04-02T17:11:52.430110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:46.595013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219529+0000","last_change":"2024-04-02T17:10:34.042237+0000","last_active":"2024-04-02T17:11:52.219529+0000","last_peered":"2024-04-02T17:11:52.219529+0000","last_clean":"2024-04-02T17:11:52.219529+0000","last_became_active":"2024-04-02T17:10:34.041822+0000","last_became_peered":"2024-04-02T17:10:34.041822+0000","last_unstale":"2024-04-02T17:11:52.219529+0000","last_undegraded":"2024-04-02T17:11:52.219529+0000","last_fullsized":"2024-04-02T17:11:52.219529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:29.979111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232232+0000","last_change":"2024-04-02T17:10:35.035233+0000","last_active":"2024-04-02T17:11:52.232232+0000","last_peered":"2024-04-02T17:11:52.232232+0000","last_clean":"2024-04-02T17:11:52.232232+0000","last_became_active":"2024-04-02T17:10:35.034079+0000","last_became_peered":"2024-04-02T17:10:35.034079+0000","last_unstale":"2024-04-02T17:11:52.232232+0000","last_undegraded":"2024-04-02T17:11:52.232232+0000","last_fullsized":"2024-04-02T17:11:52.232232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:31:14.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430747+0000","last_change":"2024-04-02T17:10:51.641235+0000","last_active":"2024-04-02T17:11:52.430747+0000","last_peered":"2024-04-02T17:11:52.430747+0000","last_clean":"2024-04-02T17:11:52.430747+0000","last_became_active":"2024-04-02T17:10:51.634715+0000","last_became_peered":"2024-04-02T17:10:51.634715+0000","last_unstale":"2024-04-02T17:11:52.430747+0000","last_undegraded":"2024-04-02T17:11:52.430747+0000","last_fullsized":"2024-04-02T17:11:52.430747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:56:38.764506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224004+0000","last_change":"2024-04-02T17:10:35.045177+0000","last_active":"2024-04-02T17:11:52.224004+0000","last_peered":"2024-04-02T17:11:52.224004+0000","last_clean":"2024-04-02T17:11:52.224004+0000","last_became_active":"2024-04-02T17:10:35.044692+0000","last_became_peered":"2024-04-02T17:10:35.044692+0000","last_unstale":"2024-04-02T17:11:52.224004+0000","last_undegraded":"2024-04-02T17:11:52.224004+0000","last_fullsized":"2024-04-02T17:11:52.224004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:27.453081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:12:53.293909+0000","last_change":"2024-04-02T17:10:34.036786+0000","last_active":"2024-04-02T17:12:53.293909+0000","last_peered":"2024-04-02T17:12:53.293909+0000","last_clean":"2024-04-02T17:12:53.293909+0000","last_became_active":"2024-04-02T17:10:34.036434+0000","last_became_peered":"2024-04-02T17:10:34.036434+0000","last_unstale":"2024-04-02T17:12:53.293909+0000","last_undegraded":"2024-04-02T17:12:53.293909+0000","last_fullsized":"2024-04-02T17:12:53.293909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:02.395298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224498+0000","last_change":"2024-04-02T17:10:35.035937+0000","last_active":"2024-04-02T17:11:52.224498+0000","last_peered":"2024-04-02T17:11:52.224498+0000","last_clean":"2024-04-02T17:11:52.224498+0000","last_became_active":"2024-04-02T17:10:35.035595+0000","last_became_peered":"2024-04-02T17:10:35.035595+0000","last_unstale":"2024-04-02T17:11:52.224498+0000","last_undegraded":"2024-04-02T17:11:52.224498+0000","last_fullsized":"2024-04-02T17:11:52.224498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:56.986218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221897+0000","last_change":"2024-04-02T17:10:34.041177+0000","last_active":"2024-04-02T17:11:52.221897+0000","last_peered":"2024-04-02T17:11:52.221897+0000","last_clean":"2024-04-02T17:11:52.221897+0000","last_became_active":"2024-04-02T17:10:34.040877+0000","last_became_peered":"2024-04-02T17:10:34.040877+0000","last_unstale":"2024-04-02T17:11:52.221897+0000","last_undegraded":"2024-04-02T17:11:52.221897+0000","last_fullsized":"2024-04-02T17:11:52.221897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:35.673332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224105+0000","last_change":"2024-04-02T17:10:35.045572+0000","last_active":"2024-04-02T17:11:52.224105+0000","last_peered":"2024-04-02T17:11:52.224105+0000","last_clean":"2024-04-02T17:11:52.224105+0000","last_became_active":"2024-04-02T17:10:35.045257+0000","last_became_peered":"2024-04-02T17:10:35.045257+0000","last_unstale":"2024-04-02T17:11:52.224105+0000","last_undegraded":"2024-04-02T17:11:52.224105+0000","last_fullsized":"2024-04-02T17:11:52.224105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:22.579188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236455+0000","last_change":"2024-04-02T17:10:51.642752+0000","last_active":"2024-04-02T17:11:52.236455+0000","last_peered":"2024-04-02T17:11:52.236455+0000","last_clean":"2024-04-02T17:11:52.236455+0000","last_became_active":"2024-04-02T17:10:51.635734+0000","last_became_peered":"2024-04-02T17:10:51.635734+0000","last_unstale":"2024-04-02T17:11:52.236455+0000","last_undegraded":"2024-04-02T17:11:52.236455+0000","last_fullsized":"2024-04-02T17:11:52.236455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:32.983077+0000","last_clean_scrub_stamp":"2024-04-02T17:10:32.983077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.358529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.224156+0000","last_change":"2024-04-02T17:10:35.032622+0000","last_active":"2024-04-02T17:11:52.224156+0000","last_peered":"2024-04-02T17:11:52.224156+0000","last_clean":"2024-04-02T17:11:52.224156+0000","last_became_active":"2024-04-02T17:10:35.032326+0000","last_became_peered":"2024-04-02T17:10:35.032326+0000","last_unstale":"2024-04-02T17:11:52.224156+0000","last_undegraded":"2024-04-02T17:11:52.224156+0000","last_fullsized":"2024-04-02T17:11:52.224156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:49:43.894069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430173+0000","last_change":"2024-04-02T17:10:35.036783+0000","last_active":"2024-04-02T17:11:52.430173+0000","last_peered":"2024-04-02T17:11:52.430173+0000","last_clean":"2024-04-02T17:11:52.430173+0000","last_became_active":"2024-04-02T17:10:35.036481+0000","last_became_peered":"2024-04-02T17:10:35.036481+0000","last_unstale":"2024-04-02T17:11:52.430173+0000","last_undegraded":"2024-04-02T17:11:52.430173+0000","last_fullsized":"2024-04-02T17:11:52.430173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:30:27.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.237295+0000","last_change":"2024-04-02T17:11:52.236694+0000","last_active":"2024-04-02T17:11:52.237295+0000","last_peered":"2024-04-02T17:11:52.237295+0000","last_clean":"2024-04-02T17:11:52.237295+0000","last_became_active":"2024-04-02T17:11:52.236128+0000","last_became_peered":"2024-04-02T17:11:52.236128+0000","last_unstale":"2024-04-02T17:11:52.237295+0000","last_undegraded":"2024-04-02T17:11:52.237295+0000","last_fullsized":"2024-04-02T17:11:52.237295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:40.601150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.219390+0000","last_change":"2024-04-02T17:10:35.040935+0000","last_active":"2024-04-02T17:11:52.219390+0000","last_peered":"2024-04-02T17:11:52.219390+0000","last_clean":"2024-04-02T17:11:52.219390+0000","last_became_active":"2024-04-02T17:10:35.040621+0000","last_became_peered":"2024-04-02T17:10:35.040621+0000","last_unstale":"2024-04-02T17:11:52.219390+0000","last_undegraded":"2024-04-02T17:11:52.219390+0000","last_fullsized":"2024-04-02T17:11:52.219390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:09.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.819694+0000","last_change":"2024-04-02T17:10:51.332185+0000","last_active":"2024-04-02T17:11:52.819694+0000","last_peered":"2024-04-02T17:11:52.819694+0000","last_clean":"2024-04-02T17:11:52.819694+0000","last_became_active":"2024-04-02T17:10:51.331547+0000","last_became_peered":"2024-04-02T17:10:51.331547+0000","last_unstale":"2024-04-02T17:11:52.819694+0000","last_undegraded":"2024-04-02T17:11:52.819694+0000","last_fullsized":"2024-04-02T17:11:52.819694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:43.766907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236916+0000","last_change":"2024-04-02T17:10:35.043320+0000","last_active":"2024-04-02T17:11:52.236916+0000","last_peered":"2024-04-02T17:11:52.236916+0000","last_clean":"2024-04-02T17:11:52.236916+0000","last_became_active":"2024-04-02T17:10:35.042806+0000","last_became_peered":"2024-04-02T17:10:35.042806+0000","last_unstale":"2024-04-02T17:11:52.236916+0000","last_undegraded":"2024-04-02T17:11:52.236916+0000","last_fullsized":"2024-04-02T17:11:52.236916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:10.718923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430480+0000","last_change":"2024-04-02T17:10:35.036779+0000","last_active":"2024-04-02T17:11:52.430480+0000","last_peered":"2024-04-02T17:11:52.430480+0000","last_clean":"2024-04-02T17:11:52.430480+0000","last_became_active":"2024-04-02T17:10:35.036435+0000","last_became_peered":"2024-04-02T17:10:35.036435+0000","last_unstale":"2024-04-02T17:11:52.430480+0000","last_undegraded":"2024-04-02T17:11:52.430480+0000","last_fullsized":"2024-04-02T17:11:52.430480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:51:37.946535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.220006+0000","last_change":"2024-04-02T17:10:35.031132+0000","last_active":"2024-04-02T17:11:52.220006+0000","last_peered":"2024-04-02T17:11:52.220006+0000","last_clean":"2024-04-02T17:11:52.220006+0000","last_became_active":"2024-04-02T17:10:35.030820+0000","last_became_peered":"2024-04-02T17:10:35.030820+0000","last_unstale":"2024-04-02T17:11:52.220006+0000","last_undegraded":"2024-04-02T17:11:52.220006+0000","last_fullsized":"2024-04-02T17:11:52.220006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:14:06.602771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.232637+0000","last_change":"2024-04-02T17:10:35.038243+0000","last_active":"2024-04-02T17:11:52.232637+0000","last_peered":"2024-04-02T17:11:52.232637+0000","last_clean":"2024-04-02T17:11:52.232637+0000","last_became_active":"2024-04-02T17:10:35.037801+0000","last_became_peered":"2024-04-02T17:10:35.037801+0000","last_unstale":"2024-04-02T17:11:52.232637+0000","last_undegraded":"2024-04-02T17:11:52.232637+0000","last_fullsized":"2024-04-02T17:11:52.232637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:25.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.221764+0000","last_change":"2024-04-02T17:10:35.037318+0000","last_active":"2024-04-02T17:11:52.221764+0000","last_peered":"2024-04-02T17:11:52.221764+0000","last_clean":"2024-04-02T17:11:52.221764+0000","last_became_active":"2024-04-02T17:10:35.036941+0000","last_became_peered":"2024-04-02T17:10:35.036941+0000","last_unstale":"2024-04-02T17:11:52.221764+0000","last_undegraded":"2024-04-02T17:11:52.221764+0000","last_fullsized":"2024-04-02T17:11:52.221764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:49:25.727996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.230748+0000","last_change":"2024-04-02T17:10:35.037685+0000","last_active":"2024-04-02T17:11:52.230748+0000","last_peered":"2024-04-02T17:11:52.230748+0000","last_clean":"2024-04-02T17:11:52.230748+0000","last_became_active":"2024-04-02T17:10:35.037247+0000","last_became_peered":"2024-04-02T17:10:35.037247+0000","last_unstale":"2024-04-02T17:11:52.230748+0000","last_undegraded":"2024-04-02T17:11:52.230748+0000","last_fullsized":"2024-04-02T17:11:52.230748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:09:02.700213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.429720+0000","last_change":"2024-04-02T17:10:35.047349+0000","last_active":"2024-04-02T17:11:52.429720+0000","last_peered":"2024-04-02T17:11:52.429720+0000","last_clean":"2024-04-02T17:11:52.429720+0000","last_became_active":"2024-04-02T17:10:35.047006+0000","last_became_peered":"2024-04-02T17:10:35.047006+0000","last_unstale":"2024-04-02T17:11:52.429720+0000","last_undegraded":"2024-04-02T17:11:52.429720+0000","last_fullsized":"2024-04-02T17:11:52.429720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:22:40.684267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.236708+0000","last_change":"2024-04-02T17:10:35.044912+0000","last_active":"2024-04-02T17:11:52.236708+0000","last_peered":"2024-04-02T17:11:52.236708+0000","last_clean":"2024-04-02T17:11:52.236708+0000","last_became_active":"2024-04-02T17:10:35.039677+0000","last_became_peered":"2024-04-02T17:10:35.039677+0000","last_unstale":"2024-04-02T17:11:52.236708+0000","last_undegraded":"2024-04-02T17:11:52.236708+0000","last_fullsized":"2024-04-02T17:11:52.236708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:33:34.023865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-02T17:11:52.430273+0000","last_change":"2024-04-02T17:10:35.035014+0000","last_active":"2024-04-02T17:11:52.430273+0000","last_peered":"2024-04-02T17:11:52.430273+0000","last_clean":"2024-04-02T17:11:52.430273+0000","last_became_active":"2024-04-02T17:10:35.034703+0000","last_became_peered":"2024-04-02T17:10:35.034703+0000","last_unstale":"2024-04-02T17:11:52.430273+0000","last_undegraded":"2024-04-02T17:11:52.430273+0000","last_fullsized":"2024-04-02T17:11:52.430273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T17:10:33.988240+0000","last_clean_scrub_stamp":"2024-04-02T17:10:33.988240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:55.037269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76148,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1383,"num_read_kb":777,"num_write":258,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":228444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":485886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579954,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1144649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":485311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":485572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1079069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":484911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":553648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":486782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":549865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":484619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":485304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1642496,"data_stored":1078125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T17:13:01.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:13:01.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 -- 172.21.15.59:0/2537801097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0102d10 msgr2=0x7f5af0103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2537801097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0102d10 0x7f5af0103190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009900 tx=0x7f5adc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 -- 172.21.15.59:0/2537801097 shutdown_connections 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2537801097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af0109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2537801097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0102d10 0x7f5af0103190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2537801097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0101b20 0x7f5af0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 -- 172.21.15.59:0/2537801097 >> 172.21.15.59:0/2537801097 conn(0x7f5af00fd0b0 msgr2=0x7f5af00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 -- 172.21.15.59:0/2537801097 shutdown_connections 2024-04-02T17:13:01.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.304+0000 7f5af4e00640 1 -- 172.21.15.59:0/2537801097 wait complete. 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 Processor -- start 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 -- start start 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0102d10 0x7f5af019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010d7f0 con 0x7f5af0102d10 2024-04-02T17:13:01.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010d670 con 0x7f5af01036d0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.305+0000 7f5af4e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5af010d970 con 0x7f5af0101b20 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aede00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48936/0 (socket says 172.21.15.59:48936) 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 -- 172.21.15.59:0/2740954762 learned_addr learned my addr 172.21.15.59:0/2740954762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aede00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59268/0 (socket says 172.21.15.59:59268) 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aed400640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0102d10 0x7f5af019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 -- 172.21.15.59:0/2740954762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 msgr2=0x7f5af019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 -- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0102d10 msgr2=0x7f5af019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0102d10 0x7f5af019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 -- 172.21.15.59:0/2740954762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af01a1c40 con 0x7f5af01036d0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aede00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.306+0000 7f5aee800640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800eb10 tx=0x7f5ae800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:01.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.307+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8004b20 con 0x7f5af01036d0 2024-04-02T17:13:01.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.307+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01a1f30 con 0x7f5af01036d0 2024-04-02T17:13:01.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.307+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01a2440 con 0x7f5af01036d0 2024-04-02T17:13:01.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.307+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8004cc0 con 0x7f5af01036d0 2024-04-02T17:13:01.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.307+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8010690 con 0x7f5af01036d0 2024-04-02T17:13:01.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.308+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af01a26d0 con 0x7f5af01036d0 2024-04-02T17:13:01.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.310+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ae8005000 con 0x7f5af01036d0 2024-04-02T17:13:01.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.311+0000 7f5ae6a00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 0x7f5ad40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.312+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5ae80d0a20 con 0x7f5af01036d0 2024-04-02T17:13:01.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.312+0000 7f5aede00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 0x7f5ad40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.313+0000 7f5aede00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 0x7f5ad40a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5af0102b70 tx=0x7f5ae0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.316+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5ae809d340 con 0x7f5af01036d0 2024-04-02T17:13:01.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.528+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5af01a28a0 con 0x7f5af01036d0 2024-04-02T17:13:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.530+0000 7f5ae6a00640 1 -- 172.21.15.59:0/2740954762 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7f5ae809cc40 con 0x7f5af01036d0 2024-04-02T17:13:01.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:01.531 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:13:01.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.532+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 msgr2=0x7f5ad40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.532+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 0x7f5ad40a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5af0102b70 tx=0x7f5ae0005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:01.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.532+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 msgr2=0x7f5af01a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.532+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800eb10 tx=0x7f5ae800c2d0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 shutdown_connections 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5ad40a3bb0 0x7f5ad40a6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af0102d10 0x7f5af019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5af01036d0 0x7f5af01a15d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 --2- 172.21.15.59:0/2740954762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af0101b20 0x7f5af019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 >> 172.21.15.59:0/2740954762 conn(0x7f5af00fd0b0 msgr2=0x7f5af0073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 shutdown_connections 2024-04-02T17:13:01.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.533+0000 7f5af4e00640 1 -- 172.21.15.59:0/2740954762 wait complete. 2024-04-02T17:13:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:01 smithi059 ceph-mon[31664]: pgmap v386: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:01 smithi059 ceph-mon[31664]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:13:01.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:13:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:01 smithi164 ceph-mon[36499]: pgmap v386: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:01 smithi164 ceph-mon[36499]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:01 smithi154 ceph-mon[36479]: pgmap v386: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:01.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:01 smithi154 ceph-mon[36479]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:01.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 -- 172.21.15.59:0/2335254330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518101900 msgr2=0x7f1518101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 --2- 172.21.15.59:0/2335254330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518101900 0x7f1518101d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1508009900 tx=0x7f150802bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 -- 172.21.15.59:0/2335254330 shutdown_connections 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 --2- 172.21.15.59:0/2335254330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1518102af0 0x7f1518102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 --2- 172.21.15.59:0/2335254330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15181034b0 0x7f1518109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 --2- 172.21.15.59:0/2335254330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518101900 0x7f1518101d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 -- 172.21.15.59:0/2335254330 >> 172.21.15.59:0/2335254330 conn(0x7f15180fce90 msgr2=0x7f15180ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 -- 172.21.15.59:0/2335254330 shutdown_connections 2024-04-02T17:13:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.772+0000 7f151d200640 1 -- 172.21.15.59:0/2335254330 wait complete. 2024-04-02T17:13:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.773+0000 7f151d200640 1 Processor -- start 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.773+0000 7f151d200640 1 -- start start 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518102af0 0x7f151819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15181034b0 0x7f15181a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f151810d5a0 con 0x7f15181034b0 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f151810d420 con 0x7f1518102af0 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f151d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f151810d720 con 0x7f1518101900 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59286/0 (socket says 172.21.15.59:59286) 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15181034b0 0x7f15181a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 -- 172.21.15.59:0/2491468627 learned_addr learned my addr 172.21.15.59:0/2491468627 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1516a00640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518102af0 0x7f151819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 -- 172.21.15.59:0/2491468627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518102af0 msgr2=0x7f151819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518102af0 0x7f151819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 -- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15181034b0 msgr2=0x7f15181a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15181034b0 0x7f15181a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.774+0000 7f1517400640 1 -- 172.21.15.59:0/2491468627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181a1a10 con 0x7f1518101900 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f1517400640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1508000c00 tx=0x7f150802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508043020 con 0x7f1518101900 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15080054e0 con 0x7f1518101900 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15181a1ca0 con 0x7f1518101900 2024-04-02T17:13:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508003c40 con 0x7f1518101900 2024-04-02T17:13:01.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.775+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181a2190 con 0x7f1518101900 2024-04-02T17:13:01.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.777+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15181a2390 con 0x7f1518101900 2024-04-02T17:13:01.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.779+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1508046020 con 0x7f1518101900 2024-04-02T17:13:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.780+0000 7f150fe00640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 0x7f14fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.780+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f15080f2a70 con 0x7f1518101900 2024-04-02T17:13:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.780+0000 7f1516a00640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 0x7f14fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.781+0000 7f1516a00640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 0x7f14fc0a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f151819b300 tx=0x7f1504005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.782+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f150804d020 con 0x7f1518101900 2024-04-02T17:13:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.992+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15181a2560 con 0x7f1518101900 2024-04-02T17:13:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.993+0000 7f150fe00640 1 -- 172.21.15.59:0/2491468627 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f15080bf390 con 0x7f1518101900 2024-04-02T17:13:01.993 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:13:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 msgr2=0x7f14fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 0x7f14fc0a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f151819b300 tx=0x7f1504005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 msgr2=0x7f1518199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1508000c00 tx=0x7f150802c650 comp rx=0 tx=0).stop 2024-04-02T17:13:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 shutdown_connections 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f14fc0a3c30 0x7f14fc0a60f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15181034b0 0x7f15181a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1518102af0 0x7f151819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 --2- 172.21.15.59:0/2491468627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1518101900 0x7f1518199e90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 >> 172.21.15.59:0/2491468627 conn(0x7f15180fce90 msgr2=0x7f1518073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 shutdown_connections 2024-04-02T17:13:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:01.996+0000 7f151d200640 1 -- 172.21.15.59:0/2491468627 wait complete. 2024-04-02T17:13:02.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:13:02.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 -- 172.21.15.59:0/1002789526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f103810dfd0 msgr2=0x7f10381103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 --2- 172.21.15.59:0/1002789526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f103810dfd0 0x7f10381103c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1030009cd0 tx=0x7f103002c0e0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 -- 172.21.15.59:0/1002789526 shutdown_connections 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 --2- 172.21.15.59:0/1002789526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1038074560 0x7f10380749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 --2- 172.21.15.59:0/1002789526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f103810dfd0 0x7f10381103c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 --2- 172.21.15.59:0/1002789526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038075b40 0x7f1038073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 -- 172.21.15.59:0/1002789526 >> 172.21.15.59:0/1002789526 conn(0x7f10380fcf70 msgr2=0x7f10380ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 -- 172.21.15.59:0/1002789526 shutdown_connections 2024-04-02T17:13:02.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.240+0000 7f103da00640 1 -- 172.21.15.59:0/1002789526 wait complete. 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.241+0000 7f103da00640 1 Processor -- start 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.241+0000 7f103da00640 1 -- start start 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038074560 0x7f103819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103810dfd0 0x7f10381a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1038113c90 con 0x7f103810dfd0 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1038113b10 con 0x7f1038075b40 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1038113e10 con 0x7f1038074560 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f1037400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f1037400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48972/0 (socket says 172.21.15.59:48972) 2024-04-02T17:13:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f1037400640 1 -- 172.21.15.59:0/3321952242 learned_addr learned my addr 172.21.15.59:0/3321952242 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f1037e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038074560 0x7f103819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.242+0000 7f103d000640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103810dfd0 0x7f10381a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 -- 172.21.15.59:0/3321952242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038074560 msgr2=0x7f103819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038074560 0x7f103819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 -- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103810dfd0 msgr2=0x7f10381a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103810dfd0 0x7f10381a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 -- 172.21.15.59:0/3321952242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10381a5ea0 con 0x7f1038075b40 2024-04-02T17:13:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1037400640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9f0 tx=0x7f102400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400cd80 con 0x7f1038075b40 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024010910 con 0x7f1038075b40 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024018990 con 0x7f1038075b40 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10381a6190 con 0x7f1038075b40 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.243+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10381a6690 con 0x7f1038075b40 2024-04-02T17:13:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.245+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10381041a0 con 0x7f1038075b40 2024-04-02T17:13:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.248+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1024010ac0 con 0x7f1038075b40 2024-04-02T17:13:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.251+0000 7f1034c00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 0x7f101c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.251+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f10240d0940 con 0x7f1038075b40 2024-04-02T17:13:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.251+0000 7f1037e00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 0x7f101c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.251+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1024099280 con 0x7f1038075b40 2024-04-02T17:13:02.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.252+0000 7f1037e00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 0x7f101c0a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1028009bd0 tx=0x7f1028005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:02.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.508+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f103802d5c0 con 0x7f1038075b40 2024-04-02T17:13:02.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.509+0000 7f1034c00640 1 -- 172.21.15.59:0/3321952242 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f102409cc10 con 0x7f1038075b40 2024-04-02T17:13:02.509 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 msgr2=0x7f101c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 0x7f101c0a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1028009bd0 tx=0x7f1028005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 msgr2=0x7f103819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9f0 tx=0x7f102400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 shutdown_connections 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f101c0a3c30 0x7f101c0a60f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103810dfd0 0x7f10381a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1038075b40 0x7f103819e7c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 --2- 172.21.15.59:0/3321952242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1038074560 0x7f103819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 >> 172.21.15.59:0/3321952242 conn(0x7f10380fcf70 msgr2=0x7f1038101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.511+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 shutdown_connections 2024-04-02T17:13:02.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.512+0000 7f103da00640 1 -- 172.21.15.59:0/3321952242 wait complete. 2024-04-02T17:13:02.534 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:13:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:02 smithi164 ceph-mon[36499]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:02 smithi164 ceph-mon[36499]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:02 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2740954762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:02.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:02 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2491468627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:13:02.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:02 smithi059 ceph-mon[31664]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:02 smithi059 ceph-mon[31664]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:02 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2740954762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:02.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:02 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2491468627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:13:02.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:02 smithi154 ceph-mon[36479]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:02 smithi154 ceph-mon[36479]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T17:13:02.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:02 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2740954762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:02.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:02 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2491468627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.760+0000 7f2f13e00640 1 -- 172.21.15.59:0/1828776449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f141035b0 msgr2=0x7f2f14109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.760+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1828776449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f141035b0 0x7f2f14109e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009860 tx=0x7f2f0c02bb70 comp rx=0 tx=0).stop 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 -- 172.21.15.59:0/1828776449 shutdown_connections 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1828776449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f141035b0 0x7f2f14109e40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1828776449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f14102bf0 0x7f2f14103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1828776449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14101a00 0x7f2f14101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 -- 172.21.15.59:0/1828776449 >> 172.21.15.59:0/1828776449 conn(0x7f2f140fcf70 msgr2=0x7f2f140ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:02.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 -- 172.21.15.59:0/1828776449 shutdown_connections 2024-04-02T17:13:02.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.761+0000 7f2f13e00640 1 -- 172.21.15.59:0/1828776449 wait complete. 2024-04-02T17:13:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.762+0000 7f2f13e00640 1 Processor -- start 2024-04-02T17:13:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.762+0000 7f2f13e00640 1 -- start start 2024-04-02T17:13:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.762+0000 7f2f13e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.762+0000 7f2f13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 0x7f2f14073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f13e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f141035b0 0x7f2f14074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f13e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410d660 con 0x7f2f14102bf0 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f13e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410d4e0 con 0x7f2f141035b0 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f13e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410d7e0 con 0x7f2f14101a00 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 0x7f2f14073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59320/0 (socket says 172.21.15.59:59320) 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 -- 172.21.15.59:0/1412346764 learned_addr learned my addr 172.21.15.59:0/1412346764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f13400640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f141035b0 0x7f2f14074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 -- 172.21.15.59:0/1412346764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f141035b0 msgr2=0x7f2f14074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f141035b0 0x7f2f14074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 -- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 msgr2=0x7f2f14073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 0x7f2f14073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12a00640 1 -- 172.21.15.59:0/1412346764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f14074cc0 con 0x7f2f14101a00 2024-04-02T17:13:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.763+0000 7f2f12000640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 0x7f2f14073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f12a00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00ccc0 con 0x7f2f14101a00 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f141a1d90 con 0x7f2f14101a00 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2efc0035f0 con 0x7f2f14101a00 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00ce60 con 0x7f2f14101a00 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.764+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141a2200 con 0x7f2f14101a00 2024-04-02T17:13:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.765+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14108d20 con 0x7f2f14101a00 2024-04-02T17:13:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.767+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2efc0037a0 con 0x7f2f14101a00 2024-04-02T17:13:02.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.769+0000 7f2f0b400640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 0x7f2ef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:02.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.769+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2efc0d0810 con 0x7f2f14101a00 2024-04-02T17:13:02.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.770+0000 7f2f12000640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 0x7f2ef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:02.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.770+0000 7f2f12000640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 0x7f2ef80a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f14075620 tx=0x7f2f04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.771+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2efc09d130 con 0x7f2f14101a00 2024-04-02T17:13:02.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.977+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f1402d730 con 0x7f2f14101a00 2024-04-02T17:13:02.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.979+0000 7f2f0b400640 1 -- 172.21.15.59:0/1412346764 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f2efc09ca30 con 0x7f2f14101a00 2024-04-02T17:13:02.978 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 msgr2=0x7f2ef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 0x7f2ef80a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f14075620 tx=0x7f2f04009500 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 msgr2=0x7f2f140758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00e9f0 tx=0x7f2efc00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 shutdown_connections 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ef80a3bb0 0x7f2ef80a6070 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f14102bf0 0x7f2f14073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f141035b0 0x7f2f14074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 --2- 172.21.15.59:0/1412346764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f14101a00 0x7f2f140758b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 >> 172.21.15.59:0/1412346764 conn(0x7f2f140fcf70 msgr2=0x7f2f14105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 shutdown_connections 2024-04-02T17:13:02.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:02.981+0000 7f2f13e00640 1 -- 172.21.15.59:0/1412346764 wait complete. 2024-04-02T17:13:03.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 -- 172.21.15.59:0/3597690508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 msgr2=0x7fc3781051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 --2- 172.21.15.59:0/3597690508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc3781051c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc364009900 tx=0x7fc36402bc40 comp rx=0 tx=0).stop 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 -- 172.21.15.59:0/3597690508 shutdown_connections 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 --2- 172.21.15.59:0/3597690508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc3781051c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 --2- 172.21.15.59:0/3597690508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378105700 0x7fc37810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 --2- 172.21.15.59:0/3597690508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378103b50 0x7fc378103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 -- 172.21.15.59:0/3597690508 >> 172.21.15.59:0/3597690508 conn(0x7fc3780ff0e0 msgr2=0x7fc378101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 -- 172.21.15.59:0/3597690508 shutdown_connections 2024-04-02T17:13:03.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.226+0000 7fc37cc00640 1 -- 172.21.15.59:0/3597690508 wait complete. 2024-04-02T17:13:03.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.227+0000 7fc37cc00640 1 Processor -- start 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.227+0000 7fc37cc00640 1 -- start start 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378103b50 0x7fc378199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc37819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810f800 con 0x7fc378104d40 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810f680 con 0x7fc378103b50 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc37cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810f980 con 0x7fc378105700 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc376a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc37819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc377e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc377400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378103b50 0x7fc378199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc377e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59336/0 (socket says 172.21.15.59:59336) 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.228+0000 7fc377e00640 1 -- 172.21.15.59:0/2189649776 learned_addr learned my addr 172.21.15.59:0/2189649776 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 -- 172.21.15.59:0/2189649776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378103b50 msgr2=0x7fc378199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378103b50 0x7fc378199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 -- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 msgr2=0x7fc37819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc37819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 -- 172.21.15.59:0/2189649776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3781a1a30 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc377e00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc37000d750 tx=0x7fc37000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370017070 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc370005500 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3781a1d20 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370004d90 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.229+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3781a2190 con 0x7fc378105700 2024-04-02T17:13:03.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.231+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37810ae70 con 0x7fc378105700 2024-04-02T17:13:03.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.233+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc370004f30 con 0x7fc378105700 2024-04-02T17:13:03.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.234+0000 7fc36fe00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 0x7fc35c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.234+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc3700cfdf0 con 0x7fc378105700 2024-04-02T17:13:03.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.235+0000 7fc377400640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 0x7fc35c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.236+0000 7fc377400640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 0x7fc35c0a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc378104ba0 tx=0x7fc368009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:03.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.239+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc37009c0c0 con 0x7fc378105700 2024-04-02T17:13:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.488+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3780627a0 con 0x7fc378105700 2024-04-02T17:13:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.490+0000 7fc36fe00640 1 -- 172.21.15.59:0/2189649776 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7fc3700a03c0 con 0x7fc378105700 2024-04-02T17:13:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:03.490 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:12:54.846637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24529,24530,24547,24560,24584]},"id":1}]} 2024-04-02T17:13:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 msgr2=0x7fc35c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 0x7fc35c0a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc378104ba0 tx=0x7fc368009500 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 msgr2=0x7fc3781a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc37000d750 tx=0x7fc37000dc10 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 shutdown_connections 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc35c0a3bb0 0x7fc35c0a6070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc378104d40 0x7fc37819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc378105700 0x7fc3781a13c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 --2- 172.21.15.59:0/2189649776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc378103b50 0x7fc378199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 >> 172.21.15.59:0/2189649776 conn(0x7fc3780ff0e0 msgr2=0x7fc378108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.492+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 shutdown_connections 2024-04-02T17:13:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.493+0000 7fc37cc00640 1 -- 172.21.15.59:0/2189649776 wait complete. 2024-04-02T17:13:03.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:13:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: pgmap v387: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:13:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:13:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3321952242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:03 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1412346764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: pgmap v387: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3321952242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.590 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:03 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1412346764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: pgmap v387: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:13:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:13:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:13:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3321952242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:03 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1412346764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:13:03.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 -- 172.21.15.59:0/1231857002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4104f30 msgr2=0x7ff5d41053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/1231857002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4104f30 0x7ff5d41053b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009900 tx=0x7ff5bc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 -- 172.21.15.59:0/1231857002 shutdown_connections 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/1231857002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4104f30 0x7ff5d41053b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/1231857002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d41058f0 0x7ff5d410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/1231857002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4103d40 0x7ff5d4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 -- 172.21.15.59:0/1231857002 >> 172.21.15.59:0/1231857002 conn(0x7ff5d40ff2f0 msgr2=0x7ff5d4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 -- 172.21.15.59:0/1231857002 shutdown_connections 2024-04-02T17:13:03.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.731+0000 7ff5d9e00640 1 -- 172.21.15.59:0/1231857002 wait complete. 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.732+0000 7ff5d9e00640 1 Processor -- start 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.732+0000 7ff5d9e00640 1 -- start start 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 0x7ff5d41a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d410f9c0 con 0x7ff5d4103d40 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d410f840 con 0x7ff5d41058f0 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d410fb40 con 0x7ff5d4104f30 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59360/0 (socket says 172.21.15.59:59360) 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5ca000640 1 -- 172.21.15.59:0/4053345199 learned_addr learned my addr 172.21.15.59:0/4053345199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44904/0 (socket says 172.21.15.59:44904) 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d3400640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 0x7ff5d41a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 -- 172.21.15.59:0/4053345199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 msgr2=0x7ff5d419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 -- 172.21.15.59:0/4053345199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 msgr2=0x7ff5d41a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 0x7ff5d41a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d2a00640 1 -- 172.21.15.59:0/4053345199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d41a1c80 con 0x7ff5d4103d40 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5ca000640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.733+0000 7ff5d3400640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 0x7ff5d41a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.734+0000 7ff5d2a00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.734+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4003970 con 0x7ff5d4103d40 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.734+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4004030 con 0x7ff5d4103d40 2024-04-02T17:13:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.734+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40107f0 con 0x7ff5d4103d40 2024-04-02T17:13:03.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.734+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d41a1f70 con 0x7ff5d4103d40 2024-04-02T17:13:03.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.735+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a2380 con 0x7ff5d4103d40 2024-04-02T17:13:03.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.736+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d410b060 con 0x7ff5d4103d40 2024-04-02T17:13:03.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.738+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff5c40041e0 con 0x7ff5d4103d40 2024-04-02T17:13:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.739+0000 7ff5d8a00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 0x7ff5b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.739+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff5c40d0c30 con 0x7ff5d4103d40 2024-04-02T17:13:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.739+0000 7ff5ca000640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 0x7ff5b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:03.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.740+0000 7ff5ca000640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 0x7ff5b80a60f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000c00 tx=0x7ff5bc02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.744+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff5c409d550 con 0x7ff5d4103d40 2024-04-02T17:13:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.951+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5d4062850 con 0x7ff5d4103d40 2024-04-02T17:13:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.956+0000 7ff5d8a00640 1 -- 172.21.15.59:0/4053345199 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22857 (secure 0 0 0) 0x7ff5c409ce50 con 0x7ff5d4103d40 2024-04-02T17:13:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:03.957 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:11:52.214481+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.958+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 msgr2=0x7ff5b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.958+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 0x7ff5b80a60f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000c00 tx=0x7ff5bc02c880 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.958+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 msgr2=0x7ff5d419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.958+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c400ef90 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.958+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 shutdown_connections 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d41058f0 0x7ff5d41a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5d4104f30 0x7ff5d419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5b80a3c30 0x7ff5b80a60f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 --2- 172.21.15.59:0/4053345199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4103d40 0x7ff5d419a060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 >> 172.21.15.59:0/4053345199 conn(0x7ff5d40ff2f0 msgr2=0x7ff5d4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:03.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 shutdown_connections 2024-04-02T17:13:03.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:03.959+0000 7ff5d9e00640 1 -- 172.21.15.59:0/4053345199 wait complete. 2024-04-02T17:13:03.980 DEBUG:tasks.fwd_scrub:Yielding 2024-04-02T17:13:03.980 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-02T17:13:03.991 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:13:03.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-02T17:13:03.992 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-02T17:13:03.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T17:13:03.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:13:03.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T17:13:03.999 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T17:13:04.021 INFO:tasks.workunit:Pulling workunits from ref 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:13:04.022 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T17:13:04.022 INFO:tasks.workunit:timeout=3h 2024-04-02T17:13:04.022 INFO:tasks.workunit:cleanup=True 2024-04-02T17:13:04.023 DEBUG:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T17:13:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T17:13:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-02 17:10:43.705953162 +0000 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-02 17:10:43.705953162 +0000 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-02 17:12:53.819723806 +0000 2024-04-02T17:13:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-02 17:10:43.705953162 +0000 2024-04-02T17:13:04.070 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T17:13:04.070 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T17:13:04.214 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 -- 172.21.15.59:0/4215237381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c101b10 msgr2=0x7f447c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 --2- 172.21.15.59:0/4215237381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c101b10 0x7f447c101f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4470009900 tx=0x7f447002bc40 comp rx=0 tx=0).stop 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 -- 172.21.15.59:0/4215237381 shutdown_connections 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 --2- 172.21.15.59:0/4215237381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 --2- 172.21.15.59:0/4215237381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c1036c0 0x7f447c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.213+0000 7f4484800640 1 --2- 172.21.15.59:0/4215237381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c101b10 0x7f447c101f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.214+0000 7f4484800640 1 -- 172.21.15.59:0/4215237381 >> 172.21.15.59:0/4215237381 conn(0x7f447c0fd0a0 msgr2=0x7f447c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.214+0000 7f4484800640 1 -- 172.21.15.59:0/4215237381 shutdown_connections 2024-04-02T17:13:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.214+0000 7f4484800640 1 -- 172.21.15.59:0/4215237381 wait complete. 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 Processor -- start 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 -- start start 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c101b10 0x7f447c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c1036c0 0x7f447c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d7e0 con 0x7f447c102d00 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d660 con 0x7f447c101b10 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4484800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10d960 con 0x7f447c1036c0 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.215+0000 7f4481400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44912/0 (socket says 172.21.15.59:44912) 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 -- 172.21.15.59:0/2956294171 learned_addr learned my addr 172.21.15.59:0/2956294171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c101b10 0x7f447c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4482800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c1036c0 0x7f447c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 -- 172.21.15.59:0/2956294171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c1036c0 msgr2=0x7f447c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c1036c0 0x7f447c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 -- 172.21.15.59:0/2956294171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c101b10 msgr2=0x7f447c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c101b10 0x7f447c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 -- 172.21.15.59:0/2956294171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1a1ce0 con 0x7f447c102d00 2024-04-02T17:13:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.216+0000 7f4481400640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f446c00e9f0 tx=0x7f446c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.217+0000 7f4476a00640 1 -- 172.21.15.59:0/2956294171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c00cd80 con 0x7f447c102d00 2024-04-02T17:13:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.217+0000 7f4476a00640 1 -- 172.21.15.59:0/2956294171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446c010910 con 0x7f447c102d00 2024-04-02T17:13:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.217+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1a1fd0 con 0x7f447c102d00 2024-04-02T17:13:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.217+0000 7f4476a00640 1 -- 172.21.15.59:0/2956294171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c018990 con 0x7f447c102d00 2024-04-02T17:13:04.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.217+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a24e0 con 0x7f447c102d00 2024-04-02T17:13:04.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.220+0000 7f4476a00640 1 -- 172.21.15.59:0/2956294171 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f446c010ac0 con 0x7f447c102d00 2024-04-02T17:13:04.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 Processor -- start 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 -- start start 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a7766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a77b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 0x55749a77e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55749a6db310 con 0x55749a7762b0 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55749a683690 con 0x55749a776ca0 2024-04-02T17:13:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.221+0000 7f448742a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55749a6d72c0 con 0x55749a77be10 2024-04-02T17:13:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4482800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a7766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 0x55749a77e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4482800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a7766d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44916/0 (socket says 172.21.15.59:44916) 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a77b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4482800640 1 -- 172.21.15.59:0/3572673195 learned_addr learned my addr 172.21.15.59:0/3572673195 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4476a00640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 0x7f44640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 -- 172.21.15.59:0/3572673195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 msgr2=0x55749a77e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4476a00640 1 -- 172.21.15.59:0/2956294171 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f446c0d0700 con 0x7f447c102d00 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 0x55749a77e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 -- 172.21.15.59:0/3572673195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 msgr2=0x55749a7766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a7766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481e00640 1 -- 172.21.15.59:0/3572673195 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749a4b77e0 con 0x55749a776ca0 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.222+0000 7f4481400640 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 0x55749a77e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.223+0000 7f4482800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 0x7f44640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.223+0000 7f4481e00640 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a77b7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4470009230 tx=0x7f4470000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b1c1b33d81e1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.223+0000 7f446b000640 1 -- 172.21.15.59:0/3572673195 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470042070 con 0x55749a776ca0 2024-04-02T17:13:04.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.223+0000 7f446b000640 1 -- 172.21.15.59:0/3572673195 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447002cb90 con 0x55749a776ca0 2024-04-02T17:13:04.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.223+0000 7f446b000640 1 -- 172.21.15.59:0/3572673195 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700046c0 con 0x55749a776ca0 2024-04-02T17:13:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 -- 172.21.15.59:0/3572673195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 msgr2=0x55749a77b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a77b7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4470009230 tx=0x7f4470000bc0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f4482800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 0x7f44640a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4478009fd0 tx=0x7f4478004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 -- 172.21.15.59:0/3572673195 shutdown_connections 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a77be10 0x55749a77e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a77b7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 --2- 172.21.15.59:0/3572673195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a7766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 -- 172.21.15.59:0/3572673195 >> 172.21.15.59:0/3572673195 conn(0x55749a70b270 msgr2=0x55749a771c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 -- 172.21.15.59:0/3572673195 shutdown_connections 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.224+0000 7f448742a740 1 -- 172.21.15.59:0/3572673195 wait complete. 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.225+0000 7f448742a740 1 -- start start 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.225+0000 7f448742a740 1 Processor -- start 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.225+0000 7f448742a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.225+0000 7f448742a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a780cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.225+0000 7f448742a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a787cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f448742a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55749a77ed60 con 0x55749a77be10 2024-04-02T17:13:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f448742a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55749a696a60 con 0x55749a776ca0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f448742a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55749a77eee0 con 0x55749a7762b0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f448742a740 10 client.0 _ll_register_callbacks cb 0x55749a69c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4482800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a787cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a780cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59392/0 (socket says 172.21.15.59:59392) 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 -- 172.21.15.59:0/3255260122 learned_addr learned my addr 172.21.15.59:0/3255260122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 -- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 msgr2=0x55749a780cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a780cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 -- 172.21.15.59:0/3255260122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 msgr2=0x55749a787cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a787cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.226+0000 7f4481e00640 1 -- 172.21.15.59:0/3255260122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749a517f70 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f4481e00640 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f447002c180 tx=0x7f4470034be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5d53ee28f0d36f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f445f400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470042040 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447002cb90 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4470005550 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4470059050 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f448742a740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.227+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55749a3af170 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447002cb90 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4470046400 con 0x55749a7762b0 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 20 client.25399 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a78c1e0 0x55749a78e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 4 client.25399 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:04.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55749a6e8850 con 0x55749a78c1e0 2024-04-02T17:13:04.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.228+0000 7f4481400640 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a78c1e0 0x55749a78e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.229+0000 7f4481400640 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a78c1e0 0x55749a78e5f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897e20e3494cc83 server_cookie=55e86f27c612f848 in_seq=0 out_seq=0 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.229+0000 7f445f400640 10 client.25399 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f445f400640 1 -- 172.21.15.59:0/3255260122 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55749a6e8850 con 0x55749a78c1e0 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f445f400640 10 client.25399 handle_command_reply: tid=0 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 client.25399 shutdown 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 msgr2=0x55749a780770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f447002c180 tx=0x7f4470034be0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 shutdown_connections 2024-04-02T17:13:04.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a78c1e0 0x55749a78e5f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a787cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f445f400640 0 client.25399 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a780cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 --2- 172.21.15.59:0/3255260122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a780770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 >> 172.21.15.59:0/3255260122 conn(0x55749a70b270 msgr2=0x55749a770e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 shutdown_connections 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 1 -- 172.21.15.59:0/3255260122 wait complete. 2024-04-02T17:13:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.230+0000 7f448742a740 20 client.25399 trim_cache size 0 max 16384 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.250+0000 7f448742a740 1 Processor -- start 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.250+0000 7f448742a740 1 -- start start 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a779150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a77be10 0x55749a779690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1ab370 con 0x55749a7762b0 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1aceb0 con 0x55749a77be10 2024-04-02T17:13:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f448742a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1aba40 con 0x55749a776ca0 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59396/0 (socket says 172.21.15.59:59396) 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481400640 1 -- 172.21.15.59:0/622838737 learned_addr learned my addr 172.21.15.59:0/622838737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481e00640 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a779150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4482800640 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a77be10 0x55749a779690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481400640 1 -- 172.21.15.59:0/622838737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a77be10 msgr2=0x55749a779690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.251+0000 7f4481400640 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a77be10 0x55749a779690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f4481400640 1 -- 172.21.15.59:0/622838737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 msgr2=0x55749a779150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f4481400640 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a779150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f4481400640 1 -- 172.21.15.59:0/622838737 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749a77eca0 con 0x55749a776ca0 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f4481400640 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f446c098e30 tx=0x7f446c0ce010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7161a3923ceefc65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f446a600640 1 -- 172.21.15.59:0/622838737 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0a0cd0 con 0x55749a776ca0 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f446a600640 1 -- 172.21.15.59:0/622838737 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446c0a5590 con 0x55749a776ca0 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.252+0000 7f446a600640 1 -- 172.21.15.59:0/622838737 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0d2e20 con 0x55749a776ca0 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- 172.21.15.59:0/622838737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 msgr2=0x55749a787cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f446c098e30 tx=0x7f446c0ce010 comp rx=0 tx=0).stop 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- 172.21.15.59:0/622838737 shutdown_connections 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a77be10 0x55749a779690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a776ca0 0x55749a787cb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 --2- 172.21.15.59:0/622838737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a7762b0 0x55749a779150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- 172.21.15.59:0/622838737 >> 172.21.15.59:0/622838737 conn(0x55749a70b270 msgr2=0x55749a792eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- 172.21.15.59:0/622838737 shutdown_connections 2024-04-02T17:13:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- 172.21.15.59:0/622838737 wait complete. 2024-04-02T17:13:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.253+0000 7f448742a740 1 -- start start 2024-04-02T17:13:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 Processor -- start 2024-04-02T17:13:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a787a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a786040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1ac7e0 con 0x55749a77be10 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1ac110 con 0x55749a776ca0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1a9350 con 0x55749a7762b0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f448742a740 10 client.0 _ll_register_callbacks cb 0x7f4478009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f4482800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f4482800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44956/0 (socket says 172.21.15.59:44956) 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f4481400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a786040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.254+0000 7f4482800640 1 -- 172.21.15.59:0/1553840516 learned_addr learned my addr 172.21.15.59:0/1553840516 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4481e00640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a787a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 -- 172.21.15.59:0/1553840516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 msgr2=0x55749a787a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a787a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 -- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 msgr2=0x55749a786040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a786040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 -- 172.21.15.59:0/1553840516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749a786b90 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4481400640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f4482800640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f447801f990 tx=0x7f4478004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0abb0c98d31dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.255+0000 7f445f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447801ed40 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4478010430 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f446c0a5590 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4478004b20 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 10 client.15291 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55749a7f3690 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446c0a5590 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4478033020 con 0x55749a77be10 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 10 client.15291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.256+0000 7f448742a740 20 client.15291 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a7f40a0 0x55749a7f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f448742a740 4 client.15291 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55749a6e2ba0 con 0x55749a7f40a0 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f4481e00640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a7f40a0 0x55749a7f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f4481e00640 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a7f40a0 0x55749a7f64c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e4bed0edc5887a server_cookie=b4a60c453049e315 in_seq=0 out_seq=0 2024-04-02T17:13:04.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.257+0000 7f445f400640 10 client.15291 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.258+0000 7f445f400640 1 -- 172.21.15.59:0/1553840516 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55749a6e2ba0 con 0x55749a7f40a0 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.258+0000 7f445f400640 10 client.15291 handle_command_reply: tid=0 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 client.15291 shutdown 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 msgr2=0x55749a786580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f447801f990 tx=0x7f4478004ab0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 shutdown_connections 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55749a7f40a0 0x55749a7f64c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55749a77be10 0x55749a786580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55749a776ca0 0x55749a786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 --2- 172.21.15.59:0/1553840516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55749a7762b0 0x55749a787a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f445f400640 0 client.15291 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 >> 172.21.15.59:0/1553840516 conn(0x55749a70b270 msgr2=0x55749a793740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 shutdown_connections 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 1 -- 172.21.15.59:0/1553840516 wait complete. 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.259+0000 7f448742a740 20 client.15291 trim_cache size 0 max 16384 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:13:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-02T17:13:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "feb5fe33-b426-4f8f-b863-67ee087a6f76", 2024-04-02T17:13:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-02T17:13:04.264 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:13:04.264 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 msgr2=0x7f44640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 0x7f44640a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4478009fd0 tx=0x7f4478004a40 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 msgr2=0x7f447c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f446c00e9f0 tx=0x7f446c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 shutdown_connections 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f44640a3c30 0x7f44640a60f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f447c102d00 0x7f447c19a600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c1036c0 0x7f447c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 --2- 172.21.15.59:0/2956294171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c101b10 0x7f447c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 >> 172.21.15.59:0/2956294171 conn(0x7f447c0fd0a0 msgr2=0x7f447c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 shutdown_connections 2024-04-02T17:13:04.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.260+0000 7f4484800640 1 -- 172.21.15.59:0/2956294171 wait complete. 2024-04-02T17:13:04.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:13:04.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 -- 172.21.15.59:0/3575978515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 msgr2=0x7fc604107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3575978515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc604107350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009900 tx=0x7fc5f002bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 -- 172.21.15.59:0/3575978515 shutdown_connections 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3575978515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc604107350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3575978515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 0x7fc604109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3575978515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc604104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 -- 172.21.15.59:0/3575978515 >> 172.21.15.59:0/3575978515 conn(0x7fc6040fbfa0 msgr2=0x7fc6040fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 -- 172.21.15.59:0/3575978515 shutdown_connections 2024-04-02T17:13:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.526+0000 7fc60ba00640 1 -- 172.21.15.59:0/3575978515 wait complete. 2024-04-02T17:13:04.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.528+0000 7fc60ba00640 1 Processor -- start 2024-04-02T17:13:04.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.528+0000 7fc60ba00640 1 -- start start 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc60419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc60419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 0x7fc6041a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d7e0 con 0x7fc604104f60 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d660 con 0x7fc604102630 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc60ba00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d960 con 0x7fc604107890 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc603e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc60419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc603e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc60419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44970/0 (socket says 172.21.15.59:44970) 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc603e00640 1 -- 172.21.15.59:0/3602259763 learned_addr learned my addr 172.21.15.59:0/3602259763 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc60419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.529+0000 7fc609a00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 0x7fc6041a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 -- 172.21.15.59:0/3602259763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 msgr2=0x7fc6041a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 0x7fc6041a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 -- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 msgr2=0x7fc60419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc60419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 -- 172.21.15.59:0/3602259763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6041a1c90 con 0x7fc604102630 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc60419a070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800ec80 tx=0x7fc5f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc601600640 1 -- 172.21.15.59:0/3602259763 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003b70 con 0x7fc604102630 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc601600640 1 -- 172.21.15.59:0/3602259763 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8003d10 con 0x7fc604102630 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc601600640 1 -- 172.21.15.59:0/3602259763 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8010a30 con 0x7fc604102630 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.530+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6041a1f80 con 0x7fc604102630 2024-04-02T17:13:04.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.531+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041a23f0 con 0x7fc604102630 2024-04-02T17:13:04.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.535+0000 7fc60e77e740 1 Processor -- start 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.535+0000 7fc60e77e740 1 -- start start 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc601600640 1 -- 172.21.15.59:0/3602259763 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc5f8003ec0 con 0x7fc604102630 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d54cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3cb4770 con 0x55d1d3d55330 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3c5c700 con 0x55d1d3d501c0 2024-04-02T17:13:04.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc60e77e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3cb0720 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc609a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc609000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d54cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc609a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59440/0 (socket says 172.21.15.59:59440) 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.536+0000 7fc609a00640 1 -- 172.21.15.59:0/2692141162 learned_addr learned my addr 172.21.15.59:0/2692141162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 -- 172.21.15.59:0/2692141162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 msgr2=0x55d1d3d54cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d54cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 -- 172.21.15.59:0/2692141162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 msgr2=0x55d1d3d577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 -- 172.21.15.59:0/2692141162 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d3a927d0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc609a00640 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009ca0 tx=0x7fc5fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b9deb464b322cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc601600640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 0x7fc5ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc601600640 1 -- 172.21.15.59:0/3602259763 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc5f80d0b10 con 0x7fc604102630 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc5f6000640 1 -- 172.21.15.59:0/2692141162 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc01e070 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc5f6000640 1 -- 172.21.15.59:0/2692141162 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc009e50 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.537+0000 7fc5f6000640 1 -- 172.21.15.59:0/2692141162 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00eaa0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 0x7fc5ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc609000640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 0x7fc5ec0a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8099700 tx=0x7fc5f80d44a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 -- 172.21.15.59:0/2692141162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 msgr2=0x55d1d3d4fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009ca0 tx=0x7fc5fc011810 comp rx=0 tx=0).stop 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 -- 172.21.15.59:0/2692141162 shutdown_connections 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d54cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2692141162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d4fbf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.538+0000 7fc60e77e740 1 -- 172.21.15.59:0/2692141162 >> 172.21.15.59:0/2692141162 conn(0x55d1d3d4ad40 msgr2=0x55d1d3d4b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.539+0000 7fc60e77e740 1 -- 172.21.15.59:0/2692141162 shutdown_connections 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.539+0000 7fc60e77e740 1 -- 172.21.15.59:0/2692141162 wait complete. 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.539+0000 7fc60e77e740 1 -- start start 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.539+0000 7fc60e77e740 1 Processor -- start 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d55330 0x55d1d3d61270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3d582e0 con 0x55d1d3d501c0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3c6fac0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1d3d58460 con 0x55d1d3d55330 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc60e77e740 10 client.0 _ll_register_callbacks cb 0x55d1d3c72180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc609a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc603e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc609a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:49108/0 (socket says 172.21.15.59:49108) 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc609a00640 1 -- 172.21.15.59:0/3379901343 learned_addr learned my addr 172.21.15.59:0/3379901343 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.540+0000 7fc609a00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d55330 0x55d1d3d61270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 -- 172.21.15.59:0/3379901343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d55330 msgr2=0x55d1d3d61270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d55330 0x55d1d3d61270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 -- 172.21.15.59:0/3379901343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 msgr2=0x55d1d3d5a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 -- 172.21.15.59:0/3379901343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d39c83f0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc603e00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc609a00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005780 tx=0x7fc5fc0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877c3e68bddbda39 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc5eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc01e040 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc5fc00ec30 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.541+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5fc009e50 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc00ee30 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc60e77e740 10 client.25406 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1d39880f0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc009e50 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc5fc019690 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc60e77e740 10 client.25406 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.542+0000 7fc60e77e740 10 client.25406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.543+0000 7fc60e77e740 20 client.25406 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.543+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3d65540 0x55d1d3d67930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.543+0000 7fc60e77e740 4 client.25406 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.543+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d3c55630 con 0x55d1d3d65540 2024-04-02T17:13:04.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.543+0000 7fc603e00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3d65540 0x55d1d3d67930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.544+0000 7fc603e00640 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3d65540 0x55d1d3d67930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8790bd18c09c892e server_cookie=920c0ed7e31733ca in_seq=0 out_seq=0 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.544+0000 7fc5eaa00640 10 client.25406 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.544+0000 7fc5eaa00640 1 -- 172.21.15.59:0/3379901343 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1d3c55630 con 0x55d1d3d65540 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.544+0000 7fc5eaa00640 10 client.25406 handle_command_reply: tid=0 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.544+0000 7fc60e77e740 1 client.25406 shutdown 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 msgr2=0x55d1d3d59d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005780 tx=0x7fc5fc0056d0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 shutdown_connections 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3d65540 0x55d1d3d67930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d55330 0x55d1d3d61270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 --2- 172.21.15.59:0/3379901343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59d30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc5eaa00640 0 client.25406 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 >> 172.21.15.59:0/3379901343 conn(0x55d1d3d4ad40 msgr2=0x55d1d3d49f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 shutdown_connections 2024-04-02T17:13:04.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 1 -- 172.21.15.59:0/3379901343 wait complete. 2024-04-02T17:13:04.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.545+0000 7fc60e77e740 20 client.25406 trim_cache size 0 max 16384 2024-04-02T17:13:04.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:04 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2189649776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:04.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:04 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/4053345199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 Processor -- start 2024-04-02T17:13:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 -- start start 2024-04-02T17:13:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041ab3c0 con 0x55d1d3d55330 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041acf00 con 0x55d1d3d501c0 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.565+0000 7fc60e77e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041aba90 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc609a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:49118/0 (socket says 172.21.15.59:49118) 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 -- 172.21.15.59:0/2143456151 learned_addr learned my addr 172.21.15.59:0/2143456151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc609a00640 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 -- 172.21.15.59:0/2143456151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 msgr2=0x55d1d3d59cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 -- 172.21.15.59:0/2143456151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 msgr2=0x55d1d3d5a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d5a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 -- 172.21.15.59:0/2143456151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d3d57c80 con 0x55d1d3d501c0 2024-04-02T17:13:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc609a00640 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.566+0000 7fc603e00640 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc5f002c170 tx=0x7fc5f002df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655705e8d331ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.567+0000 7fc5f5600640 1 -- 172.21.15.59:0/2143456151 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f004abd0 con 0x55d1d3d501c0 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.567+0000 7fc5f5600640 1 -- 172.21.15.59:0/2143456151 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f004ad70 con 0x55d1d3d501c0 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.567+0000 7fc5f5600640 1 -- 172.21.15.59:0/2143456151 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f004a850 con 0x55d1d3d501c0 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- 172.21.15.59:0/2143456151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 msgr2=0x55d1d3d61230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc5f002c170 tx=0x7fc5f002df90 comp rx=0 tx=0).stop 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- 172.21.15.59:0/2143456151 shutdown_connections 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d55330 0x55d1d3d5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d501c0 0x55d1d3d61230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 --2- 172.21.15.59:0/2143456151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d59cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- 172.21.15.59:0/2143456151 >> 172.21.15.59:0/2143456151 conn(0x55d1d3d4ad40 msgr2=0x55d1d3d51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- 172.21.15.59:0/2143456151 shutdown_connections 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- 172.21.15.59:0/2143456151 wait complete. 2024-04-02T17:13:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.568+0000 7fc60e77e740 1 -- start start 2024-04-02T17:13:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.569+0000 7fc60e77e740 1 Processor -- start 2024-04-02T17:13:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.569+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.569+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.569+0000 7fc60e77e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d55330 0x55d1d3dc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.569+0000 7fc60e77e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041ac830 con 0x55d1d3d501c0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc60e77e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041ac160 con 0x55d1d3d55330 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc60e77e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041a93a0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc60e77e740 10 client.0 _ll_register_callbacks cb 0x7fc5fc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59474/0 (socket says 172.21.15.59:59474) 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc603e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 -- 172.21.15.59:0/591894284 learned_addr learned my addr 172.21.15.59:0/591894284 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d55330 0x55d1d3dc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 -- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d55330 msgr2=0x55d1d3dc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d55330 0x55d1d3dc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 -- 172.21.15.59:0/591894284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 msgr2=0x55d1d3d5e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.570+0000 7fc609a00640 1 -- 172.21.15.59:0/591894284 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d3dc9fa0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc609a00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc019ae0 tx=0x7fc5fc009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29870c191c3af81f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc5eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc01e040 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc5fc019630 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f004ad70 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc019830 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc60e77e740 10 client.25423 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.571+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1d3dcb1d0 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f004ad70 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc5fc028430 con 0x55d1d3d4f7d0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 10 client.25423 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 20 client.25423 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3dcbaa0 0x55d1d3dcdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 4 client.25423 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.572+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1d3cc1cb0 con 0x55d1d3dcbaa0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc603e00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3dcbaa0 0x55d1d3dcdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc603e00640 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3dcbaa0 0x55d1d3dcdec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0db79fe0eeb526 server_cookie=c3772fa37dd7c8c0 in_seq=0 out_seq=0 2024-04-02T17:13:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc5eaa00640 10 client.25423 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.577 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:04 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2189649776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:04.577 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:04 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/4053345199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc5eaa00640 1 -- 172.21.15.59:0/591894284 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d1d3cc1cb0 con 0x55d1d3dcbaa0 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc5eaa00640 10 client.25423 handle_command_reply: tid=0 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.573+0000 7fc60e77e740 1 client.25423 shutdown 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 msgr2=0x55d1d3d5e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc019ae0 tx=0x7fc5fc009a70 comp rx=0 tx=0).stop 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 shutdown_connections 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d1d3dcbaa0 0x55d1d3dcdec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d1d3d501c0 0x55d1d3d5e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1d3d55330 0x55d1d3dc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 --2- 172.21.15.59:0/591894284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1d3d4f7d0 0x55d1d3d5e0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 >> 172.21.15.59:0/591894284 conn(0x55d1d3d4ad40 msgr2=0x55d1d3d51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc5eaa00640 0 client.25423 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 shutdown_connections 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 1 -- 172.21.15.59:0/591894284 wait complete. 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.574+0000 7fc60e77e740 20 client.25423 trim_cache size 0 max 16384 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T17:13:04.579 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: "feb5fe33-b426-4f8f-b863-67ee087a6f76": { 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "feb5fe33-b426-4f8f-b863-67ee087a6f76", 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:13:04.580 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 msgr2=0x7fc5ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 0x7fc5ec0a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8099700 tx=0x7fc5f80d44a0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 msgr2=0x7fc60419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc60419a070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800ec80 tx=0x7fc5f800c6a0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 shutdown_connections 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5ec0a3c30 0x7fc5ec0a60f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc604104f60 0x7fc60419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc604107890 0x7fc6041a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 --2- 172.21.15.59:0/3602259763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc604102630 0x7fc60419a070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:04.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 >> 172.21.15.59:0/3602259763 conn(0x7fc6040fbfa0 msgr2=0x7fc604101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:04.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 shutdown_connections 2024-04-02T17:13:04.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:04.575+0000 7fc60ba00640 1 -- 172.21.15.59:0/3602259763 wait complete. 2024-04-02T17:13:04.594 INFO:tasks.cephfs.filesystem:scrub status for tag:feb5fe33-b426-4f8f-b863-67ee087a6f76 - {'path': '/', 'tag': 'feb5fe33-b426-4f8f-b863-67ee087a6f76', 'options': 'recursive,force'} 2024-04-02T17:13:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:04 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2189649776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:04 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/4053345199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: pgmap v388: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: scrub queued for path: / 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: scrub summary: active paths [/] 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:05 smithi059 ceph-mon[31664]: scrub summary: idle 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: pgmap v388: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: scrub queued for path: / 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: scrub summary: active paths [/] 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:05 smithi154 ceph-mon[36479]: scrub summary: idle 2024-04-02T17:13:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: pgmap v388: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-02T17:13:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: scrub queued for path: / 2024-04-02T17:13:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: scrub summary: active paths [/] 2024-04-02T17:13:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:05.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:05 smithi164 ceph-mon[36499]: scrub summary: idle 2024-04-02T17:13:07.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:07 smithi154 ceph-mon[36479]: pgmap v389: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:07 smithi059 ceph-mon[31664]: pgmap v389: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:07 smithi164 ceph-mon[36499]: pgmap v389: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-02T17:13:09.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:09 smithi154 ceph-mon[36479]: pgmap v390: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T17:13:09.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:09 smithi059 ceph-mon[31664]: pgmap v390: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T17:13:09.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:09 smithi164 ceph-mon[36499]: pgmap v390: 97 pgs: 97 active+clean; 651 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T17:13:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:11 smithi154 ceph-mon[36479]: pgmap v391: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:11.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:11 smithi059 ceph-mon[31664]: pgmap v391: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:11.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:11 smithi164 ceph-mon[36499]: pgmap v391: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:13 smithi154 ceph-mon[36479]: pgmap v392: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:13.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:13 smithi059 ceph-mon[31664]: pgmap v392: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:13 smithi164 ceph-mon[36499]: pgmap v392: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:15.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:15 smithi154 ceph-mon[36479]: pgmap v393: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:15 smithi059 ceph-mon[31664]: pgmap v393: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:15.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:15 smithi164 ceph-mon[36499]: pgmap v393: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 -- 172.21.15.59:0/4063311425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 msgr2=0x7f5ff41083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/4063311425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff41083a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009900 tx=0x7f5fe802bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 -- 172.21.15.59:0/4063311425 shutdown_connections 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/4063311425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 0x7f5ff41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/4063311425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff41095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/4063311425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff41083a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.258+0000 7f5ffaa00640 1 -- 172.21.15.59:0/4063311425 >> 172.21.15.59:0/4063311425 conn(0x7f5ff4076f40 msgr2=0x7f5ff4079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.259+0000 7f5ffaa00640 1 -- 172.21.15.59:0/4063311425 shutdown_connections 2024-04-02T17:13:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.259+0000 7f5ffaa00640 1 -- 172.21.15.59:0/4063311425 wait complete. 2024-04-02T17:13:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.260+0000 7f5ffaa00640 1 Processor -- start 2024-04-02T17:13:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.260+0000 7f5ffaa00640 1 -- start start 2024-04-02T17:13:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 0x7f5ff41a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113c60 con 0x7f5ff4107f80 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113ae0 con 0x7f5ff4109b30 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113de0 con 0x7f5ff4109170 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49768/0 (socket says 172.21.15.59:49768) 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff9600640 1 -- 172.21.15.59:0/3003515140 learned_addr learned my addr 172.21.15.59:0/3003515140 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ffa000640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 0x7f5ff41a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff8c00640 1 -- 172.21.15.59:0/3003515140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 msgr2=0x7f5ff41a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 0x7f5ff41a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff8c00640 1 -- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 msgr2=0x7f5ff419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.261+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ff8c00640 1 -- 172.21.15.59:0/3003515140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41a5e80 con 0x7f5ff4109170 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff419e7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000e9f0 tx=0x7f5fe000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ff2000640 1 -- 172.21.15.59:0/3003515140 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000cd80 con 0x7f5ff4109170 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a6170 con 0x7f5ff4109170 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ff2000640 1 -- 172.21.15.59:0/3003515140 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe00035f0 con 0x7f5ff4109170 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.262+0000 7f5ff2000640 1 -- 172.21.15.59:0/3003515140 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0010670 con 0x7f5ff4109170 2024-04-02T17:13:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.263+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a6710 con 0x7f5ff4109170 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.267+0000 7f5ffed83740 1 Processor -- start 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.267+0000 7f5ffed83740 1 -- start start 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ff2000640 1 -- 172.21.15.59:0/3003515140 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5fe00037a0 con 0x7f5ff4109170 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006af0 0x55c81f00b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 0x55c81f00e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c81ef6bd70 con 0x55c81f006af0 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c81ef142a0 con 0x55c81f006100 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffed83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c81ef67d20 con 0x55c81f00bc60 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006af0 0x55c81f00b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35162/0 (socket says 172.21.15.59:35162) 2024-04-02T17:13:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 -- 172.21.15.59:0/1413918981 learned_addr learned my addr 172.21.15.59:0/1413918981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 0x55c81f00e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 -- 172.21.15.59:0/1413918981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 msgr2=0x55c81f00e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 0x55c81f00e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 -- 172.21.15.59:0/1413918981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006af0 msgr2=0x55c81f00b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.268+0000 7f5ffa000640 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006af0 0x55c81f00b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5ffa000640 1 -- 172.21.15.59:0/1413918981 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81ef28840 con 0x55c81f006100 2024-04-02T17:13:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 0x55c81f00e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5ffa000640 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009fd0 tx=0x7f5fec0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508f0dd5fd6fdb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5fe6600640 1 -- 172.21.15.59:0/1413918981 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec004d20 con 0x55c81f006100 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5fe6600640 1 -- 172.21.15.59:0/1413918981 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec004050 con 0x55c81f006100 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5fe6600640 1 -- 172.21.15.59:0/1413918981 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec005240 con 0x55c81f006100 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.269+0000 7f5ff2000640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 0x7f5fdc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ff2000640 1 -- 172.21.15.59:0/3003515140 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5fe00d0730 con 0x7f5ff4109170 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ff9600640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 0x7f5fdc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 -- 172.21.15.59:0/1413918981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 msgr2=0x55c81f006520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009fd0 tx=0x7f5fec0118d0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 -- 172.21.15.59:0/1413918981 shutdown_connections 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006af0 0x55c81f00b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f00bc60 0x55c81f00e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1413918981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f006520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 -- 172.21.15.59:0/1413918981 >> 172.21.15.59:0/1413918981 conn(0x55c81f001670 msgr2=0x55c81f001a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 -- 172.21.15.59:0/1413918981 shutdown_connections 2024-04-02T17:13:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ffed83740 1 -- 172.21.15.59:0/1413918981 wait complete. 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.270+0000 7f5ff9600640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 0x7f5fdc0a6060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006010 tx=0x7f5fe802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 -- start start 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 Processor -- start 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f010ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f017b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c81f00ec10 con 0x55c81f00bc60 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c81ef27660 con 0x55c81f006100 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c81f00ed90 con 0x55c81f006af0 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.271+0000 7f5ffed83740 10 client.0 _ll_register_callbacks cb 0x55c81ef27050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ffa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f017b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35174/0 (socket says 172.21.15.59:35174) 2024-04-02T17:13:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 -- 172.21.15.59:0/1995789920 learned_addr learned my addr 172.21.15.59:0/1995789920 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f010ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 -- 172.21.15.59:0/1995789920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 msgr2=0x55c81f010ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f010ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 -- 172.21.15.59:0/1995789920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 msgr2=0x55c81f017b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f017b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 -- 172.21.15.59:0/1995789920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81ef36e80 con 0x55c81f006100 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ffa000640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f017b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f010ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009230 tx=0x7f5fe8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38ebda73252f26a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.272+0000 7f5fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8046070 con 0x55c81f006100 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fec004050 con 0x55c81f006100 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5fe8004c40 con 0x55c81f006100 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fe8004ef0 con 0x55c81f006100 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5ffed83740 10 client.25424 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81ec80210 con 0x55c81f006100 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.273+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec004050 con 0x55c81f006100 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe805d020 con 0x55c81f006100 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 10 client.25424 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 20 client.25424 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f01c400 0x55c81f01e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 4 client.25424 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c81ef1bcd0 con 0x55c81f01c400 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.274+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f01c400 0x55c81f01e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.275+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f01c400 0x55c81f01e830 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3e88405fd11dd server_cookie=befb6438d8f6f491 in_seq=0 out_seq=0 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.275+0000 7f5fdaa00640 10 client.25424 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.275+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1995789920 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c81ef1bcd0 con 0x55c81f01c400 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.275+0000 7f5fdaa00640 10 client.25424 handle_command_reply: tid=0 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 client.25424 shutdown 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 msgr2=0x55c81f010590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009230 tx=0x7f5fe8009260 comp rx=0 tx=0).stop 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 shutdown_connections 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f01c400 0x55c81f01e830 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f017b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f010ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5fdaa00640 0 client.25424 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1995789920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f010590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 >> 172.21.15.59:0/1995789920 conn(0x55c81f001670 msgr2=0x55c81f000870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 shutdown_connections 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 1 -- 172.21.15.59:0/1995789920 wait complete. 2024-04-02T17:13:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.276+0000 7f5ffed83740 20 client.25424 trim_cache size 0 max 16384 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.292+0000 7f5ffed83740 1 Processor -- start 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.292+0000 7f5ffed83740 1 -- start start 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f0101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f0106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41af530 con 0x55c81f00bc60 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41b1070 con 0x55c81f006100 2024-04-02T17:13:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffed83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41afc00 con 0x55c81f006af0 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f0106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42506/0 (socket says 172.21.15.59:42506) 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 -- 172.21.15.59:0/4202035119 learned_addr learned my addr 172.21.15.59:0/4202035119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ffa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f0106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49808/0 (socket says 172.21.15.59:49808) 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff9600640 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f0101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 -- 172.21.15.59:0/4202035119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 msgr2=0x55c81f0101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f0101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.293+0000 7f5ff8c00640 1 -- 172.21.15.59:0/4202035119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 msgr2=0x55c81f0106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f0106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5ff8c00640 1 -- 172.21.15.59:0/4202035119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81ed4c620 con 0x55c81f006af0 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5ff9600640 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f0101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0003ee0 tx=0x7f5fe00d0a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca9933a8f5c1f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5fe5c00640 1 -- 172.21.15.59:0/4202035119 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00a0d60 con 0x55c81f006af0 2024-04-02T17:13:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5fe5c00640 1 -- 172.21.15.59:0/4202035119 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe00d6030 con 0x55c81f006af0 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.294+0000 7f5fe5c00640 1 -- 172.21.15.59:0/4202035119 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0099020 con 0x55c81f006af0 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- 172.21.15.59:0/4202035119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 msgr2=0x55c81f017aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0003ee0 tx=0x7f5fe00d0a30 comp rx=0 tx=0).stop 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- 172.21.15.59:0/4202035119 shutdown_connections 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f00bc60 0x55c81f0106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f017aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 --2- 172.21.15.59:0/4202035119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f006100 0x55c81f0101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- 172.21.15.59:0/4202035119 >> 172.21.15.59:0/4202035119 conn(0x55c81f001670 msgr2=0x55c81effc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- 172.21.15.59:0/4202035119 shutdown_connections 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- 172.21.15.59:0/4202035119 wait complete. 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 -- start start 2024-04-02T17:13:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.295+0000 7f5ffed83740 1 Processor -- start 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f014fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f00bc60 0x55c81f0809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41b09a0 con 0x55c81f006100 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41b02d0 con 0x55c81f00bc60 2024-04-02T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ad510 con 0x55c81f006af0 2024-04-02T17:13:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffed83740 10 client.0 _ll_register_callbacks cb 0x7f5fe80040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ff9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49818/0 (socket says 172.21.15.59:49818) 2024-04-02T17:13:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ff9600640 1 -- 172.21.15.59:0/1931824596 learned_addr learned my addr 172.21.15.59:0/1931824596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ffa000640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f00bc60 0x55c81f0809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.296+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f014fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 -- 172.21.15.59:0/1931824596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 msgr2=0x55c81f014fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f014fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 -- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f00bc60 msgr2=0x55c81f0809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f00bc60 0x55c81f0809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 -- 172.21.15.59:0/1931824596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81f080ee0 con 0x55c81f006100 2024-04-02T17:13:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ff9600640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fe802c170 tx=0x7f5fe802c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1366a18c2055ac05 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80037f0 con 0x55c81f006100 2024-04-02T17:13:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5fe8003990 con 0x55c81f006100 2024-04-02T17:13:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe00d6030 con 0x55c81f006100 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fe806e050 con 0x55c81f006100 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ffed83740 10 client.15324 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.297+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81f0821c0 con 0x55c81f006100 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe00d6030 con 0x55c81f006100 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe805d070 con 0x55c81f006100 2024-04-02T17:13:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 10 client.15324 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 20 client.15324 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f082b10 0x55c81f084f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 4 client.15324 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c81ef792b0 con 0x55c81f082b10 2024-04-02T17:13:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.298+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f082b10 0x55c81f084f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.299+0000 7f5ff8c00640 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f082b10 0x55c81f084f00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1be5123c491cb8 server_cookie=f9bd3d3d870dfe95 in_seq=0 out_seq=0 2024-04-02T17:13:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.299+0000 7f5fdaa00640 10 client.15324 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.310+0000 7f5fdaa00640 1 -- 172.21.15.59:0/1931824596 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c81ef792b0 con 0x55c81f082b10 2024-04-02T17:13:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.310+0000 7f5fdaa00640 10 client.15324 handle_command_reply: tid=0 2024-04-02T17:13:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.310+0000 7f5ffed83740 1 client.15324 shutdown 2024-04-02T17:13:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 msgr2=0x55c81f014aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fe802c170 tx=0x7f5fe802c980 comp rx=0 tx=0).stop 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 shutdown_connections 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c81f082b10 0x55c81f084f00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c81f00bc60 0x55c81f0809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c81f006af0 0x55c81f014fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 --2- 172.21.15.59:0/1931824596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c81f006100 0x55c81f014aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5fdaa00640 0 client.15324 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 >> 172.21.15.59:0/1931824596 conn(0x55c81f001670 msgr2=0x55c81f010180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 shutdown_connections 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 1 -- 172.21.15.59:0/1931824596 wait complete. 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.311+0000 7f5ffed83740 20 client.15324 trim_cache size 0 max 16384 2024-04-02T17:13:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:13:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:13:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:13:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:13:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 msgr2=0x7f5fdc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 0x7f5fdc0a6060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006010 tx=0x7f5fe802df90 comp rx=0 tx=0).stop 2024-04-02T17:13:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 msgr2=0x7f5ff419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff419e7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000e9f0 tx=0x7f5fe000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 shutdown_connections 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ff4109b30 0x7f5ff41a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff4109170 0x7f5ff419e7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5fdc0a3ba0 0x7f5fdc0a6060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 --2- 172.21.15.59:0/3003515140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4107f80 0x7f5ff419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 >> 172.21.15.59:0/3003515140 conn(0x7f5ff4076f40 msgr2=0x7f5ff410c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 shutdown_connections 2024-04-02T17:13:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:17.312+0000 7f5ffaa00640 1 -- 172.21.15.59:0/3003515140 wait complete. 2024-04-02T17:13:17.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:17 smithi154 ceph-mon[36479]: pgmap v394: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:17 smithi059 ceph-mon[31664]: pgmap v394: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:17 smithi164 ceph-mon[36499]: pgmap v394: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-04-02T17:13:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:18 smithi154 ceph-mon[36479]: pgmap v395: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:18 smithi059 ceph-mon[31664]: pgmap v395: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:18.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:18 smithi164 ceph-mon[36499]: pgmap v395: 97 pgs: 97 active+clean; 657 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:20.902 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:20 smithi154 ceph-mon[36479]: pgmap v396: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-02T17:13:21.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:20 smithi059 ceph-mon[31664]: pgmap v396: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-02T17:13:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:20 smithi164 ceph-mon[36499]: pgmap v396: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-02T17:13:22.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:22 smithi154 ceph-mon[36479]: pgmap v397: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:23.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:22 smithi059 ceph-mon[31664]: pgmap v397: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:22 smithi164 ceph-mon[36499]: pgmap v397: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:24 smithi154 ceph-mon[36479]: pgmap v398: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-02T17:13:25.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:24 smithi059 ceph-mon[31664]: pgmap v398: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-02T17:13:25.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:24 smithi164 ceph-mon[36499]: pgmap v398: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-02T17:13:26.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:26 smithi154 ceph-mon[36479]: pgmap v399: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:27.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:26 smithi059 ceph-mon[31664]: pgmap v399: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:27.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:26 smithi164 ceph-mon[36499]: pgmap v399: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:28.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:28 smithi154 ceph-mon[36479]: pgmap v400: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:29.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:28 smithi059 ceph-mon[31664]: pgmap v400: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:29.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:28 smithi164 ceph-mon[36499]: pgmap v400: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-02T17:13:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 -- 172.21.15.59:0/2785481138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34105910 msgr2=0x7f5d3410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/2785481138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34105910 0x7f5d3410c1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009be0 tx=0x7f5d2c02bef0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 -- 172.21.15.59:0/2785481138 shutdown_connections 2024-04-02T17:13:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/2785481138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34104f50 0x7f5d341053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/2785481138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34105910 0x7f5d3410c1a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/2785481138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34103d60 0x7f5d34104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.460+0000 7f5d3bc00640 1 -- 172.21.15.59:0/2785481138 >> 172.21.15.59:0/2785481138 conn(0x7f5d340ff2f0 msgr2=0x7f5d34101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.461+0000 7f5d3bc00640 1 -- 172.21.15.59:0/2785481138 shutdown_connections 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.461+0000 7f5d3bc00640 1 -- 172.21.15.59:0/2785481138 wait complete. 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.462+0000 7f5d3bc00640 1 Processor -- start 2024-04-02T17:13:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.462+0000 7f5d3bc00640 1 -- start start 2024-04-02T17:13:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 0x7f5d3419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34105910 0x7f5d341a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410fa30 con 0x7f5d34105910 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410f8b0 con 0x7f5d34103d60 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d3bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410fbb0 con 0x7f5d34104f50 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d39c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34105910 0x7f5d341a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46208/0 (socket says 172.21.15.59:46208) 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d39200640 1 -- 172.21.15.59:0/3033531066 learned_addr learned my addr 172.21.15.59:0/3033531066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.463+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 0x7f5d3419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 -- 172.21.15.59:0/3033531066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 msgr2=0x7f5d3419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 0x7f5d3419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 -- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34105910 msgr2=0x7f5d341a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34105910 0x7f5d341a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 -- 172.21.15.59:0/3033531066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d341a1bd0 con 0x7f5d34103d60 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 0x7f5d3419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d39200640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9c0 tx=0x7f5d1800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d32000640 1 -- 172.21.15.59:0/3033531066 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800cd80 con 0x7f5d34103d60 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d32000640 1 -- 172.21.15.59:0/3033531066 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d180035f0 con 0x7f5d34103d60 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d341a1ec0 con 0x7f5d34103d60 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.464+0000 7f5d32000640 1 -- 172.21.15.59:0/3033531066 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18010690 con 0x7f5d34103d60 2024-04-02T17:13:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.465+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d341a2320 con 0x7f5d34103d60 2024-04-02T17:13:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 Processor -- start 2024-04-02T17:13:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d32000640 1 -- 172.21.15.59:0/3033531066 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d180037a0 con 0x7f5d34103d60 2024-04-02T17:13:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 -- start start 2024-04-02T17:13:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf35e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562cf2c3d70 con 0x5562cf35ded0 2024-04-02T17:13:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5562cf26c2a0 con 0x5562cf35e8c0 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.468+0000 7f5d3e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5562cf2bfd20 con 0x5562cf363a30 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf35e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39692/0 (socket says 172.21.15.59:39692) 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46218/0 (socket says 172.21.15.59:46218) 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d2be00640 1 -- 172.21.15.59:0/4228818117 learned_addr learned my addr 172.21.15.59:0/4228818117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf35e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59092/0 (socket says 172.21.15.59:59092) 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 -- 172.21.15.59:0/4228818117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 msgr2=0x5562cf365ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 -- 172.21.15.59:0/4228818117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 msgr2=0x5562cf35e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf35e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d39200640 1 -- 172.21.15.59:0/4228818117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cf280840 con 0x5562cf35e8c0 2024-04-02T17:13:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.469+0000 7f5d2be00640 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d32000640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 0x7f5d100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d39200640 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ce230 tx=0x7f5d180d6b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ead1852717ca70f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d32000640 1 -- 172.21.15.59:0/3033531066 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5d180d0850 con 0x7f5d34103d60 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d2a000640 1 -- 172.21.15.59:0/4228818117 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18098d10 con 0x5562cf35e8c0 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d39c00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 0x7f5d100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d2a000640 1 -- 172.21.15.59:0/4228818117 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d180bd0c0 con 0x5562cf35e8c0 2024-04-02T17:13:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.470+0000 7f5d2a000640 1 -- 172.21.15.59:0/4228818117 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1809c3d0 con 0x5562cf35e8c0 2024-04-02T17:13:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 -- 172.21.15.59:0/4228818117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 msgr2=0x5562cf3633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ce230 tx=0x7f5d180d6b20 comp rx=0 tx=0).stop 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d39c00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 0x7f5d100a6190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006010 tx=0x7f5d2c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 -- 172.21.15.59:0/4228818117 shutdown_connections 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf365ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35e8c0 0x5562cf3633c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 --2- 172.21.15.59:0/4228818117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf35e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 -- 172.21.15.59:0/4228818117 >> 172.21.15.59:0/4228818117 conn(0x5562cf359440 msgr2=0x5562cf359840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 -- 172.21.15.59:0/4228818117 shutdown_connections 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.471+0000 7f5d3e981740 1 -- 172.21.15.59:0/4228818117 wait complete. 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 -- start start 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 Processor -- start 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf3688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562cf366980 con 0x5562cf35e8c0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5562cf27f660 con 0x5562cf35ded0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5562cf366b00 con 0x5562cf363a30 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.472+0000 7f5d3e981740 10 client.0 _ll_register_callbacks cb 0x5562cf27f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf3688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46232/0 (socket says 172.21.15.59:46232) 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 -- 172.21.15.59:0/3396358022 learned_addr learned my addr 172.21.15.59:0/3396358022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39c00640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 -- 172.21.15.59:0/3396358022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 msgr2=0x5562cf36f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 -- 172.21.15.59:0/3396358022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 msgr2=0x5562cf3688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf3688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 -- 172.21.15.59:0/3396358022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cf28ee80 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d39200640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ce200 tx=0x7f5d1809cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a8e64232a8a6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d26e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.473+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1809c5b0 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5d18003b10 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d180bd0c0 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d1809c840 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 10 client.25442 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562cefd8210 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d180bd0c0 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d1809ccd0 con 0x5562cf35ded0 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 10 client.25442 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 20 client.25442 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf373e00 0x5562cf3761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 4 client.25442 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.474+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562cf273cd0 con 0x5562cf373e00 2024-04-02T17:13:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.475+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf373e00 0x5562cf3761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.475+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf373e00 0x5562cf3761f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3afdcb18ec7f906 server_cookie=3e2b3d8818fd6607 in_seq=0 out_seq=0 2024-04-02T17:13:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.475+0000 7f5d26e00640 10 client.25442 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d26e00640 1 -- 172.21.15.59:0/3396358022 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562cf273cd0 con 0x5562cf373e00 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d26e00640 10 client.25442 handle_command_reply: tid=0 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 client.25442 shutdown 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 msgr2=0x5562cf368390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d180ce200 tx=0x7f5d1809cb70 comp rx=0 tx=0).stop 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 shutdown_connections 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf373e00 0x5562cf3761f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf3688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.476+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3396358022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf368390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d26e00640 0 client.25442 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 >> 172.21.15.59:0/3396358022 conn(0x5562cf359440 msgr2=0x5562cf358640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 shutdown_connections 2024-04-02T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d3e981740 1 -- 172.21.15.59:0/3396358022 wait complete. 2024-04-02T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.477+0000 7f5d3e981740 20 client.25442 trim_cache size 0 max 16384 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.493+0000 7f5d3e981740 1 Processor -- start 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.493+0000 7f5d3e981740 1 -- start start 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf357db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf35e8c0 0x5562cf3582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341ab250 con 0x5562cf35ded0 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341acd90 con 0x5562cf363a30 2024-04-02T17:13:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d3e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341ab920 con 0x5562cf35e8c0 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d39200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf357db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d39c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf35e8c0 0x5562cf3582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d39c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46238/0 (socket says 172.21.15.59:46238) 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.494+0000 7f5d39c00640 1 -- 172.21.15.59:0/2081692954 learned_addr learned my addr 172.21.15.59:0/2081692954 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 -- 172.21.15.59:0/2081692954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf35e8c0 msgr2=0x5562cf3582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf35e8c0 0x5562cf3582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 -- 172.21.15.59:0/2081692954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 msgr2=0x5562cf357db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf357db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 -- 172.21.15.59:0/2081692954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cf376560 con 0x5562cf363a30 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d39c00640 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c02c500 tx=0x7f5d2c042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c472f6bda00384 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d29600640 1 -- 172.21.15.59:0/2081692954 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c02c890 con 0x5562cf363a30 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d29600640 1 -- 172.21.15.59:0/2081692954 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c004040 con 0x5562cf363a30 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.495+0000 7f5d29600640 1 -- 172.21.15.59:0/2081692954 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c034670 con 0x5562cf363a30 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 -- 172.21.15.59:0/2081692954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 msgr2=0x5562cf358830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c02c500 tx=0x7f5d2c042000 comp rx=0 tx=0).stop 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 -- 172.21.15.59:0/2081692954 shutdown_connections 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf363a30 0x5562cf358830 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf35e8c0 0x5562cf3582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 --2- 172.21.15.59:0/2081692954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35ded0 0x5562cf357db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 -- 172.21.15.59:0/2081692954 >> 172.21.15.59:0/2081692954 conn(0x5562cf359440 msgr2=0x5562cf367fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 -- 172.21.15.59:0/2081692954 shutdown_connections 2024-04-02T17:13:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.496+0000 7f5d3e981740 1 -- 172.21.15.59:0/2081692954 wait complete. 2024-04-02T17:13:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 -- start start 2024-04-02T17:13:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 Processor -- start 2024-04-02T17:13:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf36d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341ac6c0 con 0x5562cf35e8c0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341abff0 con 0x5562cf35ded0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d341a9230 con 0x5562cf363a30 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.497+0000 7f5d3e981740 10 client.0 _ll_register_callbacks cb 0x7f5d180d68e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46254/0 (socket says 172.21.15.59:46254) 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 -- 172.21.15.59:0/3569648826 learned_addr learned my addr 172.21.15.59:0/3569648826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d2be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf36d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39c00640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 -- 172.21.15.59:0/3569648826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 msgr2=0x5562cf36e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 -- 172.21.15.59:0/3569648826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 msgr2=0x5562cf36d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf36d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 -- 172.21.15.59:0/3569648826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cf0a4620 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39c00640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d39200640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d180a5250 tx=0x7f5d1809d090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a353e4a12449e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d26e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.498+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1809c5b0 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5d180a52d0 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2c004040 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d18098d10 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 10 client.25448 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562cf0a4620 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c004040 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d1809bf40 con 0x5562cf35ded0 2024-04-02T17:13:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 10 client.25448 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 20 client.25448 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf3dae10 0x5562cf3dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 4 client.25448 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.499+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562cf2d12b0 con 0x5562cf3dae10 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.500+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf3dae10 0x5562cf3dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.500+0000 7f5d2be00640 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf3dae10 0x5562cf3dd230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb2465ce58a3c8f server_cookie=bc43af750e5c651f in_seq=0 out_seq=0 2024-04-02T17:13:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.500+0000 7f5d26e00640 10 client.25448 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.504+0000 7f5d26e00640 1 -- 172.21.15.59:0/3569648826 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562cf2d12b0 con 0x5562cf3dae10 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.504+0000 7f5d26e00640 10 client.25448 handle_command_reply: tid=0 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.504+0000 7f5d3e981740 1 client.25448 shutdown 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 msgr2=0x5562cf36cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d180a5250 tx=0x7f5d1809d090 comp rx=0 tx=0).stop 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 shutdown_connections 2024-04-02T17:13:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5562cf3dae10 0x5562cf3dd230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562cf35e8c0 0x5562cf36d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562cf363a30 0x5562cf36e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 --2- 172.21.15.59:0/3569648826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5562cf35ded0 0x5562cf36cd50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 >> 172.21.15.59:0/3569648826 conn(0x5562cf359440 msgr2=0x5562cf35c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d26e00640 0 client.25448 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 shutdown_connections 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 1 -- 172.21.15.59:0/3569648826 wait complete. 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.505+0000 7f5d3e981740 20 client.25448 trim_cache size 0 max 16384 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:13:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:13:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:13:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:13:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 msgr2=0x7f5d100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 0x7f5d100a6190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006010 tx=0x7f5d2c004a40 comp rx=0 tx=0).stop 2024-04-02T17:13:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 msgr2=0x7f5d34199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9c0 tx=0x7f5d1800ee80 comp rx=0 tx=0).stop 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 shutdown_connections 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5d100a3cd0 0x7f5d100a6190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d34105910 0x7f5d341a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d34104f50 0x7f5d3419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 --2- 172.21.15.59:0/3033531066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d34103d60 0x7f5d34199fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 >> 172.21.15.59:0/3033531066 conn(0x7f5d340ff2f0 msgr2=0x7f5d34108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 shutdown_connections 2024-04-02T17:13:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:30.506+0000 7f5d3bc00640 1 -- 172.21.15.59:0/3033531066 wait complete. 2024-04-02T17:13:30.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:30 smithi154 ceph-mon[36479]: pgmap v401: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 852 B/s wr, 13 op/s 2024-04-02T17:13:31.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:30 smithi059 ceph-mon[31664]: pgmap v401: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 852 B/s wr, 13 op/s 2024-04-02T17:13:31.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:30 smithi164 ceph-mon[36499]: pgmap v401: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 852 B/s wr, 13 op/s 2024-04-02T17:13:33.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:32 smithi059 ceph-mon[31664]: pgmap v402: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T17:13:33.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:32 smithi164 ceph-mon[36499]: pgmap v402: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T17:13:33.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:32 smithi154 ceph-mon[36479]: pgmap v402: 97 pgs: 97 active+clean; 658 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T17:13:34.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:13:34.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:13:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.831+0000 7fa81f200640 1 -- 172.21.15.59:0/3790351866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820101a00 msgr2=0x7fa820101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.831+0000 7fa81f200640 1 --2- 172.21.15.59:0/3790351866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820101a00 0x7fa820101e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa81002c0f0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 -- 172.21.15.59:0/3790351866 shutdown_connections 2024-04-02T17:13:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 --2- 172.21.15.59:0/3790351866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820102bf0 0x7fa820103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 --2- 172.21.15.59:0/3790351866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 0x7fa820109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 --2- 172.21.15.59:0/3790351866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820101a00 0x7fa820101e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 -- 172.21.15.59:0/3790351866 >> 172.21.15.59:0/3790351866 conn(0x7fa8200fcf70 msgr2=0x7fa8200ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 -- 172.21.15.59:0/3790351866 shutdown_connections 2024-04-02T17:13:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.832+0000 7fa81f200640 1 -- 172.21.15.59:0/3790351866 wait complete. 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.833+0000 7fa81f200640 1 Processor -- start 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.833+0000 7fa81f200640 1 -- start start 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 0x7fa82019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 0x7fa8201a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa82010d680 con 0x7fa820101a00 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa82010d500 con 0x7fa820102bf0 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa82010d800 con 0x7fa8201035b0 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60762/0 (socket says 172.21.15.59:60762) 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 -- 172.21.15.59:0/1352521419 learned_addr learned my addr 172.21.15.59:0/1352521419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81e800640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 0x7fa8201a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81d400640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 0x7fa82019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 -- 172.21.15.59:0/1352521419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 msgr2=0x7fa8201a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 0x7fa8201a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 -- 172.21.15.59:0/1352521419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 msgr2=0x7fa82019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 0x7fa82019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81de00640 1 -- 172.21.15.59:0/1352521419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8201a1b20 con 0x7fa820101a00 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.834+0000 7fa81d400640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 0x7fa82019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa81de00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa810000f80 tx=0x7fa810032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa816a00640 1 -- 172.21.15.59:0/1352521419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810041020 con 0x7fa820101a00 2024-04-02T17:13:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8201a1db0 con 0x7fa820101a00 2024-04-02T17:13:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa816a00640 1 -- 172.21.15.59:0/1352521419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa810034cd0 con 0x7fa820101a00 2024-04-02T17:13:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa816a00640 1 -- 172.21.15.59:0/1352521419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810039040 con 0x7fa820101a00 2024-04-02T17:13:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.835+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8201a2200 con 0x7fa820101a00 2024-04-02T17:13:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.838+0000 7fa827a22740 1 Processor -- start 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.838+0000 7fa816a00640 1 -- 172.21.15.59:0/1352521419 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa810044020 con 0x7fa820101a00 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 -- start start 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa62970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a9bf770 con 0x55dc8aa5a980 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a967700 con 0x55dc8aa604e0 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa827a22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a9bb720 con 0x55dc8aa5b370 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa81de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60778/0 (socket says 172.21.15.59:60778) 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa81e800640 1 -- 172.21.15.59:0/1421293369 learned_addr learned my addr 172.21.15.59:0/1421293369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.839+0000 7fa81d400640 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa62970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 -- 172.21.15.59:0/1421293369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 msgr2=0x55dc8aa5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa5fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 -- 172.21.15.59:0/1421293369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 msgr2=0x55dc8aa62970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa62970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 -- 172.21.15.59:0/1421293369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8a79d7d0 con 0x55dc8aa5a980 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81d400640 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa62970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa816a00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 0x7fa8040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.840+0000 7fa81e800640 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa818009ca0 tx=0x7fa818011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de2a0c13bc6f3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.841+0000 7fa81de00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 0x7fa8040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.841+0000 7fa80f400640 1 -- 172.21.15.59:0/1421293369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801e070 con 0x55dc8aa5a980 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.841+0000 7fa816a00640 1 -- 172.21.15.59:0/1352521419 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa8100f2280 con 0x7fa820101a00 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.841+0000 7fa80f400640 1 -- 172.21.15.59:0/1421293369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818009e50 con 0x55dc8aa5a980 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.841+0000 7fa80f400640 1 -- 172.21.15.59:0/1421293369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81800eaf0 con 0x55dc8aa5a980 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- 172.21.15.59:0/1421293369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 msgr2=0x55dc8aa5ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa81de00640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 0x7fa8040a60f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8100ba990 tx=0x7fa8100f5cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa818009ca0 tx=0x7fa818011810 comp rx=0 tx=0).stop 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- 172.21.15.59:0/1421293369 shutdown_connections 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa62970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 --2- 172.21.15.59:0/1421293369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa5ada0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- 172.21.15.59:0/1421293369 >> 172.21.15.59:0/1421293369 conn(0x55dc8aa55ef0 msgr2=0x55dc8aa562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- 172.21.15.59:0/1421293369 shutdown_connections 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- 172.21.15.59:0/1421293369 wait complete. 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 -- start start 2024-04-02T17:13:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.842+0000 7fa827a22740 1 Processor -- start 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5a980 0x55dc8aa64ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dc8aa63490 con 0x55dc8aa604e0 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc8a97aac0 con 0x55dc8aa5b370 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dc8aa63610 con 0x55dc8aa5a980 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa827a22740 10 client.0 _ll_register_callbacks cb 0x55dc8a97d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60792/0 (socket says 172.21.15.59:60792) 2024-04-02T17:13:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa81d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa65430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa81e800640 1 -- 172.21.15.59:0/3233463145 learned_addr learned my addr 172.21.15.59:0/3233463145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.843+0000 7fa81e800640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5a980 0x55dc8aa64ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 -- 172.21.15.59:0/3233463145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5a980 msgr2=0x55dc8aa64ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5a980 0x55dc8aa64ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 -- 172.21.15.59:0/3233463145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 msgr2=0x55dc8aa6c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 -- 172.21.15.59:0/3233463145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8a6d33f0 con 0x55dc8aa5b370 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa81d400640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa65430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa82019b410 tx=0x7fa808012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4263192566847e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa803e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.844+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011c70 con 0x55dc8aa5b370 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818009e50 con 0x55dc8aa5b370 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa808011e10 con 0x55dc8aa5b370 2024-04-02T17:13:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa80800f460 con 0x55dc8aa5b370 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa827a22740 10 client.25460 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc8a6930f0 con 0x55dc8aa5b370 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818009e50 con 0x55dc8aa5b370 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.845+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa808024020 con 0x55dc8aa5b370 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 10 client.25460 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 20 client.25460 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aa709c0 0x55dc8aa72de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 4 client.25460 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8a960630 con 0x55dc8aa709c0 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.846+0000 7fa81e800640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aa709c0 0x55dc8aa72de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.847+0000 7fa81e800640 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aa709c0 0x55dc8aa72de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b474af8cdf92ac0d server_cookie=ed04f4248c61054f in_seq=0 out_seq=0 2024-04-02T17:13:34.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.847+0000 7fa803e00640 10 client.25460 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa803e00640 1 -- 172.21.15.59:0/3233463145 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc8a960630 con 0x55dc8aa709c0 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa803e00640 10 client.25460 handle_command_reply: tid=0 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 client.25460 shutdown 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 msgr2=0x55dc8aa65430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa65430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa82019b410 tx=0x7fa808012b60 comp rx=0 tx=0).stop 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 shutdown_connections 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aa709c0 0x55dc8aa72de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa65430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 --2- 172.21.15.59:0/3233463145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5a980 0x55dc8aa64ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa803e00640 0 client.25460 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 >> 172.21.15.59:0/3233463145 conn(0x55dc8aa55ef0 msgr2=0x55dc8aa57140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 shutdown_connections 2024-04-02T17:13:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 1 -- 172.21.15.59:0/3233463145 wait complete. 2024-04-02T17:13:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.848+0000 7fa827a22740 20 client.25460 trim_cache size 0 max 16384 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.867+0000 7fa827a22740 1 Processor -- start 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 -- start start 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa604e0 0x55dc8aa776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ab170 con 0x55dc8aa5a980 2024-04-02T17:13:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201accb0 con 0x55dc8aa5b370 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa827a22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ab840 con 0x55dc8aa604e0 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60800/0 (socket says 172.21.15.59:60800) 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa81e800640 1 -- 172.21.15.59:0/376399634 learned_addr learned my addr 172.21.15.59:0/376399634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa81d400640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.868+0000 7fa81e800640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa604e0 0x55dc8aa776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 -- 172.21.15.59:0/376399634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa604e0 msgr2=0x55dc8aa776c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa604e0 0x55dc8aa776c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 -- 172.21.15.59:0/376399634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 msgr2=0x55dc8aa6c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 -- 172.21.15.59:0/376399634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8aa73150 con 0x55dc8aa5a980 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81d400640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa81e800640 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa818009c40 tx=0x7fa818005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a30dc7872f3bda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa80ea00640 1 -- 172.21.15.59:0/376399634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801e070 con 0x55dc8aa5a980 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa80ea00640 1 -- 172.21.15.59:0/376399634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818005590 con 0x55dc8aa5a980 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.869+0000 7fa80ea00640 1 -- 172.21.15.59:0/376399634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003a80 con 0x55dc8aa5a980 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 -- 172.21.15.59:0/376399634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 msgr2=0x55dc8aa55190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa818009c40 tx=0x7fa818005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 -- 172.21.15.59:0/376399634 shutdown_connections 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa604e0 0x55dc8aa776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 --2- 172.21.15.59:0/376399634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa55190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 -- 172.21.15.59:0/376399634 >> 172.21.15.59:0/376399634 conn(0x55dc8aa55ef0 msgr2=0x55dc8aa54a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 -- 172.21.15.59:0/376399634 shutdown_connections 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.870+0000 7fa827a22740 1 -- 172.21.15.59:0/376399634 wait complete. 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 -- start start 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 Processor -- start 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ac5e0 con 0x55dc8aa5a980 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201abf10 con 0x55dc8aa604e0 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201a9150 con 0x55dc8aa5b370 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.871+0000 7fa827a22740 10 client.0 _ll_register_callbacks cb 0x7fa808009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48734/0 (socket says 172.21.15.59:48734) 2024-04-02T17:13:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60806/0 (socket says 172.21.15.59:60806) 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81d400640 1 -- 172.21.15.59:0/4224223309 learned_addr learned my addr 172.21.15.59:0/4224223309 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 -- 172.21.15.59:0/4224223309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 msgr2=0x55dc8aa6c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 -- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 msgr2=0x55dc8aa6a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 -- 172.21.15.59:0/4224223309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8aa6aa50 con 0x55dc8aa5a980 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81d400640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa81e800640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa818009c40 tx=0x7fa818011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b17ef97b48677cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.872+0000 7fa803e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801e070 con 0x55dc8aa5a980 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa8180050b0 con 0x55dc8aa5a980 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818005590 con 0x55dc8aa5a980 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa81800ee70 con 0x55dc8aa5a980 2024-04-02T17:13:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa827a22740 10 client.15360 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc8aad7990 con 0x55dc8aa5a980 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818005590 con 0x55dc8aa5a980 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa818032070 con 0x55dc8aa5a980 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa827a22740 10 client.15360 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.873+0000 7fa827a22740 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.874+0000 7fa827a22740 20 client.15360 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.874+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aad80d0 0x55dc8aada4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.874+0000 7fa827a22740 4 client.15360 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.874+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc8a9cccb0 con 0x55dc8aad80d0 2024-04-02T17:13:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.874+0000 7fa81d400640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aad80d0 0x55dc8aada4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:34.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa81d400640 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aad80d0 0x55dc8aada4f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ab73ba9dcaefac server_cookie=ba5d03daf7defaa8 in_seq=0 out_seq=0 2024-04-02T17:13:34.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa803e00640 10 client.15360 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa803e00640 1 -- 172.21.15.59:0/4224223309 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc8a9cccb0 con 0x55dc8aad80d0 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa803e00640 10 client.15360 handle_command_reply: tid=0 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 client.15360 shutdown 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 msgr2=0x55dc8aa6bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa818009c40 tx=0x7fa818011810 comp rx=0 tx=0).stop 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 shutdown_connections 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dc8aad80d0 0x55dc8aada4f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc8aa604e0 0x55dc8aa6a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc8aa5b370 0x55dc8aa6c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 --2- 172.21.15.59:0/4224223309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dc8aa5a980 0x55dc8aa6bd00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa803e00640 0 client.15360 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 >> 172.21.15.59:0/4224223309 conn(0x55dc8aa55ef0 msgr2=0x55dc8aa59150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 shutdown_connections 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 1 -- 172.21.15.59:0/4224223309 wait complete. 2024-04-02T17:13:34.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.875+0000 7fa827a22740 20 client.15360 trim_cache size 0 max 16384 2024-04-02T17:13:34.879 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:13:34.879 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-02T17:13:34.879 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-02T17:13:34.879 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 msgr2=0x7fa8040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 0x7fa8040a60f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8100ba990 tx=0x7fa8100f5cc0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 msgr2=0x7fa820199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa810000f80 tx=0x7fa810032ec0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 shutdown_connections 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8201035b0 0x7fa8201a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa820102bf0 0x7fa82019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa8040a3c30 0x7fa8040a60f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 --2- 172.21.15.59:0/1352521419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa820101a00 0x7fa820199f00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 >> 172.21.15.59:0/1352521419 conn(0x7fa8200fcf70 msgr2=0x7fa820073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 shutdown_connections 2024-04-02T17:13:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:34.876+0000 7fa81f200640 1 -- 172.21.15.59:0/1352521419 wait complete. 2024-04-02T17:13:34.894 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T17:13:34.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:13:34.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:34 smithi059 ceph-mon[31664]: pgmap v403: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:35.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:34 smithi164 ceph-mon[36499]: pgmap v403: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:35.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 -- 172.21.15.59:0/2523388722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 msgr2=0x7f81bc102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 --2- 172.21.15.59:0/2523388722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc102fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009900 tx=0x7f81a402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:35.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 -- 172.21.15.59:0/2523388722 shutdown_connections 2024-04-02T17:13:35.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 --2- 172.21.15.59:0/2523388722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc102fd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 --2- 172.21.15.59:0/2523388722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc103510 0x7f81bc109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 --2- 172.21.15.59:0/2523388722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc101960 0x7f81bc101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 -- 172.21.15.59:0/2523388722 >> 172.21.15.59:0/2523388722 conn(0x7f81bc0fced0 msgr2=0x7f81bc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 -- 172.21.15.59:0/2523388722 shutdown_connections 2024-04-02T17:13:35.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.116+0000 7f81bbe00640 1 -- 172.21.15.59:0/2523388722 wait complete. 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 Processor -- start 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 -- start start 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc101960 0x7f81bc195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 0x7f81bc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc10b400 con 0x7f81bc102b50 2024-04-02T17:13:35.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc10b280 con 0x7f81bc103510 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.117+0000 7f81bbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc10b580 con 0x7f81bc101960 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60828/0 (socket says 172.21.15.59:60828) 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 -- 172.21.15.59:0/3718687095 learned_addr learned my addr 172.21.15.59:0/3718687095 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81baa00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc101960 0x7f81bc195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81bb400640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 0x7f81bc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 -- 172.21.15.59:0/3718687095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc101960 msgr2=0x7f81bc195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc101960 0x7f81bc195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 -- 172.21.15.59:0/3718687095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 msgr2=0x7f81bc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 0x7f81bc19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 -- 172.21.15.59:0/3718687095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc19d680 con 0x7f81bc102b50 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81bb400640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 0x7f81bc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.118+0000 7f81ba000640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009230 tx=0x7f81a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.119+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4041070 con 0x7f81bc102b50 2024-04-02T17:13:35.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.119+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a40044d0 con 0x7f81bc102b50 2024-04-02T17:13:35.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.119+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4034440 con 0x7f81bc102b50 2024-04-02T17:13:35.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.119+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc19d910 con 0x7f81bc102b50 2024-04-02T17:13:35.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.119+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc19ddb0 con 0x7f81bc102b50 2024-04-02T17:13:35.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.120+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc108c80 con 0x7f81bc102b50 2024-04-02T17:13:35.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.122+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81a4044070 con 0x7f81bc102b50 2024-04-02T17:13:35.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.123+0000 7f81b3400640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 0x7f81a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.124+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f81a40f2090 con 0x7f81bc102b50 2024-04-02T17:13:35.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.124+0000 7f81baa00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 0x7f81a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.124+0000 7f81baa00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 0x7f81a00a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81bc1029b0 tx=0x7f81ac009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.128+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f81a40be9b0 con 0x7f81bc102b50 2024-04-02T17:13:35.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:34 smithi154 ceph-mon[36479]: pgmap v403: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.375+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81bc02d370 con 0x7f81bc102b50 2024-04-02T17:13:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.377+0000 7f81b3400640 1 -- 172.21.15.59:0/3718687095 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f81a40b8e20 con 0x7f81bc102b50 2024-04-02T17:13:35.377 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:13:35.377 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:12:54.846637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24529,24530,24547,24560,24584]},"id":1}]} 2024-04-02T17:13:35.377 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2024-04-02T17:13:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 msgr2=0x7f81a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 0x7f81a00a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81bc1029b0 tx=0x7f81ac009720 comp rx=0 tx=0).stop 2024-04-02T17:13:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 msgr2=0x7f81bc195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009230 tx=0x7f81a4005d50 comp rx=0 tx=0).stop 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 shutdown_connections 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f81a00a3c30 0x7f81a00a60f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81bc102b50 0x7f81bc195fa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81bc103510 0x7f81bc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 --2- 172.21.15.59:0/3718687095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81bc101960 0x7f81bc195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 >> 172.21.15.59:0/3718687095 conn(0x7f81bc0fced0 msgr2=0x7f81bc105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 shutdown_connections 2024-04-02T17:13:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.380+0000 7f81bbe00640 1 -- 172.21.15.59:0/3718687095 wait complete. 2024-04-02T17:13:35.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T17:13:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 -- 172.21.15.59:0/547510880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c102460 msgr2=0x7f697c104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 --2- 172.21.15.59:0/547510880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c102460 0x7f697c104850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f696c009ca0 tx=0x7f696c02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 -- 172.21.15.59:0/547510880 shutdown_connections 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 --2- 172.21.15.59:0/547510880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c1076c0 0x7f697c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 --2- 172.21.15.59:0/547510880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 0x7f697c107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 --2- 172.21.15.59:0/547510880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c102460 0x7f697c104850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 -- 172.21.15.59:0/547510880 >> 172.21.15.59:0/547510880 conn(0x7f697c0fbe70 msgr2=0x7f697c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 -- 172.21.15.59:0/547510880 shutdown_connections 2024-04-02T17:13:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.636+0000 7f6980e00640 1 -- 172.21.15.59:0/547510880 wait complete. 2024-04-02T17:13:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.637+0000 7f6980e00640 1 Processor -- start 2024-04-02T17:13:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 -- start start 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 0x7f697c100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c1076c0 0x7f697c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10d690 con 0x7f697c102460 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10d510 con 0x7f697c104d90 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f6980e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10d810 con 0x7f697c1076c0 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f697b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f697b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60848/0 (socket says 172.21.15.59:60848) 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.638+0000 7f697b400640 1 -- 172.21.15.59:0/139010076 learned_addr learned my addr 172.21.15.59:0/139010076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697be00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c1076c0 0x7f697c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697aa00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 0x7f697c100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 -- 172.21.15.59:0/139010076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c1076c0 msgr2=0x7f697c100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c1076c0 0x7f697c100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 -- 172.21.15.59:0/139010076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 msgr2=0x7f697c100410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 0x7f697c100410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 -- 172.21.15.59:0/139010076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1011e0 con 0x7f697c102460 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.639+0000 7f697b400640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f696c0325f0 tx=0x7f696c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.640+0000 7f6973e00640 1 -- 172.21.15.59:0/139010076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c041020 con 0x7f697c102460 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.640+0000 7f6973e00640 1 -- 172.21.15.59:0/139010076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f696c034cd0 con 0x7f697c102460 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.640+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1a1df0 con 0x7f697c102460 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.640+0000 7f6973e00640 1 -- 172.21.15.59:0/139010076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c039040 con 0x7f697c102460 2024-04-02T17:13:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.640+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a21f0 con 0x7f697c102460 2024-04-02T17:13:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.643+0000 7f698517a740 1 Processor -- start 2024-04-02T17:13:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.643+0000 7f698517a740 1 -- start start 2024-04-02T17:13:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.643+0000 7f6973e00640 1 -- 172.21.15.59:0/139010076 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f696c044020 con 0x7f697c102460 2024-04-02T17:13:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.643+0000 7f698517a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f698517a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f698517a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f698517a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55576ce82400 con 0x55576cf40e20 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f698517a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55576ce66f10 con 0x55576cf46980 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f698517a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55576cea66f0 con 0x55576cf41810 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60862/0 (socket says 172.21.15.59:60862) 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 -- 172.21.15.59:0/2351098083 learned_addr learned my addr 172.21.15.59:0/2351098083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697aa00640 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697b400640 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 -- 172.21.15.59:0/2351098083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 msgr2=0x55576cf46310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf46310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 -- 172.21.15.59:0/2351098083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 msgr2=0x55576cf48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.644+0000 7f697be00640 1 -- 172.21.15.59:0/2351098083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576cbba3f0 con 0x55576cf40e20 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f697be00640 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6974009ca0 tx=0x7f6974011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30481761d8ce9d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f6973e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 0x7f69600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f6970c00640 1 -- 172.21.15.59:0/2351098083 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697401e070 con 0x55576cf40e20 2024-04-02T17:13:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f6970c00640 1 -- 172.21.15.59:0/2351098083 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974009e50 con 0x55576cf40e20 2024-04-02T17:13:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f6970c00640 1 -- 172.21.15.59:0/2351098083 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400ea80 con 0x55576cf40e20 2024-04-02T17:13:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f6973e00640 1 -- 172.21.15.59:0/139010076 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f696c0f2210 con 0x7f697c102460 2024-04-02T17:13:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.645+0000 7f697b400640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 0x7f69600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 -- 172.21.15.59:0/2351098083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 msgr2=0x55576cf41240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6974009ca0 tx=0x7f6974011810 comp rx=0 tx=0).stop 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f697b400640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 0x7f69600a6070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f696c0ba920 tx=0x7f696c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 -- 172.21.15.59:0/2351098083 shutdown_connections 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 --2- 172.21.15.59:0/2351098083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf41240 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 -- 172.21.15.59:0/2351098083 >> 172.21.15.59:0/2351098083 conn(0x55576cf08a50 msgr2=0x55576cf3c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 -- 172.21.15.59:0/2351098083 shutdown_connections 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.646+0000 7f698517a740 1 -- 172.21.15.59:0/2351098083 wait complete. 2024-04-02T17:13:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 -- start start 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 Processor -- start 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf4b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf52610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55576cea26a0 con 0x55576cf40e20 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55576ce4e980 con 0x55576cf46980 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55576ce61cb0 con 0x55576cf41810 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.647+0000 7f698517a740 10 client.0 _ll_register_callbacks cb 0x55576ce616a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60866/0 (socket says 172.21.15.59:60866) 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 -- 172.21.15.59:0/4163681880 learned_addr learned my addr 172.21.15.59:0/4163681880 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697aa00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf4b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf52610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 -- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 msgr2=0x55576cf4b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf4b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 -- 172.21.15.59:0/4163681880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 msgr2=0x55576cf52610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf52610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 -- 172.21.15.59:0/4163681880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576cc847d0 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697aa00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf4b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f697be00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6974011d10 tx=0x7f697400e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c720ee56697e8540 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.648+0000 7f6965600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697401e040 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f697400ed30 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6974009e50 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69740035f0 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f698517a740 10 client.15378 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55576cf55980 con 0x55576cf40e20 2024-04-02T17:13:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.649+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974009e50 con 0x55576cf40e20 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6974024070 con 0x55576cf40e20 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 10 client.15378 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 10 client.15378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 20 client.15378 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cf56de0 0x55576cf591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 4 client.15378 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55576ce66040 con 0x55576cf56de0 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.650+0000 7f697aa00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cf56de0 0x55576cf591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.651+0000 7f697aa00640 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cf56de0 0x55576cf591d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9472d937ae978f8 server_cookie=de7730bc4c6800ac in_seq=0 out_seq=0 2024-04-02T17:13:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.651+0000 7f6965600640 10 client.15378 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:13:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f6965600640 1 -- 172.21.15.59:0/4163681880 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55576ce66040 con 0x55576cf56de0 2024-04-02T17:13:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f6965600640 10 client.15378 handle_command_reply: tid=0 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 client.15378 shutdown 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 msgr2=0x55576cf4b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6974011d10 tx=0x7f697400e3b0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 shutdown_connections 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cf56de0 0x55576cf591d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf4b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 --2- 172.21.15.59:0/4163681880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b090 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f6965600640 0 client.15378 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 >> 172.21.15.59:0/4163681880 conn(0x55576cf08a50 msgr2=0x55576cf3baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 shutdown_connections 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 1 -- 172.21.15.59:0/4163681880 wait complete. 2024-04-02T17:13:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.652+0000 7f698517a740 20 client.15378 trim_cache size 0 max 16384 2024-04-02T17:13:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.667+0000 7f698517a740 1 Processor -- start 2024-04-02T17:13:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.667+0000 7f698517a740 1 -- start start 2024-04-02T17:13:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1ab1f0 con 0x55576cf40e20 2024-04-02T17:13:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1acd30 con 0x55576cf46980 2024-04-02T17:13:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f698517a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1ab8c0 con 0x55576cf41810 2024-04-02T17:13:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f697aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f697aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48794/0 (socket says 172.21.15.59:48794) 2024-04-02T17:13:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f697aa00640 1 -- 172.21.15.59:0/2503502982 learned_addr learned my addr 172.21.15.59:0/2503502982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.668+0000 7f697be00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697be00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 -- 172.21.15.59:0/2503502982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 msgr2=0x55576cf400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 -- 172.21.15.59:0/2503502982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 msgr2=0x55576cf4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 -- 172.21.15.59:0/2503502982 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576cf40a60 con 0x55576cf41810 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697be00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:13:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.669+0000 7f697aa00640 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f697c101b40 tx=0x7f6968005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09ad4e5a7b5acc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.672+0000 7f6967e00640 1 -- 172.21.15.59:0/2503502982 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968004030 con 0x55576cf41810 2024-04-02T17:13:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.672+0000 7f6967e00640 1 -- 172.21.15.59:0/2503502982 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6968005500 con 0x55576cf41810 2024-04-02T17:13:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.672+0000 7f6967e00640 1 -- 172.21.15.59:0/2503502982 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696800fdd0 con 0x55576cf41810 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- 172.21.15.59:0/2503502982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 msgr2=0x55576cf525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f697c101b40 tx=0x7f6968005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- 172.21.15.59:0/2503502982 shutdown_connections 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf46980 0x55576cf400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf41810 0x55576cf525d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 --2- 172.21.15.59:0/2503502982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf40e20 0x55576cf4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- 172.21.15.59:0/2503502982 >> 172.21.15.59:0/2503502982 conn(0x55576cf08a50 msgr2=0x55576cf3f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- 172.21.15.59:0/2503502982 shutdown_connections 2024-04-02T17:13:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- 172.21.15.59:0/2503502982 wait complete. 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.673+0000 7f698517a740 1 -- start start 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 Processor -- start 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 0x55576cf4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf511f0 0x55576cf51670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1ac660 con 0x55576cf511f0 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1abf90 con 0x55576cf40e20 2024-04-02T17:13:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1a91d0 con 0x55576cf46980 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f698517a740 10 client.0 _ll_register_callbacks cb 0x7f6974009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f697aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f697be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 0x55576cf4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f697aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48804/0 (socket says 172.21.15.59:48804) 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.674+0000 7f697aa00640 1 -- 172.21.15.59:0/88259896 learned_addr learned my addr 172.21.15.59:0/88259896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf511f0 0x55576cf51670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 -- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 msgr2=0x55576cf501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 -- 172.21.15.59:0/88259896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf511f0 msgr2=0x55576cf51670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf511f0 0x55576cf51670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 -- 172.21.15.59:0/88259896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576cf51bb0 con 0x55576cf40e20 2024-04-02T17:13:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697aa00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f697be00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 0x55576cf4fca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69740059f0 tx=0x7f6974001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a77149155f42c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.675+0000 7f6965600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697401e040 con 0x55576cf40e20 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f697400ec10 con 0x55576cf40e20 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6968005500 con 0x55576cf40e20 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6974019640 con 0x55576cf40e20 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f698517a740 10 client.25484 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55576cfbd5c0 con 0x55576cf40e20 2024-04-02T17:13:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.676+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6968005500 con 0x55576cf40e20 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6974024040 con 0x55576cf40e20 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 10 client.25484 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 10 client.25484 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 20 client.25484 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cfbde90 0x55576cfc02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 4 client.25484 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55576ceb3c30 con 0x55576cfbde90 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.677+0000 7f697aa00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cfbde90 0x55576cfc02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f697aa00640 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cfbde90 0x55576cfc02b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ce8a5c028133384 server_cookie=ba3575632e88a575 in_seq=0 out_seq=0 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f6965600640 10 client.25484 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f6965600640 1 -- 172.21.15.59:0/88259896 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55576ceb3c30 con 0x55576cfbde90 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f6965600640 10 client.25484 handle_command_reply: tid=0 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 client.25484 shutdown 2024-04-02T17:13:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 msgr2=0x55576cf4fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 0x55576cf4fca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69740059f0 tx=0x7f6974001e50 comp rx=0 tx=0).stop 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 shutdown_connections 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x55576cfbde90 0x55576cfc02b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55576cf511f0 0x55576cf51670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576cf46980 0x55576cf501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 --2- 172.21.15.59:0/88259896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55576cf40e20 0x55576cf4fca0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 >> 172.21.15.59:0/88259896 conn(0x55576cf08a50 msgr2=0x55576cf40260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.678+0000 7f6965600640 0 client.25484 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 shutdown_connections 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f698517a740 1 -- 172.21.15.59:0/88259896 wait complete. 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f698517a740 20 client.25484 trim_cache size 0 max 16384 2024-04-02T17:13:35.680 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:13:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 msgr2=0x7f69600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 0x7f69600a6070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f696c0ba920 tx=0x7f696c02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 msgr2=0x7f697c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f696c0325f0 tx=0x7f696c032ec0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 shutdown_connections 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.679+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697c1076c0 0x7f697c100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f697c104d90 0x7f697c100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69600a3bb0 0x7f69600a6070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 --2- 172.21.15.59:0/139010076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f697c102460 0x7f697c101dd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 >> 172.21.15.59:0/139010076 conn(0x7f697c0fbe70 msgr2=0x7f697c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 shutdown_connections 2024-04-02T17:13:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.680+0000 7f6980e00640 1 -- 172.21.15.59:0/139010076 wait complete. 2024-04-02T17:13:35.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T17:13:35.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 -- 172.21.15.59:0/3007406563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc103c00 msgr2=0x7f30fc104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 --2- 172.21.15.59:0/3007406563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc103c00 0x7f30fc104020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009900 tx=0x7f30ec02bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:35.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 -- 172.21.15.59:0/3007406563 shutdown_connections 2024-04-02T17:13:35.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 --2- 172.21.15.59:0/3007406563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 0x7f30fc105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 --2- 172.21.15.59:0/3007406563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc1057b0 0x7f30fc10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 --2- 172.21.15.59:0/3007406563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc103c00 0x7f30fc104020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.902+0000 7f30fb200640 1 -- 172.21.15.59:0/3007406563 >> 172.21.15.59:0/3007406563 conn(0x7f30fc0ff170 msgr2=0x7f30fc1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.903+0000 7f30fb200640 1 -- 172.21.15.59:0/3007406563 shutdown_connections 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.903+0000 7f30fb200640 1 -- 172.21.15.59:0/3007406563 wait complete. 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.904+0000 7f30fb200640 1 Processor -- start 2024-04-02T17:13:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.904+0000 7f30fb200640 1 -- start start 2024-04-02T17:13:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 0x7f30fc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc1057b0 0x7f30fc1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10f880 con 0x7f30fc104df0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10f700 con 0x7f30fc103c00 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc10fa00 con 0x7f30fc1057b0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36222/0 (socket says 172.21.15.59:36222) 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 -- 172.21.15.59:0/2018223669 learned_addr learned my addr 172.21.15.59:0/2018223669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f3e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 0x7f30fc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30fa800640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc1057b0 0x7f30fc1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 -- 172.21.15.59:0/2018223669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc1057b0 msgr2=0x7f30fc1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc1057b0 0x7f30fc1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 -- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 msgr2=0x7f30fc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 0x7f30fc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.905+0000 7f30f9e00640 1 -- 172.21.15.59:0/2018223669 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc1a1ae0 con 0x7f30fc103c00 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30f9e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30ec000c00 tx=0x7f30ec02c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30f2a00640 1 -- 172.21.15.59:0/2018223669 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec041070 con 0x7f30fc103c00 2024-04-02T17:13:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30f2a00640 1 -- 172.21.15.59:0/2018223669 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30ec004450 con 0x7f30fc103c00 2024-04-02T17:13:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30f2a00640 1 -- 172.21.15.59:0/2018223669 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec034440 con 0x7f30fc103c00 2024-04-02T17:13:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc1a1d70 con 0x7f30fc103c00 2024-04-02T17:13:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.906+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a21f0 con 0x7f30fc103c00 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.909+0000 7f30f2a00640 1 -- 172.21.15.59:0/2018223669 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f30ec044070 con 0x7f30fc103c00 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.909+0000 7f31037af740 1 Processor -- start 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 -- start start 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 0x55b1b02d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02db980 0x55b1b02dde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b1b0217400 con 0x55b1b02d6810 2024-04-02T17:13:35.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1b01fbf10 con 0x55b1b02db980 2024-04-02T17:13:35.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f31037af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1b023b6f0 con 0x55b1b02d5e20 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f30f9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f30f9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02db310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60916/0 (socket says 172.21.15.59:60916) 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f30f9e00640 1 -- 172.21.15.59:0/2841951467 learned_addr learned my addr 172.21.15.59:0/2841951467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f30fa800640 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 0x55b1b02d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.910+0000 7f30f3e00640 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02db980 0x55b1b02dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 -- 172.21.15.59:0/2841951467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02db980 msgr2=0x55b1b02dde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02db980 0x55b1b02dde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 -- 172.21.15.59:0/2841951467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 msgr2=0x55b1b02db310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02db310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 -- 172.21.15.59:0/2841951467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1aff4f3f0 con 0x55b1b02d5e20 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30fa800640 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 0x55b1b02d6240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009ed0 tx=0x7f30f40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d838eec7a2a8cc18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.911+0000 7f30f2a00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 0x7f30e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30f2a00640 1 -- 172.21.15.59:0/2018223669 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f30ec0f1f90 con 0x7f30fc103c00 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30f9e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 0x7f30e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30eb000640 1 -- 172.21.15.59:0/2841951467 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4003b70 con 0x55b1b02d5e20 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30eb000640 1 -- 172.21.15.59:0/2841951467 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f4015070 con 0x55b1b02d5e20 2024-04-02T17:13:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30eb000640 1 -- 172.21.15.59:0/2841951467 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4011870 con 0x55b1b02d5e20 2024-04-02T17:13:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f31037af740 1 -- 172.21.15.59:0/2841951467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 msgr2=0x55b1b02d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f30f9e00640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 0x7f30e00a60f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0bdf40 tx=0x7f30ec0f59d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.912+0000 7f31037af740 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 0x55b1b02d6240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009ed0 tx=0x7f30f40107a0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 -- 172.21.15.59:0/2841951467 shutdown_connections 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02db980 0x55b1b02dde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 --2- 172.21.15.59:0/2841951467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02d5e20 0x55b1b02d6240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 -- 172.21.15.59:0/2841951467 >> 172.21.15.59:0/2841951467 conn(0x55b1b029da50 msgr2=0x55b1b02d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 -- 172.21.15.59:0/2841951467 shutdown_connections 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 -- 172.21.15.59:0/2841951467 wait complete. 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 -- start start 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.913+0000 7f31037af740 1 Processor -- start 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b1b02376a0 con 0x55b1b02d5e20 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1b01e3980 con 0x55b1b02d6810 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1b01f6cb0 con 0x55b1b02db980 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f31037af740 10 client.0 _ll_register_callbacks cb 0x55b1b01f66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f30fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f30fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60926/0 (socket says 172.21.15.59:60926) 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f30fa800640 1 -- 172.21.15.59:0/2950581762 learned_addr learned my addr 172.21.15.59:0/2950581762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f30f3e00640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.914+0000 7f30fa800640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 -- 172.21.15.59:0/2950581762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 msgr2=0x55b1b02e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 -- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 msgr2=0x55b1b02e0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02e0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 -- 172.21.15.59:0/2950581762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b00197d0 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30fa800640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009370 tx=0x7f30f40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c4b94743b2fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30df400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004c40 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f30f4004de0 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.915+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f4015070 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30f4034050 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 10 client.15396 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b02ea9c0 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f4015070 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f30f40055e0 con 0x55b1b02d5e20 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 10 client.15396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 20 client.15396 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:35.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b02ebd10 0x55b1b02ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 4 client.15396 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.916+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1b01fb040 con 0x55b1b02ebd10 2024-04-02T17:13:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.917+0000 7f30f3e00640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b02ebd10 0x55b1b02ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.917+0000 7f30f3e00640 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b02ebd10 0x55b1b02ee100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51af74ee658f32d4 server_cookie=c9425e5fa37d8509 in_seq=0 out_seq=0 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.917+0000 7f30df400640 10 client.15396 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f30df400640 1 -- 172.21.15.59:0/2950581762 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1b01fb040 con 0x55b1b02ebd10 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f30df400640 10 client.15396 handle_command_reply: tid=0 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 client.15396 shutdown 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 msgr2=0x55b1b02e00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009370 tx=0x7f30f40092e0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 shutdown_connections 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b02ebd10 0x55b1b02ee100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 --2- 172.21.15.59:0/2950581762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02e00d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f30df400640 0 client.15396 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 >> 172.21.15.59:0/2950581762 conn(0x55b1b029da50 msgr2=0x55b1b02d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.918+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 shutdown_connections 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.919+0000 7f31037af740 1 -- 172.21.15.59:0/2950581762 wait complete. 2024-04-02T17:13:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.919+0000 7f31037af740 20 client.15396 trim_cache size 0 max 16384 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.933+0000 7f31037af740 1 Processor -- start 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.933+0000 7f31037af740 1 -- start start 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1ab160 con 0x55b1b02d5e20 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1acca0 con 0x55b1b02d6810 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f31037af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1ab830 con 0x55b1b02db980 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60930/0 (socket says 172.21.15.59:60930) 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30fa800640 1 -- 172.21.15.59:0/773790634 learned_addr learned my addr 172.21.15.59:0/773790634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30fa800640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 -- 172.21.15.59:0/773790634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 msgr2=0x55b1b02e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 -- 172.21.15.59:0/773790634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 msgr2=0x55b1b02f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30f3e00640 1 -- 172.21.15.59:0/773790634 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b02ee470 con 0x55b1b02d6810 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.934+0000 7f30fa800640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:35.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.935+0000 7f30f3e00640 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02f3e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b1b02ee860 tx=0x7f30e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c907f1e1308cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.935+0000 7f30ea600640 1 -- 172.21.15.59:0/773790634 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4023030 con 0x55b1b02d6810 2024-04-02T17:13:35.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.935+0000 7f30ea600640 1 -- 172.21.15.59:0/773790634 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4005500 con 0x55b1b02d6810 2024-04-02T17:13:35.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.935+0000 7f30ea600640 1 -- 172.21.15.59:0/773790634 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e400fcd0 con 0x55b1b02d6810 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- 172.21.15.59:0/773790634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 msgr2=0x55b1b02f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02f3e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b1b02ee860 tx=0x7f30e4005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- 172.21.15.59:0/773790634 shutdown_connections 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d6810 0x55b1b02f3e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 --2- 172.21.15.59:0/773790634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d5e20 0x55b1b02f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- 172.21.15.59:0/773790634 >> 172.21.15.59:0/773790634 conn(0x55b1b029da50 msgr2=0x55b1b02d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- 172.21.15.59:0/773790634 shutdown_connections 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- 172.21.15.59:0/773790634 wait complete. 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.936+0000 7f31037af740 1 -- start start 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 Processor -- start 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1ac5d0 con 0x55b1b02d6810 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1abf00 con 0x55b1b02d5e20 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1a9140 con 0x55b1b02db980 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f31037af740 10 client.0 _ll_register_callbacks cb 0x7f30f4009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f30fa800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f30fa800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36270/0 (socket says 172.21.15.59:36270) 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f30fa800640 1 -- 172.21.15.59:0/3729308348 learned_addr learned my addr 172.21.15.59:0/3729308348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f30f3e00640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.937+0000 7f30fa800640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 -- 172.21.15.59:0/3729308348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 msgr2=0x55b1b02e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 -- 172.21.15.59:0/3729308348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 msgr2=0x55b1b02e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:35.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 -- 172.21.15.59:0/3729308348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b02e6bf0 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30fa800640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f40059b0 tx=0x7f30f4020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b251c053e10f5862 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30df400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4041060 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4005500 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.938+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f30f4003d80 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30f4011ab0 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 10 client.25502 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b0352440 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4005500 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f30f4045030 con 0x55b1b02d5e20 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 10 client.25502 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 10 client.25502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 20 client.25502 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:35.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b0352d30 0x55b1b0355150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:35.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 4 client.25502 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:13:35.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.939+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1b0248c30 con 0x55b1b0352d30 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.940+0000 7f30f3e00640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b0352d30 0x55b1b0355150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.940+0000 7f30f3e00640 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b0352d30 0x55b1b0355150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=621ad11694c0a0e1 server_cookie=32979ca534fc6de3 in_seq=0 out_seq=0 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.940+0000 7f30df400640 10 client.25502 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f30df400640 1 -- 172.21.15.59:0/3729308348 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1b0248c30 con 0x55b1b0352d30 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f30df400640 10 client.25502 handle_command_reply: tid=0 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 client.25502 shutdown 2024-04-02T17:13:35.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 msgr2=0x55b1b02e50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f40059b0 tx=0x7f30f4020040 comp rx=0 tx=0).stop 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 shutdown_connections 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55b1b0352d30 0x55b1b0355150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1b02d6810 0x55b1b02e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1b02db980 0x55b1b02e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f30df400640 0 client.25502 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 --2- 172.21.15.59:0/3729308348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1b02d5e20 0x55b1b02e50b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 >> 172.21.15.59:0/3729308348 conn(0x55b1b029da50 msgr2=0x55b1b02f2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 shutdown_connections 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 1 -- 172.21.15.59:0/3729308348 wait complete. 2024-04-02T17:13:35.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.941+0000 7f31037af740 20 client.25502 trim_cache size 0 max 16384 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 msgr2=0x7f30e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 0x7f30e00a60f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0bdf40 tx=0x7f30ec0f59d0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 msgr2=0x7f30fc199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30ec000c00 tx=0x7f30ec02c640 comp rx=0 tx=0).stop 2024-04-02T17:13:35.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 shutdown_connections 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f30e00a3c30 0x7f30e00a60f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30fc104df0 0x7f30fc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30fc1057b0 0x7f30fc1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 --2- 172.21.15.59:0/2018223669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30fc103c00 0x7f30fc199ec0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 >> 172.21.15.59:0/2018223669 conn(0x7f30fc0ff170 msgr2=0x7f30fc1080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 shutdown_connections 2024-04-02T17:13:35.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:35.942+0000 7f30fb200640 1 -- 172.21.15.59:0/2018223669 wait complete. 2024-04-02T17:13:35.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:35 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3718687095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:35.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T17:13:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:35 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3718687095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:36.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:35 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3718687095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:13:36.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 -- 172.21.15.59:0/3256330540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 msgr2=0x7f3840109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 --2- 172.21.15.59:0/3256330540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f3840109e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3838009ca0 tx=0x7f383802c0f0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 -- 172.21.15.59:0/3256330540 shutdown_connections 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 --2- 172.21.15.59:0/3256330540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f3840103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 --2- 172.21.15.59:0/3256330540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f3840109e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 --2- 172.21.15.59:0/3256330540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 0x7f3840101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 -- 172.21.15.59:0/3256330540 >> 172.21.15.59:0/3256330540 conn(0x7f38400fcf70 msgr2=0x7f38400ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 -- 172.21.15.59:0/3256330540 shutdown_connections 2024-04-02T17:13:36.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.160+0000 7f3845e00640 1 -- 172.21.15.59:0/3256330540 wait complete. 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.161+0000 7f3845e00640 1 Processor -- start 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 -- start start 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 0x7f3840197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f384019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d660 con 0x7f3840102bf0 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d4e0 con 0x7f38401035b0 2024-04-02T17:13:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f384010d7e0 con 0x7f3840101a00 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3837e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3837e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60962/0 (socket says 172.21.15.59:60962) 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3837e00640 1 -- 172.21.15.59:0/2875514737 learned_addr learned my addr 172.21.15.59:0/2875514737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3845400640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f384019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.162+0000 7f3844a00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 0x7f3840197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 -- 172.21.15.59:0/2875514737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 msgr2=0x7f3840197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 0x7f3840197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 -- 172.21.15.59:0/2875514737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 msgr2=0x7f384019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f384019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 -- 172.21.15.59:0/2875514737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384019f8e0 con 0x7f3840102bf0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3837e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f382800d950 tx=0x7f382800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f383e000640 1 -- 172.21.15.59:0/2875514737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828017030 con 0x7f3840102bf0 2024-04-02T17:13:36.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f383e000640 1 -- 172.21.15.59:0/2875514737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3828004020 con 0x7f3840102bf0 2024-04-02T17:13:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f383e000640 1 -- 172.21.15.59:0/2875514737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38280047a0 con 0x7f3840102bf0 2024-04-02T17:13:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.163+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384019fbd0 con 0x7f3840102bf0 2024-04-02T17:13:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.164+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a00d0 con 0x7f3840102bf0 2024-04-02T17:13:36.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.167+0000 7f383e000640 1 -- 172.21.15.59:0/2875514737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38280041d0 con 0x7f3840102bf0 2024-04-02T17:13:36.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.167+0000 7f384a003740 1 Processor -- start 2024-04-02T17:13:36.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 -- start start 2024-04-02T17:13:36.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bfd90 0x5587180c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558717ff7400 con 0x5587180bfd90 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558717fdbf10 con 0x5587180ba230 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f384a003740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55871801b6f0 con 0x5587180bac20 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f3845400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f3845400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36296/0 (socket says 172.21.15.59:36296) 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f3845400640 1 -- 172.21.15.59:0/2571801147 learned_addr learned my addr 172.21.15.59:0/2571801147 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f3837e00640 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bfd90 0x5587180c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.168+0000 7f3844a00640 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 -- 172.21.15.59:0/2571801147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 msgr2=0x5587180bf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180bf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 -- 172.21.15.59:0/2571801147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bfd90 msgr2=0x5587180c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bfd90 0x5587180c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 -- 172.21.15.59:0/2571801147 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558717d2f3f0 con 0x5587180ba230 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f383e000640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 0x7f38240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3845400640 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f383802c700 tx=0x7f38380323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=313a7c8b28a3d6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f383e000640 1 -- 172.21.15.59:0/2875514737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f38280cfdb0 con 0x7f3840102bf0 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3835c00640 1 -- 172.21.15.59:0/2571801147 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838043070 con 0x5587180ba230 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3835c00640 1 -- 172.21.15.59:0/2571801147 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383802cca0 con 0x5587180ba230 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3835c00640 1 -- 172.21.15.59:0/2571801147 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838039040 con 0x5587180ba230 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.169+0000 7f3844a00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 0x7f38240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 -- 172.21.15.59:0/2571801147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 msgr2=0x5587180ba650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f383802c700 tx=0x7f38380323d0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 -- 172.21.15.59:0/2571801147 shutdown_connections 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bfd90 0x5587180c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180bf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f3844a00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 0x7f38240a60f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5587180bb020 tx=0x7f3830005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 --2- 172.21.15.59:0/2571801147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180ba230 0x5587180ba650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 -- 172.21.15.59:0/2571801147 >> 172.21.15.59:0/2571801147 conn(0x55871807da50 msgr2=0x5587180b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 -- 172.21.15.59:0/2571801147 shutdown_connections 2024-04-02T17:13:36.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.170+0000 7f384a003740 1 -- 172.21.15.59:0/2571801147 wait complete. 2024-04-02T17:13:36.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 -- start start 2024-04-02T17:13:36.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 Processor -- start 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180ba230 0x5587180c4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587180176a0 con 0x5587180bac20 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558717fc3980 con 0x5587180bfd90 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558717fd6cb0 con 0x5587180ba230 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.171+0000 7f384a003740 10 client.0 _ll_register_callbacks cb 0x558717fd66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60980/0 (socket says 172.21.15.59:60980) 2024-04-02T17:13:36.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3845400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 -- 172.21.15.59:0/3794300981 learned_addr learned my addr 172.21.15.59:0/3794300981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3844a00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180ba230 0x5587180c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 -- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180ba230 msgr2=0x5587180c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180ba230 0x5587180c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 -- 172.21.15.59:0/3794300981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 msgr2=0x5587180cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 -- 172.21.15.59:0/3794300981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558717df97d0 con 0x5587180bac20 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f3837e00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3828098430 tx=0x7f38280051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc251163fac6058 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.172+0000 7f382e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382809b930 con 0x5587180bac20 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3828096b40 con 0x5587180bac20 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383802cca0 con 0x5587180bac20 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f382809c6d0 con 0x5587180bac20 2024-04-02T17:13:36.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f384a003740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587180ced70 con 0x5587180bac20 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383802cca0 con 0x5587180bac20 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.173+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f382809b420 con 0x5587180bac20 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 10 client.15414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 20 client.15414 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5587180d04c0 0x5587180d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 4 client.15414 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558717fdb040 con 0x5587180d04c0 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.174+0000 7f3844a00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5587180d04c0 0x5587180d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.175+0000 7f3844a00640 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5587180d04c0 0x5587180d28d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=634e1536a96eb35c server_cookie=3a6f5a5c5f6e337a in_seq=0 out_seq=0 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.175+0000 7f382e000640 10 client.15414 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f382e000640 1 -- 172.21.15.59:0/3794300981 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558717fdb040 con 0x5587180d04c0 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f382e000640 10 client.15414 handle_command_reply: tid=0 2024-04-02T17:13:36.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 client.15414 shutdown 2024-04-02T17:13:36.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 msgr2=0x5587180c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3828098430 tx=0x7f38280051f0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 shutdown_connections 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5587180d04c0 0x5587180d28d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180bac20 0x5587180c4950 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 --2- 172.21.15.59:0/3794300981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180ba230 0x5587180c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f382e000640 0 client.15414 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 >> 172.21.15.59:0/3794300981 conn(0x55871807da50 msgr2=0x5587180b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 shutdown_connections 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.176+0000 7f384a003740 1 -- 172.21.15.59:0/3794300981 wait complete. 2024-04-02T17:13:36.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.177+0000 7f384a003740 20 client.15414 trim_cache size 0 max 16384 2024-04-02T17:13:36.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.191+0000 7f384a003740 1 Processor -- start 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.191+0000 7f384a003740 1 -- start start 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 0x5587180b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 0x5587180d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a6dd0 con 0x5587180ba230 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a8910 con 0x5587180bac20 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f384a003740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a74a0 con 0x5587180bfd90 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60984/0 (socket says 172.21.15.59:60984) 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 -- 172.21.15.59:0/496099558 learned_addr learned my addr 172.21.15.59:0/496099558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3837e00640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 0x5587180b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3845400640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 0x5587180d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 -- 172.21.15.59:0/496099558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 msgr2=0x5587180d7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 0x5587180d7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 -- 172.21.15.59:0/496099558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 msgr2=0x5587180b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 0x5587180b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3844a00640 1 -- 172.21.15.59:0/496099558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587180bb4b0 con 0x5587180ba230 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3845400640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 0x5587180d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.192+0000 7f3837e00640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 0x5587180b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.193+0000 7f3844a00640 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3840198b70 tx=0x7f383001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e427848aac42ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.193+0000 7f3835200640 1 -- 172.21.15.59:0/496099558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300044a0 con 0x5587180ba230 2024-04-02T17:13:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.193+0000 7f3835200640 1 -- 172.21.15.59:0/496099558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383002b900 con 0x5587180ba230 2024-04-02T17:13:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.193+0000 7f3835200640 1 -- 172.21.15.59:0/496099558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383001fdb0 con 0x5587180ba230 2024-04-02T17:13:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- 172.21.15.59:0/496099558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 msgr2=0x5587180c2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3840198b70 tx=0x7f383001b000 comp rx=0 tx=0).stop 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- 172.21.15.59:0/496099558 shutdown_connections 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bfd90 0x5587180d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bac20 0x5587180b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 --2- 172.21.15.59:0/496099558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c2f60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- 172.21.15.59:0/496099558 >> 172.21.15.59:0/496099558 conn(0x55871807da50 msgr2=0x5587180b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- 172.21.15.59:0/496099558 shutdown_connections 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- 172.21.15.59:0/496099558 wait complete. 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 -- start start 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.194+0000 7f384a003740 1 Processor -- start 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a8240 con 0x5587180ba230 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a7b70 con 0x5587180bfd90 2024-04-02T17:13:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a4db0 con 0x5587180bac20 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f384a003740 10 client.0 _ll_register_callbacks cb 0x7f38280d34e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f3844a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f3844a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60988/0 (socket says 172.21.15.59:60988) 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f3844a00640 1 -- 172.21.15.59:0/3452720459 learned_addr learned my addr 172.21.15.59:0/3452720459 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f3845400640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.195+0000 7f3837e00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 -- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 msgr2=0x5587180c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 -- 172.21.15.59:0/3452720459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 msgr2=0x5587180ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 -- 172.21.15.59:0/3452720459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587180cb020 con 0x5587180ba230 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3845400640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f3844a00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3830004550 tx=0x7f3830004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d291646dc46eba47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f382e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:36.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830039020 con 0x5587180ba230 2024-04-02T17:13:36.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3830030070 con 0x5587180ba230 2024-04-02T17:13:36.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383002b900 con 0x5587180ba230 2024-04-02T17:13:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.196+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3830044050 con 0x5587180ba230 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55871813b590 con 0x5587180ba230 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383002b900 con 0x5587180ba230 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f38300051b0 con 0x5587180ba230 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 10 client.15420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 20 client.15420 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55871813bfc0 0x55871813e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 4 client.15420 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558718028c30 con 0x55871813bfc0 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.197+0000 7f3837e00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55871813bfc0 0x55871813e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f3837e00640 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55871813bfc0 0x55871813e3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55e96110c22320a3 server_cookie=ee18491cc0727e19 in_seq=0 out_seq=0 2024-04-02T17:13:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f382e000640 10 client.15420 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f382e000640 1 -- 172.21.15.59:0/3452720459 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558718028c30 con 0x55871813bfc0 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f382e000640 10 client.15420 handle_command_reply: tid=0 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f384a003740 1 client.15420 shutdown 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 msgr2=0x5587180c5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.198+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3830004550 tx=0x7f3830004580 comp rx=0 tx=0).stop 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 shutdown_connections 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55871813bfc0 0x55871813e3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587180bfd90 0x5587180ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587180bac20 0x5587180c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 --2- 172.21.15.59:0/3452720459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587180ba230 0x5587180c5b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f382e000640 0 client.15420 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 >> 172.21.15.59:0/3452720459 conn(0x55871807da50 msgr2=0x5587180b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 shutdown_connections 2024-04-02T17:13:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 1 -- 172.21.15.59:0/3452720459 wait complete. 2024-04-02T17:13:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f384a003740 20 client.15420 trim_cache size 0 max 16384 2024-04-02T17:13:36.202 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 msgr2=0x7f38240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 0x7f38240a60f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5587180bb020 tx=0x7f3830005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 msgr2=0x7f38401981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.199+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f382800d950 tx=0x7f382800de10 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 shutdown_connections 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f38240a3c30 0x7f38240a60f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3840102bf0 0x7f38401981e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38401035b0 0x7f384019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 --2- 172.21.15.59:0/2875514737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3840101a00 0x7f3840197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 >> 172.21.15.59:0/2875514737 conn(0x7f38400fcf70 msgr2=0x7f3840105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 shutdown_connections 2024-04-02T17:13:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.200+0000 7f3845e00640 1 -- 172.21.15.59:0/2875514737 wait complete. 2024-04-02T17:13:36.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T17:13:36.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.432+0000 7f6d47e00640 1 -- 172.21.15.59:0/681467240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 msgr2=0x7f6d48109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.432+0000 7f6d47e00640 1 --2- 172.21.15.59:0/681467240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d48109690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009900 tx=0x7f6d3002bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:36.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 -- 172.21.15.59:0/681467240 shutdown_connections 2024-04-02T17:13:36.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 --2- 172.21.15.59:0/681467240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 0x7f6d48110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 --2- 172.21.15.59:0/681467240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d48109690 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 --2- 172.21.15.59:0/681467240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 0x7f6d48108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 -- 172.21.15.59:0/681467240 >> 172.21.15.59:0/681467240 conn(0x7f6d48076fd0 msgr2=0x7f6d480793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 -- 172.21.15.59:0/681467240 shutdown_connections 2024-04-02T17:13:36.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.433+0000 7f6d47e00640 1 -- 172.21.15.59:0/681467240 wait complete. 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 Processor -- start 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 -- start start 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 0x7f6d4819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 0x7f6d481a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48113cb0 con 0x7f6d48109bd0 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48113b30 con 0x7f6d48108020 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.434+0000 7f6d47e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48113e30 con 0x7f6d48109210 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48950/0 (socket says 172.21.15.59:48950) 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 -- 172.21.15.59:0/4057684997 learned_addr learned my addr 172.21.15.59:0/4057684997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d47400640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 0x7f6d481a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d46a00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 0x7f6d4819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 -- 172.21.15.59:0/4057684997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 msgr2=0x7f6d4819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 0x7f6d4819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 -- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 msgr2=0x7f6d481a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 0x7f6d481a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.435+0000 7f6d3e000640 1 -- 172.21.15.59:0/4057684997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d481a5f00 con 0x7f6d48109210 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009230 tx=0x7f6d3002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d3fe00640 1 -- 172.21.15.59:0/4057684997 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30041070 con 0x7f6d48109210 2024-04-02T17:13:36.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d3fe00640 1 -- 172.21.15.59:0/4057684997 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30004450 con 0x7f6d48109210 2024-04-02T17:13:36.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d3fe00640 1 -- 172.21.15.59:0/4057684997 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003e20 con 0x7f6d48109210 2024-04-02T17:13:36.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d481a6190 con 0x7f6d48109210 2024-04-02T17:13:36.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.436+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a6680 con 0x7f6d48109210 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.439+0000 7f6d50807740 1 Processor -- start 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.439+0000 7f6d50807740 1 -- start start 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.439+0000 7f6d3fe00640 1 -- 172.21.15.59:0/4057684997 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d30044070 con 0x7f6d48109210 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036dae20 0x55e4036db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40361c400 con 0x55e4036db810 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e403600f10 con 0x55e4036dae20 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d50807740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4036406f0 con 0x55e4036e0980 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d47400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036dae20 0x55e4036db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32776/0 (socket says 172.21.15.59:32776) 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d46a00640 1 -- 172.21.15.59:0/3421519915 learned_addr learned my addr 172.21.15.59:0/3421519915 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.440+0000 7f6d3e000640 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 -- 172.21.15.59:0/3421519915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 msgr2=0x55e4036e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 -- 172.21.15.59:0/3421519915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036dae20 msgr2=0x55e4036db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036dae20 0x55e4036db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 -- 172.21.15.59:0/3421519915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4033543f0 con 0x55e4036db810 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d46a00640 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5708e67cb685f702 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d3fe00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 0x7f6d2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d37e00640 1 -- 172.21.15.59:0/3421519915 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38004220 con 0x55e4036db810 2024-04-02T17:13:36.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d37e00640 1 -- 172.21.15.59:0/3421519915 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d380043c0 con 0x55e4036db810 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d37e00640 1 -- 172.21.15.59:0/3421519915 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38019910 con 0x55e4036db810 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.441+0000 7f6d3fe00640 1 -- 172.21.15.59:0/4057684997 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6d300f1ff0 con 0x7f6d48109210 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d47400640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 0x7f6d2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 -- 172.21.15.59:0/3421519915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 msgr2=0x55e4036e0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d38011810 comp rx=0 tx=0).stop 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 -- 172.21.15.59:0/3421519915 shutdown_connections 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036db810 0x55e4036e0310 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 --2- 172.21.15.59:0/3421519915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036dae20 0x55e4036db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 -- 172.21.15.59:0/3421519915 >> 172.21.15.59:0/3421519915 conn(0x55e4036a2a50 msgr2=0x55e4036d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d47400640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 0x7f6d2c0a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55e4036dc070 tx=0x7f6d40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 -- 172.21.15.59:0/3421519915 shutdown_connections 2024-04-02T17:13:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.442+0000 7f6d50807740 1 -- 172.21.15.59:0/3421519915 wait complete. 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 -- start start 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 Processor -- start 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40363c6a0 con 0x55e4036dae20 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e4035e8980 con 0x55e4036e0980 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.443+0000 7f6d50807740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4035fbcb0 con 0x55e4036db810 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d50807740 10 client.0 _ll_register_callbacks cb 0x55e4035fb6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48968/0 (socket says 172.21.15.59:48968) 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 -- 172.21.15.59:0/3132403751 learned_addr learned my addr 172.21.15.59:0/3132403751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d47400640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 -- 172.21.15.59:0/3132403751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 msgr2=0x55e4036ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 -- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 msgr2=0x55e4036e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.444+0000 7f6d3e000640 1 -- 172.21.15.59:0/3132403751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40341e7d0 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d3e000640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002cfd0 tx=0x7f6d30005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67399e5ed2b3d4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3004b040 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6d300ba950 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d380043c0 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d300bde20 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d50807740 10 client.25549 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.445+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4036ef920 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d380043c0 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d300be540 con 0x55e4036db810 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 10 client.25549 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 10 client.25549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 20 client.25549 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e4036f0ff0 0x55e4036f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 4 client.25549 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e403600040 con 0x55e4036f0ff0 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.446+0000 7f6d46a00640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e4036f0ff0 0x55e4036f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.447+0000 7f6d46a00640 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e4036f0ff0 0x55e4036f3440 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7144d4ce76cceb9f server_cookie=36b7baaf55a480a8 in_seq=0 out_seq=0 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.447+0000 7f6d34c00640 10 client.25549 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d34c00640 1 -- 172.21.15.59:0/3132403751 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e403600040 con 0x55e4036f0ff0 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d34c00640 10 client.25549 handle_command_reply: tid=0 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 client.25549 shutdown 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 msgr2=0x55e4036e5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002cfd0 tx=0x7f6d30005730 comp rx=0 tx=0).stop 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 shutdown_connections 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e4036f0ff0 0x55e4036f3440 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036e5570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 --2- 172.21.15.59:0/3132403751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d34c00640 0 client.25549 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 >> 172.21.15.59:0/3132403751 conn(0x55e4036a2a50 msgr2=0x55e4036d5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 shutdown_connections 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 1 -- 172.21.15.59:0/3132403751 wait complete. 2024-04-02T17:13:36.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.448+0000 7f6d50807740 20 client.25549 trim_cache size 0 max 16384 2024-04-02T17:13:36.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.463+0000 7f6d50807740 1 Processor -- start 2024-04-02T17:13:36.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.463+0000 7f6d50807740 1 -- start start 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481af4c0 con 0x55e4036dae20 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481b1000 con 0x55e4036e0980 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d50807740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481afb90 con 0x55e4036db810 2024-04-02T17:13:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48980/0 (socket says 172.21.15.59:48980) 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 -- 172.21.15.59:0/4055045786 learned_addr learned my addr 172.21.15.59:0/4055045786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d46a00640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d47400640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 -- 172.21.15.59:0/4055045786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 msgr2=0x55e4036f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 -- 172.21.15.59:0/4055045786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 msgr2=0x55e4036d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d3e000640 1 -- 172.21.15.59:0/4055045786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4036d6cf0 con 0x55e4036db810 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.464+0000 7f6d47400640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.465+0000 7f6d3e000640 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006010 tx=0x7f6d3002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa94f8c42a604979 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.465+0000 7f6d37400640 1 -- 172.21.15.59:0/4055045786 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3004b050 con 0x55e4036db810 2024-04-02T17:13:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.465+0000 7f6d37400640 1 -- 172.21.15.59:0/4055045786 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d300b8e50 con 0x55e4036db810 2024-04-02T17:13:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.465+0000 7f6d37400640 1 -- 172.21.15.59:0/4055045786 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300bdc40 con 0x55e4036db810 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- 172.21.15.59:0/4055045786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 msgr2=0x55e4036ec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006010 tx=0x7f6d3002bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- 172.21.15.59:0/4055045786 shutdown_connections 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036e0980 0x55e4036f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036db810 0x55e4036ec570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 --2- 172.21.15.59:0/4055045786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- 172.21.15.59:0/4055045786 >> 172.21.15.59:0/4055045786 conn(0x55e4036a2a50 msgr2=0x55e4036d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- 172.21.15.59:0/4055045786 shutdown_connections 2024-04-02T17:13:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- 172.21.15.59:0/4055045786 wait complete. 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 -- start start 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.466+0000 7f6d50807740 1 Processor -- start 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481b0930 con 0x55e4036dae20 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481b0260 con 0x55e4036db810 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481ad4a0 con 0x55e4036e0980 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d50807740 10 client.0 _ll_register_callbacks cb 0x7f6d300f5740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d3e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36378/0 (socket says 172.21.15.59:36378) 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d3e000640 1 -- 172.21.15.59:0/2913930236 learned_addr learned my addr 172.21.15.59:0/2913930236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.467+0000 7f6d47400640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 -- 172.21.15.59:0/2913930236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 msgr2=0x55e4036eb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036eb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 -- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 msgr2=0x55e4036e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 -- 172.21.15.59:0/2913930236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4036eb900 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d3e000640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002cfd0 tx=0x7f6d300f5510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d86385577a190a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d34c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300db030 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d300b8e50 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6d300c6a10 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d3004b050 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d50807740 10 client.25535 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.468+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4036eb900 con 0x55e4036db810 2024-04-02T17:13:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d300b8e50 con 0x55e4036db810 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d300c2120 con 0x55e4036db810 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 10 client.25535 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 10 client.25535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 20 client.25535 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e403757bf0 0x55e40375a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 4 client.25535 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e40364dc30 con 0x55e403757bf0 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.469+0000 7f6d46a00640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e403757bf0 0x55e40375a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d46a00640 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e403757bf0 0x55e40375a010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fa06d53dc535107 server_cookie=20afa630d4d18058 in_seq=0 out_seq=0 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d34c00640 10 client.25535 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d34c00640 1 -- 172.21.15.59:0/2913930236 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e40364dc30 con 0x55e403757bf0 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d34c00640 10 client.25535 handle_command_reply: tid=0 2024-04-02T17:13:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 client.25535 shutdown 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 msgr2=0x55e4036ea300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002cfd0 tx=0x7f6d300f5510 comp rx=0 tx=0).stop 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 shutdown_connections 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55e403757bf0 0x55e40375a010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4036e0980 0x55e4036eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4036db810 0x55e4036ea300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 --2- 172.21.15.59:0/2913930236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4036dae20 0x55e4036e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d34c00640 0 client.25535 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.470+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 >> 172.21.15.59:0/2913930236 conn(0x55e4036a2a50 msgr2=0x55e4036d5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 shutdown_connections 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d50807740 1 -- 172.21.15.59:0/2913930236 wait complete. 2024-04-02T17:13:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d50807740 20 client.25535 trim_cache size 0 max 16384 2024-04-02T17:13:36.475 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:13:36.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 msgr2=0x7f6d2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 0x7f6d2c0a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55e4036dc070 tx=0x7f6d40009500 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 msgr2=0x7f6d4819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.471+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009230 tx=0x7f6d3002c650 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 shutdown_connections 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6d2c0a3bb0 0x7f6d2c0a6070 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d48109bd0 0x7f6d481a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d48109210 0x7f6d4819e870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 --2- 172.21.15.59:0/4057684997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d48108020 0x7f6d4819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 >> 172.21.15.59:0/4057684997 conn(0x7f6d48076fd0 msgr2=0x7f6d4810c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 shutdown_connections 2024-04-02T17:13:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.472+0000 7f6d47e00640 1 -- 172.21.15.59:0/4057684997 wait complete. 2024-04-02T17:13:36.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T17:13:36.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 -- 172.21.15.59:0/2040727017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 msgr2=0x7fdbfc105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 --2- 172.21.15.59:0/2040727017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc105390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009900 tx=0x7fdbe402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:36.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 -- 172.21.15.59:0/2040727017 shutdown_connections 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 --2- 172.21.15.59:0/2040727017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc105390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 --2- 172.21.15.59:0/2040727017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 --2- 172.21.15.59:0/2040727017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc103d20 0x7fdbfc104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 -- 172.21.15.59:0/2040727017 >> 172.21.15.59:0/2040727017 conn(0x7fdbfc0ff2b0 msgr2=0x7fdbfc1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 -- 172.21.15.59:0/2040727017 shutdown_connections 2024-04-02T17:13:36.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.710+0000 7fdc02c00640 1 -- 172.21.15.59:0/2040727017 wait complete. 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 Processor -- start 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 -- start start 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc103d20 0x7fdbfc19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc10f980 con 0x7fdbfc104f10 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc10f800 con 0x7fdbfc103d20 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.711+0000 7fdc02c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc10fb00 con 0x7fdbfc1058d0 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32804/0 (socket says 172.21.15.59:32804) 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdbf3400640 1 -- 172.21.15.59:0/4038848754 learned_addr learned my addr 172.21.15.59:0/4038848754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc103d20 0x7fdbfc19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 -- 172.21.15.59:0/4038848754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc103d20 msgr2=0x7fdbfc19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc103d20 0x7fdbfc19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 -- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 msgr2=0x7fdbfc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 -- 172.21.15.59:0/4038848754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc1a1c40 con 0x7fdbfc1058d0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdbf3400640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.712+0000 7fdc00c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc1a15d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009fd0 tx=0x7fdbf400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.713+0000 7fdbf9600640 1 -- 172.21.15.59:0/4038848754 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400ce20 con 0x7fdbfc1058d0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.713+0000 7fdbf9600640 1 -- 172.21.15.59:0/4038848754 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf4005020 con 0x7fdbfc1058d0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.713+0000 7fdbf9600640 1 -- 172.21.15.59:0/4038848754 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4010720 con 0x7fdbfc1058d0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.713+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbfc1a1f30 con 0x7fdbfc1058d0 2024-04-02T17:13:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.713+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbfc1a23f0 con 0x7fdbfc1058d0 2024-04-02T17:13:36.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.716+0000 7fdbf9600640 1 -- 172.21.15.59:0/4038848754 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdbf4005540 con 0x7fdbfc1058d0 2024-04-02T17:13:36.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.716+0000 7fdc05a34740 1 Processor -- start 2024-04-02T17:13:36.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.716+0000 7fdc05a34740 1 -- start start 2024-04-02T17:13:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedf0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedf5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedf7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562eced31400 con 0x562ecedf57d0 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562eced15f10 con 0x562ecedf0660 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdc05a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eced556f0 con 0x562ecedefc70 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedf0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedf0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49040/0 (socket says 172.21.15.59:49040) 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdbf3400640 1 -- 172.21.15.59:0/4048134206 learned_addr learned my addr 172.21.15.59:0/4048134206 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedf5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.717+0000 7fdbf3400640 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedf7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 -- 172.21.15.59:0/4048134206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 msgr2=0x562ecedf0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedf0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 -- 172.21.15.59:0/4048134206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 msgr2=0x562ecedf7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedf7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 -- 172.21.15.59:0/4048134206 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ecea693f0 con 0x562ecedf0660 2024-04-02T17:13:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf9600640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 0x7fdbe00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedf5160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009bf0 tx=0x7fdbec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a0247560f6215d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf9600640 1 -- 172.21.15.59:0/4038848754 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdbf40d0b10 con 0x7fdbfc1058d0 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf3400640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 0x7fdbe00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf1200640 1 -- 172.21.15.59:0/4048134206 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec009da0 con 0x562ecedf0660 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf1200640 1 -- 172.21.15.59:0/4048134206 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbec005020 con 0x562ecedf0660 2024-04-02T17:13:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.718+0000 7fdbf1200640 1 -- 172.21.15.59:0/4048134206 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004b60 con 0x562ecedf0660 2024-04-02T17:13:36.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 -- 172.21.15.59:0/4048134206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 msgr2=0x562ecedf5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedf5160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009bf0 tx=0x7fdbec011810 comp rx=0 tx=0).stop 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdbf3400640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 0x7fdbe00a60f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562ecedf0ec0 tx=0x7fdbe402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 -- 172.21.15.59:0/4048134206 shutdown_connections 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedf7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedf5160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 --2- 172.21.15.59:0/4048134206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedf0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 -- 172.21.15.59:0/4048134206 >> 172.21.15.59:0/4048134206 conn(0x562ecedb7a50 msgr2=0x562ecedeb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 -- 172.21.15.59:0/4048134206 shutdown_connections 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.719+0000 7fdc05a34740 1 -- 172.21.15.59:0/4048134206 wait complete. 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.720+0000 7fdc05a34740 1 -- start start 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.720+0000 7fdc05a34740 1 Processor -- start 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.720+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 0x562ecedf9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecee01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562eced516a0 con 0x562ecedf0660 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ececfd980 con 0x562ecedefc70 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eced10cb0 con 0x562ecedf57d0 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc05a34740 10 client.0 _ll_register_callbacks cb 0x562eced106a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32828/0 (socket says 172.21.15.59:32828) 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 -- 172.21.15.59:0/1316013601 learned_addr learned my addr 172.21.15.59:0/1316013601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdc00c00640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecee01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 0x562ecedf9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 -- 172.21.15.59:0/1316013601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 msgr2=0x562ecee01430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecee01430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 -- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 msgr2=0x562ecedf9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 0x562ecedf9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbf3400640 1 -- 172.21.15.59:0/1316013601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eceb337d0 con 0x562ecedf0660 2024-04-02T17:13:36.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.721+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 0x562ecedf9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdbf3400640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006010 tx=0x7fdbe4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca2059f4fb7f648 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdbe9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4051070 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdbe40053c0 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbec005020 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe405f050 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdc05a34740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.722+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ecee04810 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbec005020 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe4056460 con 0x562ecedf0660 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 20 client.15450 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee05ef0 0x562ecee08370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 4 client.15450 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:36.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.723+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eced15040 con 0x562ecee05ef0 2024-04-02T17:13:36.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.724+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee05ef0 0x562ecee08370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.724+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee05ef0 0x562ecee08370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ae0203db5797f server_cookie=28ac4b926c7a7cde in_seq=0 out_seq=0 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.724+0000 7fdbe9600640 10 client.15450 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdbe9600640 1 -- 172.21.15.59:0/1316013601 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eced15040 con 0x562ecee05ef0 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdbe9600640 10 client.15450 handle_command_reply: tid=0 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 client.15450 shutdown 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 msgr2=0x562ecedfa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4006010 tx=0x7fdbe4009260 comp rx=0 tx=0).stop 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 shutdown_connections 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee05ef0 0x562ecee08370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf0660 0x562ecedfa3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecee01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdbe9600640 0 client.15450 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 --2- 172.21.15.59:0/1316013601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedefc70 0x562ecedf9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 >> 172.21.15.59:0/1316013601 conn(0x562ecedb7a50 msgr2=0x562ecedea8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 shutdown_connections 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.725+0000 7fdc05a34740 1 -- 172.21.15.59:0/1316013601 wait complete. 2024-04-02T17:13:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.726+0000 7fdc05a34740 20 client.15450 trim_cache size 0 max 16384 2024-04-02T17:13:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 Processor -- start 2024-04-02T17:13:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 -- start start 2024-04-02T17:13:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedefc70 0x562ecedeb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecee013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1ab310 con 0x562ecedefc70 2024-04-02T17:13:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1ace50 con 0x562ecedf0660 2024-04-02T17:13:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.741+0000 7fdc05a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1ab9e0 con 0x562ecedf57d0 2024-04-02T17:13:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecee013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49062/0 (socket says 172.21.15.59:49062) 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 -- 172.21.15.59:0/306809739 learned_addr learned my addr 172.21.15.59:0/306809739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedefc70 0x562ecedeb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 -- 172.21.15.59:0/306809739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 msgr2=0x562ecee013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecee013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 -- 172.21.15.59:0/306809739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedefc70 msgr2=0x562ecedeb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedefc70 0x562ecedeb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 -- 172.21.15.59:0/306809739 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ecedebb80 con 0x562ecedf57d0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdc00c00640 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40ce550 tx=0x7fdbf40ce420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24cea97be477d940 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.742+0000 7fdbebe00640 1 -- 172.21.15.59:0/306809739 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40b5070 con 0x562ecedf57d0 2024-04-02T17:13:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdbebe00640 1 -- 172.21.15.59:0/306809739 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf40b6260 con 0x562ecedf57d0 2024-04-02T17:13:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdbebe00640 1 -- 172.21.15.59:0/306809739 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40a5770 con 0x562ecedf57d0 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 -- 172.21.15.59:0/306809739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 msgr2=0x562ecedee820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40ce550 tx=0x7fdbf40ce420 comp rx=0 tx=0).stop 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 -- 172.21.15.59:0/306809739 shutdown_connections 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedf57d0 0x562ecedee820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecee013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 --2- 172.21.15.59:0/306809739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedefc70 0x562ecedeb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 -- 172.21.15.59:0/306809739 >> 172.21.15.59:0/306809739 conn(0x562ecedb7a50 msgr2=0x562ecedf1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 -- 172.21.15.59:0/306809739 shutdown_connections 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.743+0000 7fdc05a34740 1 -- 172.21.15.59:0/306809739 wait complete. 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 -- start start 2024-04-02T17:13:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 Processor -- start 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedfe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedfef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1ac780 con 0x562ecedf57d0 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1ac0b0 con 0x562ecedf0660 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1a92f0 con 0x562ecedefc70 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.744+0000 7fdc05a34740 10 client.0 _ll_register_callbacks cb 0x7fdbe4004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32858/0 (socket says 172.21.15.59:32858) 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 -- 172.21.15.59:0/286937815 learned_addr learned my addr 172.21.15.59:0/286937815 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedfe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdbf3400640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedfef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 -- 172.21.15.59:0/286937815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 msgr2=0x562ecedfe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedfe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 -- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 msgr2=0x562ecedfef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedfef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdc00c00640 1 -- 172.21.15.59:0/286937815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ecee00450 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.745+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedfe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdc00c00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c760 tx=0x7fdbf40a5b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15cc07afb51fbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdbe9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf407f020 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdbf407bba0 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf40b6260 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbf40b5030 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdc05a34740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ecee6c4b0 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.746+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf40b6260 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdbf409c210 con 0x562ecedf57d0 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 20 client.15456 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee6cc40 0x562ecee6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 4 client.15456 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:13:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eced62c30 con 0x562ecee6cc40 2024-04-02T17:13:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.747+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee6cc40 0x562ecee6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee6cc40 0x562ecee6f030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92bb6a7b51ac5c41 server_cookie=cc6ba5cbacf8717f in_seq=0 out_seq=0 2024-04-02T17:13:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdbe9600640 10 client.15456 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdbe9600640 1 -- 172.21.15.59:0/286937815 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eced62c30 con 0x562ecee6cc40 2024-04-02T17:13:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdbe9600640 10 client.15456 handle_command_reply: tid=0 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 client.15456 shutdown 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 msgr2=0x562ecedffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c760 tx=0x7fdbf40a5b20 comp rx=0 tx=0).stop 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 shutdown_connections 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ecee6cc40 0x562ecee6f030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ecedf57d0 0x562ecedffe60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ecedf0660 0x562ecedfef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 --2- 172.21.15.59:0/286937815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ecedefc70 0x562ecedfe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdbe9600640 0 client.15456 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.748+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 >> 172.21.15.59:0/286937815 conn(0x562ecedb7a50 msgr2=0x562ecedea010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 shutdown_connections 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc05a34740 1 -- 172.21.15.59:0/286937815 wait complete. 2024-04-02T17:13:36.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc05a34740 20 client.15456 trim_cache size 0 max 16384 2024-04-02T17:13:36.752 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:13:36.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 msgr2=0x7fdbe00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 0x7fdbe00a60f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562ecedf0ec0 tx=0x7fdbe402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 msgr2=0x7fdbfc1a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc1a15d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009fd0 tx=0x7fdbf400ef10 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 shutdown_connections 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdbe00a3c30 0x7fdbe00a60f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc104f10 0x7fdbfc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1058d0 0x7fdbfc1a15d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 --2- 172.21.15.59:0/4038848754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbfc103d20 0x7fdbfc19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.749+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 >> 172.21.15.59:0/4038848754 conn(0x7fdbfc0ff2b0 msgr2=0x7fdbfc108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.750+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 shutdown_connections 2024-04-02T17:13:36.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:36.750+0000 7fdc02c00640 1 -- 172.21.15.59:0/4038848754 wait complete. 2024-04-02T17:13:36.764 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:36 smithi059 ceph-mon[31664]: pgmap v404: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:37.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:36 smithi164 ceph-mon[36499]: pgmap v404: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:36 smithi154 ceph-mon[36479]: pgmap v404: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:37.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T17:13:37.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:13:37.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T17:13:38.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 -- 172.21.15.59:0/3105093402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 msgr2=0x7f234c107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 --2- 172.21.15.59:0/3105093402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c107180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2334009900 tx=0x7f233402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 -- 172.21.15.59:0/3105093402 shutdown_connections 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 --2- 172.21.15.59:0/3105093402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c107180 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 --2- 172.21.15.59:0/3105093402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c1076c0 0x7f234c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 --2- 172.21.15.59:0/3105093402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c102460 0x7f234c104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 -- 172.21.15.59:0/3105093402 >> 172.21.15.59:0/3105093402 conn(0x7f234c0fbe70 msgr2=0x7f234c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 -- 172.21.15.59:0/3105093402 shutdown_connections 2024-04-02T17:13:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.011+0000 7f2350e00640 1 -- 172.21.15.59:0/3105093402 wait complete. 2024-04-02T17:13:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.012+0000 7f2350e00640 1 Processor -- start 2024-04-02T17:13:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.012+0000 7f2350e00640 1 -- start start 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.012+0000 7f2350e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c102460 0x7f234c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.012+0000 7f2350e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.012+0000 7f2350e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 0x7f234c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2350e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f234c10d6b0 con 0x7f234c104d90 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2350e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f234c10d530 con 0x7f234c1076c0 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2350e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f234c10d830 con 0x7f234c102460 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f234b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c102460 0x7f234c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32880/0 (socket says 172.21.15.59:32880) 2024-04-02T17:13:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 -- 172.21.15.59:0/899879955 learned_addr learned my addr 172.21.15.59:0/899879955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f234be00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 0x7f234c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 -- 172.21.15.59:0/899879955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c102460 msgr2=0x7f234c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c102460 0x7f234c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 -- 172.21.15.59:0/899879955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 msgr2=0x7f234c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 0x7f234c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f2342a00640 1 -- 172.21.15.59:0/899879955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c1a1b40 con 0x7f234c104d90 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.013+0000 7f234be00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 0x7f234c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2342a00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2334000c00 tx=0x7f233402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2348c00640 1 -- 172.21.15.59:0/899879955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334043020 con 0x7f234c104d90 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2348c00640 1 -- 172.21.15.59:0/899879955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23340054e0 con 0x7f234c104d90 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2348c00640 1 -- 172.21.15.59:0/899879955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334003c40 con 0x7f234c104d90 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c1a1dd0 con 0x7f234c104d90 2024-04-02T17:13:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.014+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1a22e0 con 0x7f234c104d90 2024-04-02T17:13:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.018+0000 7f2355016740 1 Processor -- start 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 -- start start 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2348c00640 1 -- 172.21.15.59:0/899879955 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2334046020 con 0x7f234c104d90 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c09094890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094e60 0x564c09099960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 0x564c0909c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c08ff9310 con 0x564c09099fd0 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564c08fa1690 con 0x564c09094470 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2355016740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c08ff52c0 con 0x564c09094e60 2024-04-02T17:13:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 0x564c0909c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f234be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c09094890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.019+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 0x564c0909c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32896/0 (socket says 172.21.15.59:32896) 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f2342a00640 1 -- 172.21.15.59:0/2707388216 learned_addr learned my addr 172.21.15.59:0/2707388216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234b400640 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094e60 0x564c09099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 -- 172.21.15.59:0/2707388216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094e60 msgr2=0x564c09099960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094e60 0x564c09099960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 -- 172.21.15.59:0/2707388216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 msgr2=0x564c0909c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 0x564c0909c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 -- 172.21.15.59:0/2707388216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c08dd57e0 con 0x564c09094470 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f234be00640 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c09094890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2344005fd0 tx=0x7f2344011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ccd3b31b8b7e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.020+0000 7f2340c00640 1 -- 172.21.15.59:0/2707388216 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344004d20 con 0x564c09094470 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2348c00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 0x7f23300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2340c00640 1 -- 172.21.15.59:0/2707388216 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234400f9a0 con 0x564c09094470 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2340c00640 1 -- 172.21.15.59:0/2707388216 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440044a0 con 0x564c09094470 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2348c00640 1 -- 172.21.15.59:0/899879955 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f23340f2380 con 0x7f234c104d90 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f234b400640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 0x7f23300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2355016740 1 -- 172.21.15.59:0/2707388216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 msgr2=0x564c09094890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.021+0000 7f2355016740 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c09094890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2344005fd0 tx=0x7f2344011bf0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 -- 172.21.15.59:0/2707388216 shutdown_connections 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09099fd0 0x564c0909c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094e60 0x564c09099960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 --2- 172.21.15.59:0/2707388216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c09094890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 -- 172.21.15.59:0/2707388216 >> 172.21.15.59:0/2707388216 conn(0x564c09029270 msgr2=0x564c0908fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f234b400640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 0x7f23300a60f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x564c09095260 tx=0x7f233c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 -- 172.21.15.59:0/2707388216 shutdown_connections 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 -- 172.21.15.59:0/2707388216 wait complete. 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 -- start start 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.022+0000 7f2355016740 1 Processor -- start 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09099fd0 0x564c090a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c0909cf80 con 0x564c09094e60 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564c08fb4a60 con 0x564c09094470 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564c0909d100 con 0x564c09099fd0 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2355016740 10 client.0 _ll_register_callbacks cb 0x564c08fba030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f234b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f234b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36494/0 (socket says 172.21.15.59:36494) 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f234b400640 1 -- 172.21.15.59:0/1506809801 learned_addr learned my addr 172.21.15.59:0/1506809801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32912/0 (socket says 172.21.15.59:32912) 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.023+0000 7f234be00640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09099fd0 0x564c090a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 -- 172.21.15.59:0/1506809801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09099fd0 msgr2=0x564c090a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09099fd0 0x564c090a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 -- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 msgr2=0x564c0909e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 -- 172.21.15.59:0/1506809801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c08e35f70 con 0x564c09094e60 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f234b400640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2342a00640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23340be300 tx=0x7f23340f5a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970e4762f8aab21b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2339600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:38.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23340ba040 con 0x564c09094e60 2024-04-02T17:13:38.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f23340bc900 con 0x564c09094e60 2024-04-02T17:13:38.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234400f9a0 con 0x564c09094e60 2024-04-02T17:13:38.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.024+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23340ba8a0 con 0x564c09094e60 2024-04-02T17:13:38.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:38.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c08ccd170 con 0x564c09094e60 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234400f9a0 con 0x564c09094e60 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23340c6de0 con 0x564c09094e60 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 20 client.15468 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c090aa7f0 0x564c090acc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 4 client.15468 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.025+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c09006850 con 0x564c090aa7f0 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.026+0000 7f234b400640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c090aa7f0 0x564c090acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.027+0000 7f234b400640 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c090aa7f0 0x564c090acc30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b5937b98f69949 server_cookie=6d2936092491f1d1 in_seq=0 out_seq=0 2024-04-02T17:13:38.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.027+0000 7f2339600640 10 client.15468 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.027+0000 7f2339600640 1 -- 172.21.15.59:0/1506809801 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c09006850 con 0x564c090aa7f0 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.027+0000 7f2339600640 10 client.15468 handle_command_reply: tid=0 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.027+0000 7f2355016740 1 client.15468 shutdown 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 msgr2=0x564c0909eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23340be300 tx=0x7f23340f5a50 comp rx=0 tx=0).stop 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 shutdown_connections 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c090aa7f0 0x564c090acc30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c0909eef0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09099fd0 0x564c090a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 --2- 172.21.15.59:0/1506809801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094470 0x564c0909e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2339600640 0 client.15468 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 >> 172.21.15.59:0/1506809801 conn(0x564c09029270 msgr2=0x564c0908ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 shutdown_connections 2024-04-02T17:13:38.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 1 -- 172.21.15.59:0/1506809801 wait complete. 2024-04-02T17:13:38.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.028+0000 7f2355016740 20 client.15468 trim_cache size 0 max 16384 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.050+0000 7f2355016740 1 Processor -- start 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.050+0000 7f2355016740 1 -- start start 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09029270 0x564c09093730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ab190 con 0x564c09094470 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1accd0 con 0x564c09094e60 2024-04-02T17:13:38.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2355016740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ab860 con 0x564c09029270 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f234be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f234be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36506/0 (socket says 172.21.15.59:36506) 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f234be00640 1 -- 172.21.15.59:0/1417399897 learned_addr learned my addr 172.21.15.59:0/1417399897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f2342a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32924/0 (socket says 172.21.15.59:32924) 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.051+0000 7f234b400640 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09029270 0x564c09093730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 -- 172.21.15.59:0/1417399897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09029270 msgr2=0x564c09093730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09029270 0x564c09093730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 -- 172.21.15.59:0/1417399897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 msgr2=0x564c090a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 -- 172.21.15.59:0/1417399897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c090ad2b0 con 0x564c09094e60 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f2342a00640 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f234be00640 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f234400f790 tx=0x7f23440039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73815fd317706ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.052+0000 7f233be00640 1 -- 172.21.15.59:0/1417399897 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344003cd0 con 0x564c09094e60 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f233be00640 1 -- 172.21.15.59:0/1417399897 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234401f040 con 0x564c09094e60 2024-04-02T17:13:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f233be00640 1 -- 172.21.15.59:0/1417399897 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344005940 con 0x564c09094e60 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 -- 172.21.15.59:0/1417399897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 msgr2=0x564c090b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f234400f790 tx=0x7f23440039e0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 -- 172.21.15.59:0/1417399897 shutdown_connections 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094470 0x564c090a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09094e60 0x564c090b1530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 --2- 172.21.15.59:0/1417399897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09029270 0x564c09093730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 -- 172.21.15.59:0/1417399897 >> 172.21.15.59:0/1417399897 conn(0x564c0909e570 msgr2=0x564c09093020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 -- 172.21.15.59:0/1417399897 shutdown_connections 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.053+0000 7f2355016740 1 -- 172.21.15.59:0/1417399897 wait complete. 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 -- start start 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 Processor -- start 2024-04-02T17:13:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 0x564c090a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ac600 con 0x564c09094e60 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1abf30 con 0x564c09029270 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.054+0000 7f2355016740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1a9170 con 0x564c09094470 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f2355016740 10 client.0 _ll_register_callbacks cb 0x7f2334005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f2342a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 0x564c090a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32936/0 (socket says 172.21.15.59:32936) 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36508/0 (socket says 172.21.15.59:36508) 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234be00640 1 -- 172.21.15.59:0/1160142325 learned_addr learned my addr 172.21.15.59:0/1160142325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 -- 172.21.15.59:0/1160142325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 msgr2=0x564c090a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 0x564c090a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 -- 172.21.15.59:0/1160142325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 msgr2=0x564c09110db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234b400640 1 -- 172.21.15.59:0/1160142325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c09093c70 con 0x564c09029270 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f234be00640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.055+0000 7f2342a00640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 0x564c090a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.056+0000 7f234b400640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x564c090ad480 tx=0x7f233c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce3618d94a8fe0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.056+0000 7f2339600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.056+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c003db0 con 0x564c09029270 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.056+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f233c02b380 con 0x564c09029270 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.056+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234401f040 con 0x564c09029270 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f233c02b580 con 0x564c09029270 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 10 client.25571 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c09090b30 con 0x564c09029270 2024-04-02T17:13:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234401f040 con 0x564c09029270 2024-04-02T17:13:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f233c004a10 con 0x564c09029270 2024-04-02T17:13:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 10 client.25571 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 20 client.25571 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c09099fd0 0x564c09114d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 4 client.25571 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.057+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c09090130 con 0x564c09099fd0 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.058+0000 7f2342a00640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c09099fd0 0x564c09114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.058+0000 7f2342a00640 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c09099fd0 0x564c09114d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c0962ca2857a75 server_cookie=7cf29698c1319ba3 in_seq=0 out_seq=0 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.058+0000 7f2339600640 10 client.25571 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2339600640 1 -- 172.21.15.59:0/1160142325 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c09090130 con 0x564c09099fd0 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2339600640 10 client.25571 handle_command_reply: tid=0 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 client.25571 shutdown 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 msgr2=0x564c090a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x564c090ad480 tx=0x7f233c0046c0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 shutdown_connections 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564c09099fd0 0x564c09114d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c09094e60 0x564c09110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c09094470 0x564c090a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 --2- 172.21.15.59:0/1160142325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c09029270 0x564c090a4e50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2339600640 0 client.25571 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 >> 172.21.15.59:0/1160142325 conn(0x564c0909e570 msgr2=0x564c0908e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 shutdown_connections 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.060+0000 7f2355016740 1 -- 172.21.15.59:0/1160142325 wait complete. 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2355016740 20 client.25571 trim_cache size 0 max 16384 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:13:38.062 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-02T17:13:38.063 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "95587257-c9dd-4a61-9042-291ab39c735f", 2024-04-02T17:13:38.063 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-02T17:13:38.063 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:13:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 msgr2=0x7f23300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 0x7f23300a60f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x564c09095260 tx=0x7f233c005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 msgr2=0x7f234c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2334000c00 tx=0x7f233402c650 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 shutdown_connections 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f23300a3c30 0x7f23300a60f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f234c104d90 0x7f234c19a460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f234c1076c0 0x7f234c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 --2- 172.21.15.59:0/899879955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f234c102460 0x7f234c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.061+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 >> 172.21.15.59:0/899879955 conn(0x7f234c0fbe70 msgr2=0x7f234c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.062+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 shutdown_connections 2024-04-02T17:13:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.062+0000 7f2350e00640 1 -- 172.21.15.59:0/899879955 wait complete. 2024-04-02T17:13:38.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:13:38.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 -- 172.21.15.59:0/3370954445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73801058d0 msgr2=0x7f738010c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 --2- 172.21.15.59:0/3370954445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73801058d0 0x7f738010c160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f737c009ca0 tx=0x7f737c02c100 comp rx=0 tx=0).stop 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 -- 172.21.15.59:0/3370954445 shutdown_connections 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 --2- 172.21.15.59:0/3370954445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7380104f10 0x7f7380105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 --2- 172.21.15.59:0/3370954445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73801058d0 0x7f738010c160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 --2- 172.21.15.59:0/3370954445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 0x7f7380104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 -- 172.21.15.59:0/3370954445 >> 172.21.15.59:0/3370954445 conn(0x7f73800ff2b0 msgr2=0x7f73801016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 -- 172.21.15.59:0/3370954445 shutdown_connections 2024-04-02T17:13:38.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.305+0000 7f7388800640 1 -- 172.21.15.59:0/3370954445 wait complete. 2024-04-02T17:13:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.306+0000 7f7388800640 1 Processor -- start 2024-04-02T17:13:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.306+0000 7f7388800640 1 -- start start 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 0x7f7380199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73801058d0 0x7f73801a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f738010f9a0 con 0x7f73801058d0 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f738010f820 con 0x7f7380103d20 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7388800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f738010fb20 con 0x7f7380104f10 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49144/0 (socket says 172.21.15.59:49144) 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 -- 172.21.15.59:0/3165738453 learned_addr learned my addr 172.21.15.59:0/3165738453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7386800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73801058d0 0x7f73801a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385e00640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 0x7f7380199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 -- 172.21.15.59:0/3165738453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 msgr2=0x7f7380199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 0x7f7380199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 -- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73801058d0 msgr2=0x7f73801a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73801058d0 0x7f73801a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.307+0000 7f7385400640 1 -- 172.21.15.59:0/3165738453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73801a1bd0 con 0x7f7380104f10 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f7385400640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f737000e9f0 tx=0x7f737000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f737aa00640 1 -- 172.21.15.59:0/3165738453 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737000cd80 con 0x7f7380104f10 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f737aa00640 1 -- 172.21.15.59:0/3165738453 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7370010910 con 0x7f7380104f10 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73801a1ec0 con 0x7f7380104f10 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f737aa00640 1 -- 172.21.15.59:0/3165738453 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370018990 con 0x7f7380104f10 2024-04-02T17:13:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.308+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801a2380 con 0x7f7380104f10 2024-04-02T17:13:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.312+0000 7f738b5b1740 1 Processor -- start 2024-04-02T17:13:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.312+0000 7f738b5b1740 1 -- start start 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 0x55f7ba6c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba621770 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba5c9700 con 0x55f7ba6c2330 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f738b5b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba61d720 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f737aa00640 1 -- 172.21.15.59:0/3165738453 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7370010ac0 con 0x7f7380104f10 2024-04-02T17:13:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f7385e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f7385e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32976/0 (socket says 172.21.15.59:32976) 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f7385e00640 1 -- 172.21.15.59:0/2137385982 learned_addr learned my addr 172.21.15.59:0/2137385982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.313+0000 7f7386800640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385400640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 0x55f7ba6c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 -- 172.21.15.59:0/2137385982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 msgr2=0x55f7ba6bcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6bcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 -- 172.21.15.59:0/2137385982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 msgr2=0x55f7ba6c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 0x55f7ba6c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 -- 172.21.15.59:0/2137385982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ba3ff7d0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7386800640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6bcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385400640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 0x55f7ba6c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:13:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f7385e00640 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7374009ca0 tx=0x7f7374011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88a29a6ab5d825 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f736f000640 1 -- 172.21.15.59:0/2137385982 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374004220 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f736f000640 1 -- 172.21.15.59:0/2137385982 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73740043c0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.314+0000 7f736f000640 1 -- 172.21.15.59:0/2137385982 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374019900 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f737aa00640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 0x7f73680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f737aa00640 1 -- 172.21.15.59:0/3165738453 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f73700d07d0 con 0x7f7380104f10 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f7386800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 0x7f73680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 -- 172.21.15.59:0/2137385982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 msgr2=0x55f7ba6c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7374009ca0 tx=0x7f7374011810 comp rx=0 tx=0).stop 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 -- 172.21.15.59:0/2137385982 shutdown_connections 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c1cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6c2330 0x55f7ba6c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 --2- 172.21.15.59:0/2137385982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6bcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.315+0000 7f738b5b1740 1 -- 172.21.15.59:0/2137385982 >> 172.21.15.59:0/2137385982 conn(0x55f7ba6b7d40 msgr2=0x55f7ba6b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.316+0000 7f738b5b1740 1 -- 172.21.15.59:0/2137385982 shutdown_connections 2024-04-02T17:13:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.316+0000 7f738b5b1740 1 -- 172.21.15.59:0/2137385982 wait complete. 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.316+0000 7f7386800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 0x7f73680a6070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f737c02c640 tx=0x7f737c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.316+0000 7f738b5b1740 1 -- start start 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.316+0000 7f738b5b1740 1 Processor -- start 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba6c52e0 con 0x55f7ba6c2330 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba5dcac0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba6c5460 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f738b5b1740 10 client.0 _ll_register_callbacks cb 0x55f7ba5df180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f7386800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f7385400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f7386800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32986/0 (socket says 172.21.15.59:32986) 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f7386800640 1 -- 172.21.15.59:0/681700481 learned_addr learned my addr 172.21.15.59:0/681700481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.317+0000 7f7385e00640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 -- 172.21.15.59:0/681700481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 msgr2=0x55f7ba6c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 -- 172.21.15.59:0/681700481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 msgr2=0x55f7ba6ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 -- 172.21.15.59:0/681700481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ba3353f0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7386800640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385e00640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7385400640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c7260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f737000c6a0 tx=0x7f7370011ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd99c0267becba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7363400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73700d0dc0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f73700bbbb0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.318+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73740043c0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73700b7070 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 10 client.25580 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7ba2f50f0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73740043c0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f737009bef0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 10 client.25580 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 20 client.25580 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba6d2670 0x55f7ba6d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 4 client.25580 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.319+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7ba5c2630 con 0x55f7ba6d2670 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.320+0000 7f7385e00640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba6d2670 0x55f7ba6d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.320+0000 7f7385e00640 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba6d2670 0x55f7ba6d4ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2b4b7c0c2506dc server_cookie=3e63e05024a8cc11 in_seq=0 out_seq=0 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.320+0000 7f7363400640 10 client.25580 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f7363400640 1 -- 172.21.15.59:0/681700481 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7ba5c2630 con 0x55f7ba6d2670 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f7363400640 10 client.25580 handle_command_reply: tid=0 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 client.25580 shutdown 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 msgr2=0x55f7ba6c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c7260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f737000c6a0 tx=0x7f7370011ee0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 shutdown_connections 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba6d2670 0x55f7ba6d4ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6c7260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 --2- 172.21.15.59:0/681700481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 >> 172.21.15.59:0/681700481 conn(0x55f7ba6b7d40 msgr2=0x55f7ba6b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.321+0000 7f7363400640 0 client.25580 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.322+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 shutdown_connections 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.322+0000 7f738b5b1740 1 -- 172.21.15.59:0/681700481 wait complete. 2024-04-02T17:13:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.322+0000 7f738b5b1740 20 client.25580 trim_cache size 0 max 16384 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.341+0000 7f738b5b1740 1 Processor -- start 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.341+0000 7f738b5b1740 1 -- start start 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.341+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.341+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.341+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f738b5b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73801ab310 con 0x55f7ba6c2330 2024-04-02T17:13:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f738b5b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73801ace50 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f738b5b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73801ab9e0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32992/0 (socket says 172.21.15.59:32992) 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 -- 172.21.15.59:0/3124225739 learned_addr learned my addr 172.21.15.59:0/3124225739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7385400640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7385e00640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 -- 172.21.15.59:0/3124225739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 msgr2=0x55f7ba6ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 -- 172.21.15.59:0/3124225739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 msgr2=0x55f7ba6b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7386800640 1 -- 172.21.15.59:0/3124225739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ba6d5130 con 0x55f7ba6c2330 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.342+0000 7f7385e00640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.343+0000 7f7386800640 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f737c037040 tx=0x7f737c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d260140e7a0c9ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.343+0000 7f736e600640 1 -- 172.21.15.59:0/3124225739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c039040 con 0x55f7ba6c2330 2024-04-02T17:13:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.343+0000 7f736e600640 1 -- 172.21.15.59:0/3124225739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f737c052070 con 0x55f7ba6c2330 2024-04-02T17:13:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.343+0000 7f736e600640 1 -- 172.21.15.59:0/3124225739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c0407d0 con 0x55f7ba6c2330 2024-04-02T17:13:38.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- 172.21.15.59:0/3124225739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 msgr2=0x55f7ba6b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f737c037040 tx=0x7f737c040000 comp rx=0 tx=0).stop 2024-04-02T17:13:38.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- 172.21.15.59:0/3124225739 shutdown_connections 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6b6c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3124225739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- 172.21.15.59:0/3124225739 >> 172.21.15.59:0/3124225739 conn(0x55f7ba6b7d40 msgr2=0x55f7ba6be750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- 172.21.15.59:0/3124225739 shutdown_connections 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- 172.21.15.59:0/3124225739 wait complete. 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 -- start start 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.344+0000 7f738b5b1740 1 Processor -- start 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73801ac780 con 0x55f7ba6c2330 2024-04-02T17:13:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73801ac0b0 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73801a92f0 con 0x55f7ba6bd1c0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f738b5b1740 10 client.0 _ll_register_callbacks cb 0x7f7370098d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f7385e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f7386800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f7385e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36552/0 (socket says 172.21.15.59:36552) 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f7385e00640 1 -- 172.21.15.59:0/3898654014 learned_addr learned my addr 172.21.15.59:0/3898654014 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.345+0000 7f7385400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 -- 172.21.15.59:0/3898654014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 msgr2=0x55f7ba6cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 -- 172.21.15.59:0/3898654014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 msgr2=0x55f7ba6ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 -- 172.21.15.59:0/3898654014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ba6cd2f0 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7385e00640 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7374013010 tx=0x7f73740097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e361d09ec80e388 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7363400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374031030 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f737c052070 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.346+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f737401f040 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7374005040 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 10 client.25586 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7ba741400 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f737c052070 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73740035f0 con 0x55f7ba6bc7d0 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 10 client.25586 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 20 client.25586 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba741ae0 0x55f7ba743f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:38.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 4 client.25586 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.347+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7ba62ecb0 con 0x55f7ba741ae0 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f7385400640 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba741ae0 0x55f7ba743f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f7385400640 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba741ae0 0x55f7ba743f00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4821047abed74b7e server_cookie=16c6d65cd52be6e6 in_seq=0 out_seq=0 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f7363400640 10 client.25586 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f7363400640 1 -- 172.21.15.59:0/3898654014 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f7ba62ecb0 con 0x55f7ba741ae0 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f7363400640 10 client.25586 handle_command_reply: tid=0 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.348+0000 7f738b5b1740 1 client.25586 shutdown 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 msgr2=0x55f7ba6cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7374013010 tx=0x7f73740097e0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 shutdown_connections 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7ba741ae0 0x55f7ba743f00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7ba6c2330 0x55f7ba6ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f7ba6bd1c0 0x55f7ba6cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 --2- 172.21.15.59:0/3898654014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f7ba6bc7d0 0x55f7ba6cb7b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 >> 172.21.15.59:0/3898654014 conn(0x55f7ba6b7d40 msgr2=0x55f7ba6bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f7363400640 0 client.25586 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 shutdown_connections 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 1 -- 172.21.15.59:0/3898654014 wait complete. 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.349+0000 7f738b5b1740 20 client.25586 trim_cache size 0 max 16384 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout: "95587257-c9dd-4a61-9042-291ab39c735f": { 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-02T17:13:38.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "95587257-c9dd-4a61-9042-291ab39c735f", 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 msgr2=0x7f73680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 0x7f73680a6070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f737c02c640 tx=0x7f737c02e040 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 msgr2=0x7f738019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f737000e9f0 tx=0x7f737000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 shutdown_connections 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f73680a3bb0 0x7f73680a6070 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73801058d0 0x7f73801a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7380104f10 0x7f738019a4f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 --2- 172.21.15.59:0/3165738453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7380103d20 0x7f7380199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 >> 172.21.15.59:0/3165738453 conn(0x7f73800ff2b0 msgr2=0x7f7380108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 shutdown_connections 2024-04-02T17:13:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:13:38.350+0000 7f7388800640 1 -- 172.21.15.59:0/3165738453 wait complete. 2024-04-02T17:13:38.366 INFO:tasks.cephfs.filesystem:scrub status for tag:95587257-c9dd-4a61-9042-291ab39c735f - {'path': '/', 'tag': '95587257-c9dd-4a61-9042-291ab39c735f', 'options': 'recursive,force'} 2024-04-02T17:13:39.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:38 smithi059 ceph-mon[31664]: pgmap v405: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:39.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:38 smithi164 ceph-mon[36499]: pgmap v405: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:39.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:38 smithi154 ceph-mon[36479]: pgmap v405: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:13:40.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:39 smithi059 ceph-mon[31664]: scrub queued for path: / 2024-04-02T17:13:40.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:39 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:40.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:39 smithi059 ceph-mon[31664]: scrub summary: active paths [/] 2024-04-02T17:13:40.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:39 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:40.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:39 smithi164 ceph-mon[36499]: scrub queued for path: / 2024-04-02T17:13:40.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:39 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:40.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:39 smithi164 ceph-mon[36499]: scrub summary: active paths [/] 2024-04-02T17:13:40.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:39 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:40.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:39 smithi154 ceph-mon[36479]: scrub queued for path: / 2024-04-02T17:13:40.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:39 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:40.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:39 smithi154 ceph-mon[36479]: scrub summary: active paths [/] 2024-04-02T17:13:40.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:39 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:13:41.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:40 smithi059 ceph-mon[31664]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:41.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:40 smithi059 ceph-mon[31664]: scrub summary: idle 2024-04-02T17:13:41.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:40 smithi164 ceph-mon[36499]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:41.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:40 smithi164 ceph-mon[36499]: scrub summary: idle 2024-04-02T17:13:41.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:40 smithi154 ceph-mon[36479]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:13:41.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:40 smithi154 ceph-mon[36479]: scrub summary: idle 2024-04-02T17:13:43.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:42 smithi059 ceph-mon[31664]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:13:43.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:42 smithi164 ceph-mon[36499]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:13:43.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:42 smithi154 ceph-mon[36479]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 -- 172.21.15.59:0/4237095411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404100890 msgr2=0x7f8404100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 --2- 172.21.15.59:0/4237095411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404100890 0x7f8404100cb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009900 tx=0x7f83f402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 -- 172.21.15.59:0/4237095411 shutdown_connections 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 --2- 172.21.15.59:0/4237095411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f8404108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 --2- 172.21.15.59:0/4237095411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404101a80 0x7f8404101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 --2- 172.21.15.59:0/4237095411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404100890 0x7f8404100cb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.662+0000 7f8408c00640 1 -- 172.21.15.59:0/4237095411 >> 172.21.15.59:0/4237095411 conn(0x7f84040fbe00 msgr2=0x7f84040fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.663+0000 7f8408c00640 1 -- 172.21.15.59:0/4237095411 shutdown_connections 2024-04-02T17:13:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.663+0000 7f8408c00640 1 -- 172.21.15.59:0/4237095411 wait complete. 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 Processor -- start 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 -- start start 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 0x7f8404199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 0x7f840419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f84041a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f840410b410 con 0x7f8404101a80 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f840410b290 con 0x7f8404100890 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.664+0000 7f8408c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f840410b590 con 0x7f8404102440 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f83faa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 0x7f840419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 0x7f8404199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 0x7f8404199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46258/0 (socket says 172.21.15.59:46258) 2024-04-02T17:13:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f84041a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f83faa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 0x7f840419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44090/0 (socket says 172.21.15.59:44090) 2024-04-02T17:13:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403400640 1 -- 172.21.15.59:0/2528297889 learned_addr learned my addr 172.21.15.59:0/2528297889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 -- 172.21.15.59:0/2528297889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 msgr2=0x7f8404199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 0x7f8404199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 -- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 msgr2=0x7f840419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 0x7f840419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 -- 172.21.15.59:0/2528297889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041a1aa0 con 0x7f8404102440 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.665+0000 7f8403e00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f84041a1430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098f0 tx=0x7f83fc009db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.666+0000 7f8400c00640 1 -- 172.21.15.59:0/2528297889 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc003b70 con 0x7f8404102440 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.666+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041a1d90 con 0x7f8404102440 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.666+0000 7f8400c00640 1 -- 172.21.15.59:0/2528297889 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc003d10 con 0x7f8404102440 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.666+0000 7f8400c00640 1 -- 172.21.15.59:0/2528297889 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc014420 con 0x7f8404102440 2024-04-02T17:13:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.666+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a22b0 con 0x7f8404102440 2024-04-02T17:13:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.669+0000 7f840ce80740 1 Processor -- start 2024-04-02T17:13:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f8400c00640 1 -- 172.21.15.59:0/2528297889 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83fc00e830 con 0x7f8404102440 2024-04-02T17:13:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 -- start start 2024-04-02T17:13:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cf48941d70 con 0x55cf489e1c60 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cf488ea2a0 con 0x55cf489dc100 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f840ce80740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf4893dd20 con 0x55cf489dcaf0 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.670+0000 7f83faa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:46260/0 (socket says 172.21.15.59:46260) 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 -- 172.21.15.59:0/2169959745 learned_addr learned my addr 172.21.15.59:0/2169959745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f8403400640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 -- 172.21.15.59:0/2169959745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 msgr2=0x55cf489e15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 -- 172.21.15.59:0/2169959745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 msgr2=0x55cf489e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f83faa00640 1 -- 172.21.15.59:0/2169959745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf488fe840 con 0x55cf489dc100 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f8403400640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.671+0000 7f8400c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 0x7f83e80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f83faa00640 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83ec012840 tx=0x7f83ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4e1de833e518f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f8400c00640 1 -- 172.21.15.59:0/2528297889 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f83fc0d00f0 con 0x7f8404102440 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f8403400640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 0x7f83e80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f83f3e00640 1 -- 172.21.15.59:0/2169959745 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec010770 con 0x55cf489dc100 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f83f3e00640 1 -- 172.21.15.59:0/2169959745 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ec010df0 con 0x55cf489dc100 2024-04-02T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.672+0000 7f83f3e00640 1 -- 172.21.15.59:0/2169959745 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00eb00 con 0x55cf489dc100 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- 172.21.15.59:0/2169959745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 msgr2=0x55cf489dc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83ec012840 tx=0x7f83ec0046c0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f8403400640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 0x7f83e80a6060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83f402c8e0 tx=0x7f83f402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- 172.21.15.59:0/2169959745 shutdown_connections 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 --2- 172.21.15.59:0/2169959745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dc100 0x55cf489dc520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- 172.21.15.59:0/2169959745 >> 172.21.15.59:0/2169959745 conn(0x55cf489d7670 msgr2=0x55cf489d7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- 172.21.15.59:0/2169959745 shutdown_connections 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- 172.21.15.59:0/2169959745 wait complete. 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.673+0000 7f840ce80740 1 -- start start 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 Processor -- start 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 0x55cf489edc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cf489e4c10 con 0x55cf489dc100 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cf488fd660 con 0x55cf489e1c60 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf489e4d90 con 0x55cf489dcaf0 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f840ce80740 10 client.0 _ll_register_callbacks cb 0x55cf488fd050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.674+0000 7f8403400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44112/0 (socket says 172.21.15.59:44112) 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 -- 172.21.15.59:0/3730920441 learned_addr learned my addr 172.21.15.59:0/3730920441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403e00640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 0x55cf489edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f83faa00640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 -- 172.21.15.59:0/3730920441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 msgr2=0x55cf489e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 -- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 msgr2=0x55cf489edc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 0x55cf489edc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 -- 172.21.15.59:0/3730920441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4890ce80 con 0x55cf489dc100 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403e00640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 0x55cf489edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f8403400640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f83f402e010 tx=0x7f83f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a19b1de71740b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.675+0000 7f83f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f404e030 con 0x55cf489dc100 2024-04-02T17:13:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f83f4004b50 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec010df0 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83f4004ef0 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f840ce80740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf48656210 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.676+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ec010df0 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f83f4003c30 con 0x55cf489dc100 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 20 client.15504 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf489f2570 0x55cf489f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 4 client.15504 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf488f1cd0 con 0x55cf489f2570 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.677+0000 7f83faa00640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf489f2570 0x55cf489f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.678+0000 7f83faa00640 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf489f2570 0x55cf489f4980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b548da8eaf0349a server_cookie=49ed70f9df75f5f in_seq=0 out_seq=0 2024-04-02T17:13:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.678+0000 7f83f0c00640 10 client.15504 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f83f0c00640 1 -- 172.21.15.59:0/3730920441 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf488f1cd0 con 0x55cf489f2570 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f83f0c00640 10 client.15504 handle_command_reply: tid=0 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 client.15504 shutdown 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 msgr2=0x55cf489e66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f83f402e010 tx=0x7f83f402bc30 comp rx=0 tx=0).stop 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 shutdown_connections 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf489f2570 0x55cf489f4980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489e1c60 0x55cf489edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dcaf0 0x55cf489e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 --2- 172.21.15.59:0/3730920441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489dc100 0x55cf489e66a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 >> 172.21.15.59:0/3730920441 conn(0x55cf489d7670 msgr2=0x55cf489d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f83f0c00640 0 client.15504 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 shutdown_connections 2024-04-02T17:13:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.679+0000 7f840ce80740 1 -- 172.21.15.59:0/3730920441 wait complete. 2024-04-02T17:13:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.680+0000 7f840ce80740 20 client.15504 trim_cache size 0 max 16384 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.696+0000 7f840ce80740 1 Processor -- start 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.696+0000 7f840ce80740 1 -- start start 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84041ab210 con 0x55cf489e1c60 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84041acd50 con 0x55cf489dcaf0 2024-04-02T17:13:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f840ce80740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84041ab8e0 con 0x55cf489dc100 2024-04-02T17:13:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f83faa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52838/0 (socket says 172.21.15.59:52838) 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 -- 172.21.15.59:0/1752680172 learned_addr learned my addr 172.21.15.59:0/1752680172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 -- 172.21.15.59:0/1752680172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 msgr2=0x55cf489edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 -- 172.21.15.59:0/1752680172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 msgr2=0x55cf489e5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.697+0000 7f8403400640 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.698+0000 7f8403400640 1 -- 172.21.15.59:0/1752680172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf48722620 con 0x55cf489dc100 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.698+0000 7f83faa00640 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.698+0000 7f8403400640 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f40046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78eba90f8bde092 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.698+0000 7f83f3400640 1 -- 172.21.15.59:0/1752680172 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f404e040 con 0x55cf489dc100 2024-04-02T17:13:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.699+0000 7f83f3400640 1 -- 172.21.15.59:0/1752680172 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4061020 con 0x55cf489dc100 2024-04-02T17:13:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.699+0000 7f83f3400640 1 -- 172.21.15.59:0/1752680172 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4005380 con 0x55cf489dc100 2024-04-02T17:13:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 -- 172.21.15.59:0/1752680172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 msgr2=0x55cf489e69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f40046e0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 -- 172.21.15.59:0/1752680172 shutdown_connections 2024-04-02T17:13:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 --2- 172.21.15.59:0/1752680172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489e69c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 -- 172.21.15.59:0/1752680172 >> 172.21.15.59:0/1752680172 conn(0x55cf489d7670 msgr2=0x55cf489e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 -- 172.21.15.59:0/1752680172 shutdown_connections 2024-04-02T17:13:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.700+0000 7f840ce80740 1 -- 172.21.15.59:0/1752680172 wait complete. 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.701+0000 7f840ce80740 1 -- start start 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.701+0000 7f840ce80740 1 Processor -- start 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84041ac680 con 0x55cf489e1c60 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84041abfb0 con 0x55cf489dcaf0 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84041a91f0 con 0x55cf489dc100 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f840ce80740 10 client.0 _ll_register_callbacks cb 0x7f83f40044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52840/0 (socket says 172.21.15.59:52840) 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 -- 172.21.15.59:0/2095628814 learned_addr learned my addr 172.21.15.59:0/2095628814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f83faa00640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 -- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 msgr2=0x55cf489ea450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489ea450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 -- 172.21.15.59:0/2095628814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 msgr2=0x55cf489ea990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489ea990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f8403400640 1 -- 172.21.15.59:0/2095628814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf489eafd0 con 0x55cf489dc100 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.702+0000 7f83faa00640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f8403400640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005ec0 tx=0x7f83f4005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa08ddd1b0219f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f83f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f406b060 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f83f4034c90 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f4061020 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83f404e040 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f840ce80740 10 client.25627 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.703+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf48a57bb0 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4061020 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f83f4003bc0 con 0x55cf489dc100 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 10 client.25627 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 20 client.25627 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf48a58500 0x55cf48a5a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 4 client.25627 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.704+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf4894f2b0 con 0x55cf48a58500 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.705+0000 7f83faa00640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf48a58500 0x55cf48a5a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.705+0000 7f83faa00640 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf48a58500 0x55cf48a5a920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20295bee7600847 server_cookie=a310cf2bbce1d987 in_seq=0 out_seq=0 2024-04-02T17:13:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.705+0000 7f83f0c00640 10 client.25627 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f83f0c00640 1 -- 172.21.15.59:0/2095628814 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf4894f2b0 con 0x55cf48a58500 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f83f0c00640 10 client.25627 handle_command_reply: tid=0 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 client.25627 shutdown 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 msgr2=0x55cf489ed960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005ec0 tx=0x7f83f4005970 comp rx=0 tx=0).stop 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 shutdown_connections 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cf48a58500 0x55cf48a5a920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf489e1c60 0x55cf489ea990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf489dcaf0 0x55cf489ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 --2- 172.21.15.59:0/2095628814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf489dc100 0x55cf489ed960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f83f0c00640 0 client.25627 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.713+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 >> 172.21.15.59:0/2095628814 conn(0x55cf489d7670 msgr2=0x55cf489da8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.714+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 shutdown_connections 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.714+0000 7f840ce80740 1 -- 172.21.15.59:0/2095628814 wait complete. 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.714+0000 7f840ce80740 20 client.25627 trim_cache size 0 max 16384 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:13:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 msgr2=0x7f83e80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 0x7f83e80a6060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83f402c8e0 tx=0x7f83f402e040 comp rx=0 tx=0).stop 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 msgr2=0x7f84041a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f84041a1430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098f0 tx=0x7f83fc009db0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 shutdown_connections 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f83e80a3ba0 0x7f83e80a6060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8404101a80 0x7f840419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8404102440 0x7f84041a1430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 --2- 172.21.15.59:0/2528297889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8404100890 0x7f8404199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 >> 172.21.15.59:0/2528297889 conn(0x7f84040fbe00 msgr2=0x7f8404190ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 shutdown_connections 2024-04-02T17:13:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:43.715+0000 7f8408c00640 1 -- 172.21.15.59:0/2528297889 wait complete. 2024-04-02T17:13:45.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:44 smithi059 ceph-mon[31664]: pgmap v408: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T17:13:45.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:44 smithi164 ceph-mon[36499]: pgmap v408: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T17:13:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:44 smithi154 ceph-mon[36479]: pgmap v408: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T17:13:47.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:46 smithi059 ceph-mon[31664]: pgmap v409: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:46 smithi164 ceph-mon[36499]: pgmap v409: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:47.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:46 smithi154 ceph-mon[36479]: pgmap v409: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:13:49.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:48 smithi059 ceph-mon[31664]: pgmap v410: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-02T17:13:49.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:48 smithi164 ceph-mon[36499]: pgmap v410: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-02T17:13:49.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:48 smithi154 ceph-mon[36479]: pgmap v410: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-02T17:13:51.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:50 smithi059 ceph-mon[31664]: pgmap v411: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:51.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:50 smithi164 ceph-mon[36499]: pgmap v411: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:51.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:50 smithi154 ceph-mon[36479]: pgmap v411: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:53.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:52 smithi059 ceph-mon[31664]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:52 smithi164 ceph-mon[36499]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:53.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:52 smithi154 ceph-mon[36479]: pgmap v412: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:13:55.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:54 smithi059 ceph-mon[31664]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:55.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:54 smithi164 ceph-mon[36499]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:54 smithi154 ceph-mon[36479]: pgmap v413: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 -- 172.21.15.59:0/2652485929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124101c30 msgr2=0x7f51241020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 --2- 172.21.15.59:0/2652485929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124101c30 0x7f51241020b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5110009900 tx=0x7f511002bc50 comp rx=0 tx=0).stop 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 -- 172.21.15.59:0/2652485929 shutdown_connections 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 --2- 172.21.15.59:0/2652485929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f5124108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 --2- 172.21.15.59:0/2652485929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124101c30 0x7f51241020b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 --2- 172.21.15.59:0/2652485929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124100a40 0x7f5124100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 -- 172.21.15.59:0/2652485929 >> 172.21.15.59:0/2652485929 conn(0x7f51240fbfb0 msgr2=0x7f51240fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 -- 172.21.15.59:0/2652485929 shutdown_connections 2024-04-02T17:13:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.874+0000 7f5129a00640 1 -- 172.21.15.59:0/2652485929 wait complete. 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.875+0000 7f5129a00640 1 Processor -- start 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 -- start start 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124100a40 0x7f5124199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124101c30 0x7f512419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f512410b610 con 0x7f51241025f0 2024-04-02T17:13:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f512410b490 con 0x7f5124101c30 2024-04-02T17:13:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.876+0000 7f5129a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f512410b790 con 0x7f5124100a40 2024-04-02T17:13:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54140/0 (socket says 172.21.15.59:54140) 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 -- 172.21.15.59:0/3673485043 learned_addr learned my addr 172.21.15.59:0/3673485043 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5122a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124100a40 0x7f5124199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5122000640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124101c30 0x7f512419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 -- 172.21.15.59:0/3673485043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124100a40 msgr2=0x7f5124199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124100a40 0x7f5124199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 -- 172.21.15.59:0/3673485043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124101c30 msgr2=0x7f512419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124101c30 0x7f512419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.877+0000 7f5123400640 1 -- 172.21.15.59:0/3673485043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241a1b80 con 0x7f51241025f0 2024-04-02T17:13:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.878+0000 7f5123400640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f511c009e20 tx=0x7f511c00e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.878+0000 7f511b400640 1 -- 172.21.15.59:0/3673485043 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c004b20 con 0x7f51241025f0 2024-04-02T17:13:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.878+0000 7f511b400640 1 -- 172.21.15.59:0/3673485043 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511c004cc0 con 0x7f51241025f0 2024-04-02T17:13:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.878+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241a1e70 con 0x7f51241025f0 2024-04-02T17:13:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.878+0000 7f511b400640 1 -- 172.21.15.59:0/3673485043 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c005980 con 0x7f51241025f0 2024-04-02T17:13:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.879+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a2330 con 0x7f51241025f0 2024-04-02T17:13:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.883+0000 7f511b400640 1 -- 172.21.15.59:0/3673485043 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f511c01b020 con 0x7f51241025f0 2024-04-02T17:13:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.883+0000 7f512c7af740 1 Processor -- start 2024-04-02T17:13:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 -- start start 2024-04-02T17:13:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x5556197862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x55561978b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x55561978dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5556196ebd70 con 0x55561978ba30 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5556196942a0 con 0x5556197868c0 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.884+0000 7f512c7af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5556196e7d20 con 0x555619785ed0 2024-04-02T17:13:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x5556197862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x5556197862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39438/0 (socket says 172.21.15.59:39438) 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 -- 172.21.15.59:0/1083556687 learned_addr learned my addr 172.21.15.59:0/1083556687 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122a00640 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x55561978b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x55561978dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 -- 172.21.15.59:0/1083556687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 msgr2=0x5556197862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x5556197862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 -- 172.21.15.59:0/1083556687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 msgr2=0x55561978b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x55561978b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.885+0000 7f5122000640 1 -- 172.21.15.59:0/1083556687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556196a8840 con 0x55561978ba30 2024-04-02T17:13:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f511b400640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 0x7f51080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f511b400640 1 -- 172.21.15.59:0/3673485043 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f511c0cf990 con 0x7f51241025f0 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f5122000640 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x55561978dec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5110005c40 tx=0x7f5110000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ddc1ce36471ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f5122a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 0x7f51080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f510fe00640 1 -- 172.21.15.59:0/1083556687 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110004040 con 0x55561978ba30 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f510fe00640 1 -- 172.21.15.59:0/1083556687 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110047070 con 0x55561978ba30 2024-04-02T17:13:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.886+0000 7f510fe00640 1 -- 172.21.15.59:0/1083556687 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110042440 con 0x55561978ba30 2024-04-02T17:13:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f5122a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 0x7f51080a6190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x555619786cc0 tx=0x7f5114005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 -- 172.21.15.59:0/1083556687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 msgr2=0x55561978dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x55561978dec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5110005c40 tx=0x7f5110000bc0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 -- 172.21.15.59:0/1083556687 shutdown_connections 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x55561978dec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x55561978b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 --2- 172.21.15.59:0/1083556687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x5556197862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.887+0000 7f512c7af740 1 -- 172.21.15.59:0/1083556687 >> 172.21.15.59:0/1083556687 conn(0x555619781440 msgr2=0x555619781840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.888+0000 7f512c7af740 1 -- 172.21.15.59:0/1083556687 shutdown_connections 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.888+0000 7f512c7af740 1 -- 172.21.15.59:0/1083556687 wait complete. 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.888+0000 7f512c7af740 1 -- start start 2024-04-02T17:13:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.888+0000 7f512c7af740 1 Processor -- start 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x555619790990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x5556197979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55561978e980 con 0x55561978ba30 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5556196a7660 con 0x5556197868c0 2024-04-02T17:13:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55561978eb00 con 0x555619785ed0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f512c7af740 10 client.0 _ll_register_callbacks cb 0x5556196a7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f5122a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f5122a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39454/0 (socket says 172.21.15.59:39454) 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f5122a00640 1 -- 172.21.15.59:0/1711050281 learned_addr learned my addr 172.21.15.59:0/1711050281 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.889+0000 7f5123400640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x5556197979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122000640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x555619790990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 -- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 msgr2=0x555619790990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x555619790990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 -- 172.21.15.59:0/1711050281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 msgr2=0x5556197979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x5556197979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 -- 172.21.15.59:0/1711050281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556196b6e80 con 0x555619785ed0 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122000640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x555619790990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f5122a00640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5114004600 tx=0x7f5114004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=676c002e295149a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.890+0000 7f510cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511401f630 con 0x555619785ed0 2024-04-02T17:13:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f511401f7d0 con 0x555619785ed0 2024-04-02T17:13:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110047070 con 0x555619785ed0 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f511401f9d0 con 0x555619785ed0 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f512c7af740 10 client.25639 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.891+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555619400210 con 0x555619785ed0 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5110047070 con 0x555619785ed0 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5114031020 con 0x555619785ed0 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 10 client.25639 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 20 client.25639 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55561979c310 0x55561979e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 4 client.25639 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:13:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.892+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55561969bcd0 con 0x55561979c310 2024-04-02T17:13:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.893+0000 7f5122000640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55561979c310 0x55561979e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.893+0000 7f5122000640 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55561979c310 0x55561979e7a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba646e9b45dbfd9 server_cookie=6a9ea2dfff356a5c in_seq=0 out_seq=0 2024-04-02T17:13:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.893+0000 7f510cc00640 10 client.25639 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.894+0000 7f510cc00640 1 -- 172.21.15.59:0/1711050281 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55561969bcd0 con 0x55561979c310 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.894+0000 7f510cc00640 10 client.25639 handle_command_reply: tid=0 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.894+0000 7f512c7af740 1 client.25639 shutdown 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 msgr2=0x555619790450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5114004600 tx=0x7f5114004730 comp rx=0 tx=0).stop 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 shutdown_connections 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55561979c310 0x55561979e7a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55561978ba30 0x5556197979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556197868c0 0x555619790990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 --2- 172.21.15.59:0/1711050281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619790450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f510cc00640 0 client.25639 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 >> 172.21.15.59:0/1711050281 conn(0x555619781440 msgr2=0x555619780640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 shutdown_connections 2024-04-02T17:13:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 1 -- 172.21.15.59:0/1711050281 wait complete. 2024-04-02T17:13:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.895+0000 7f512c7af740 20 client.25639 trim_cache size 0 max 16384 2024-04-02T17:13:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 Processor -- start 2024-04-02T17:13:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 -- start start 2024-04-02T17:13:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 0x555619790060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 0x555619797990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x5556197905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ab300 con 0x555619785ed0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ace40 con 0x55561978ba30 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.912+0000 7f512c7af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ab9d0 con 0x5556197868c0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 0x555619790060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 0x555619797990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 0x555619797990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39460/0 (socket says 172.21.15.59:39460) 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122000640 1 -- 172.21.15.59:0/1192387290 learned_addr learned my addr 172.21.15.59:0/1192387290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5123400640 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x5556197905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 -- 172.21.15.59:0/1192387290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 msgr2=0x555619797990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 0x555619797990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 -- 172.21.15.59:0/1192387290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 msgr2=0x5556197905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x5556197905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5122a00640 1 -- 172.21.15.59:0/1192387290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556194cc620 con 0x555619785ed0 2024-04-02T17:13:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.913+0000 7f5123400640 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x5556197905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.914+0000 7f5122a00640 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 0x555619790060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5114033fd0 tx=0x7f5114009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa519cbefecfc81b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.914+0000 7f510f400640 1 -- 172.21.15.59:0/1192387290 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114029560 con 0x555619785ed0 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.914+0000 7f510f400640 1 -- 172.21.15.59:0/1192387290 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114038020 con 0x555619785ed0 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.914+0000 7f510f400640 1 -- 172.21.15.59:0/1192387290 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114033460 con 0x555619785ed0 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 -- 172.21.15.59:0/1192387290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 msgr2=0x555619790060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 0x555619790060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5114033fd0 tx=0x7f5114009c00 comp rx=0 tx=0).stop 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 -- 172.21.15.59:0/1192387290 shutdown_connections 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x5556197905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556197868c0 0x555619797990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 --2- 172.21.15.59:0/1192387290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555619785ed0 0x555619790060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 -- 172.21.15.59:0/1192387290 >> 172.21.15.59:0/1192387290 conn(0x555619781440 msgr2=0x55561978eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 -- 172.21.15.59:0/1192387290 shutdown_connections 2024-04-02T17:13:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.915+0000 7f512c7af740 1 -- 172.21.15.59:0/1192387290 wait complete. 2024-04-02T17:13:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 -- start start 2024-04-02T17:13:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 Processor -- start 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619794ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x555619795fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ac770 con 0x5556197868c0 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ac0a0 con 0x55561978ba30 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51241a92e0 con 0x555619785ed0 2024-04-02T17:13:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.916+0000 7f512c7af740 10 client.0 _ll_register_callbacks cb 0x7f51140044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54176/0 (socket says 172.21.15.59:54176) 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619794ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122000640 1 -- 172.21.15.59:0/4049422512 learned_addr learned my addr 172.21.15.59:0/4049422512 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5123400640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x555619795fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 -- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 msgr2=0x555619795fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x555619795fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 -- 172.21.15.59:0/4049422512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 msgr2=0x555619795020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.917+0000 7f5122a00640 1 -- 172.21.15.59:0/4049422512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556197965d0 con 0x555619785ed0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f5122000640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f5122a00640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619794ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114033eb0 tx=0x7f5114003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8afd1775b289f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f510cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:13:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114040070 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f51140053d0 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.918+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5114038020 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51140055f0 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 10 client.25645 fetch_fsmap learned FSMap version 28 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556197978e0 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114038020 con 0x555619785ed0 2024-04-02T17:13:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5114043030 con 0x555619785ed0 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 10 client.25645 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 20 client.25645 populate_metadata read hostname 'smithi059' 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555619802c60 0x555619805080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 4 client.25645 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.919+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556196f92b0 con 0x555619802c60 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.920+0000 7f5122000640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555619802c60 0x555619805080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.920+0000 7f5122000640 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555619802c60 0x555619805080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327adde2770fc20d server_cookie=e661b8e899e22b28 in_seq=0 out_seq=0 2024-04-02T17:13:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.920+0000 7f510cc00640 10 client.25645 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f510cc00640 1 -- 172.21.15.59:0/4049422512 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556196f92b0 con 0x555619802c60 2024-04-02T17:13:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f510cc00640 10 client.25645 handle_command_reply: tid=0 2024-04-02T17:13:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 client.25645 shutdown 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 msgr2=0x555619794ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619794ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114033eb0 tx=0x7f5114003f70 comp rx=0 tx=0).stop 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 shutdown_connections 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555619802c60 0x555619805080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5556197868c0 0x555619795020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55561978ba30 0x555619795fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 --2- 172.21.15.59:0/4049422512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555619785ed0 0x555619794ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 >> 172.21.15.59:0/4049422512 conn(0x555619781440 msgr2=0x555619790040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f510cc00640 0 client.25645 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 shutdown_connections 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.925+0000 7f512c7af740 1 -- 172.21.15.59:0/4049422512 wait complete. 2024-04-02T17:13:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.926+0000 7f512c7af740 20 client.25645 trim_cache size 0 max 16384 2024-04-02T17:13:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:13:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:13:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:13:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.926+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 msgr2=0x7f51080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.926+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 0x7f51080a6190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x555619786cc0 tx=0x7f5114005f90 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.926+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 msgr2=0x7f51241a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.926+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f511c009e20 tx=0x7f511c00e490 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 shutdown_connections 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f51080a3cd0 0x7f51080a6190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51241025f0 0x7f51241a1510 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5124101c30 0x7f512419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 --2- 172.21.15.59:0/3673485043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5124100a40 0x7f5124199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 >> 172.21.15.59:0/3673485043 conn(0x7f51240fbfb0 msgr2=0x7f5124104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 shutdown_connections 2024-04-02T17:13:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:13:56.927+0000 7f5129a00640 1 -- 172.21.15.59:0/3673485043 wait complete. 2024-04-02T17:13:57.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:56 smithi059 ceph-mon[31664]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-02T17:13:57.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:56 smithi164 ceph-mon[36499]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-02T17:13:57.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:56 smithi154 ceph-mon[36479]: pgmap v414: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-02T17:13:59.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:13:58 smithi059 ceph-mon[31664]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-02T17:13:59.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:13:58 smithi164 ceph-mon[36499]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-02T17:13:59.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:13:58 smithi154 ceph-mon[36479]: pgmap v415: 97 pgs: 97 active+clean; 669 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-02T17:14:01.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:00 smithi059 ceph-mon[31664]: pgmap v416: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:14:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:00 smithi164 ceph-mon[36499]: pgmap v416: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:14:01.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:00 smithi154 ceph-mon[36479]: pgmap v416: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T17:14:03.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:02 smithi059 ceph-mon[31664]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:03.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:14:03.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:02 smithi164 ceph-mon[36499]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:03.074 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:14:03.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:02 smithi154 ceph-mon[36479]: pgmap v417: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:03.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:14:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:14:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:14:04.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:14:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:14:04.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:14:04.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:14:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:05 smithi059 ceph-mon[31664]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:05 smithi164 ceph-mon[36499]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:05 smithi154 ceph-mon[36479]: pgmap v418: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:07.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:07 smithi059 ceph-mon[31664]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:07 smithi164 ceph-mon[36499]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:07 smithi154 ceph-mon[36479]: pgmap v419: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:08.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:14:08.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:14:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.596+0000 7f23a7e00640 1 -- 172.21.15.59:0/1765845538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a8101a00 msgr2=0x7f23a8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.596+0000 7f23a7e00640 1 --2- 172.21.15.59:0/1765845538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a8101a00 0x7f23a8101e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2398009860 tx=0x7f239802bb70 comp rx=0 tx=0).stop 2024-04-02T17:14:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 -- 172.21.15.59:0/1765845538 shutdown_connections 2024-04-02T17:14:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 --2- 172.21.15.59:0/1765845538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 0x7f23a8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 --2- 172.21.15.59:0/1765845538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a81035b0 0x7f23a8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 --2- 172.21.15.59:0/1765845538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a8101a00 0x7f23a8101e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 -- 172.21.15.59:0/1765845538 >> 172.21.15.59:0/1765845538 conn(0x7f23a80fcf70 msgr2=0x7f23a80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 -- 172.21.15.59:0/1765845538 shutdown_connections 2024-04-02T17:14:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.597+0000 7f23a7e00640 1 -- 172.21.15.59:0/1765845538 wait complete. 2024-04-02T17:14:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 Processor -- start 2024-04-02T17:14:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 -- start start 2024-04-02T17:14:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 0x7f23a8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a81035b0 0x7f23a819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23a810d680 con 0x7f23a8102bf0 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23a810d500 con 0x7f23a8101a00 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.598+0000 7f23a7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23a810d800 con 0x7f23a81035b0 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 0x7f23a8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37298/0 (socket says 172.21.15.59:37298) 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 -- 172.21.15.59:0/2905699908 learned_addr learned my addr 172.21.15.59:0/2905699908 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a7400640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a81035b0 0x7f23a819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 -- 172.21.15.59:0/2905699908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a81035b0 msgr2=0x7f23a819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a81035b0 0x7f23a819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 -- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 msgr2=0x7f23a8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 0x7f23a8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.599+0000 7f23a6a00640 1 -- 172.21.15.59:0/2905699908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a819f900 con 0x7f23a8101a00 2024-04-02T17:14:08.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f239802c0b0 tx=0x7f2398032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f239f400640 1 -- 172.21.15.59:0/2905699908 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398041020 con 0x7f23a8101a00 2024-04-02T17:14:08.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f239f400640 1 -- 172.21.15.59:0/2905699908 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239802c9f0 con 0x7f23a8101a00 2024-04-02T17:14:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f239f400640 1 -- 172.21.15.59:0/2905699908 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398039040 con 0x7f23a8101a00 2024-04-02T17:14:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a8073d30 con 0x7f23a8101a00 2024-04-02T17:14:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.600+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a80741b0 con 0x7f23a8101a00 2024-04-02T17:14:08.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.603+0000 7f239f400640 1 -- 172.21.15.59:0/2905699908 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2398044020 con 0x7f23a8101a00 2024-04-02T17:14:08.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 Processor -- start 2024-04-02T17:14:08.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 -- start start 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b266131770 con 0x55b2661d5190 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b2660d9700 con 0x55b2661dacf0 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.605+0000 7f23b05c2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b26612d720 con 0x55b2661d5b80 2024-04-02T17:14:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f239f400640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 0x7f238c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f239f400640 1 -- 172.21.15.59:0/2905699908 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f23980f1d30 con 0x7f23a8101a00 2024-04-02T17:14:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46138/0 (socket says 172.21.15.59:46138) 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a7400640 1 -- 172.21.15.59:0/865794940 learned_addr learned my addr 172.21.15.59:0/865794940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a7400640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 0x7f238c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.606+0000 7f23a6a00640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 -- 172.21.15.59:0/865794940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 msgr2=0x55b2661da680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661da680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 -- 172.21.15.59:0/865794940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 msgr2=0x55b2661dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 -- 172.21.15.59:0/865794940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b265f0f7d0 con 0x55b2661d5190 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a6000640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a6a00640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.607+0000 7f23a7400640 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f23a0010f40 tx=0x7f23a000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d900f77e5eb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.608+0000 7f23a7400640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 0x7f238c0a60f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23a000ece0 tx=0x7f23a001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.608+0000 7f2393e00640 1 -- 172.21.15.59:0/865794940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0022070 con 0x55b2661d5190 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.608+0000 7f2393e00640 1 -- 172.21.15.59:0/865794940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a00108c0 con 0x55b2661d5190 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.608+0000 7f2393e00640 1 -- 172.21.15.59:0/865794940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0005630 con 0x55b2661d5190 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- 172.21.15.59:0/865794940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 msgr2=0x55b2661d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f23a0010f40 tx=0x7f23a000e7e0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- 172.21.15.59:0/865794940 shutdown_connections 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 --2- 172.21.15.59:0/865794940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661d55b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- 172.21.15.59:0/865794940 >> 172.21.15.59:0/865794940 conn(0x55b2661c7ef0 msgr2=0x55b2661c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- 172.21.15.59:0/865794940 shutdown_connections 2024-04-02T17:14:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- 172.21.15.59:0/865794940 wait complete. 2024-04-02T17:14:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.609+0000 7f23b05c2740 1 -- start start 2024-04-02T17:14:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 Processor -- start 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b2661ddbe0 con 0x55b2661d5190 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b2660ecac0 con 0x55b2661dacf0 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.610+0000 7f23b05c2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b2661ddd60 con 0x55b2661d5b80 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46140/0 (socket says 172.21.15.59:46140) 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 -- 172.21.15.59:0/176778314 learned_addr learned my addr 172.21.15.59:0/176778314 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23b05c2740 10 client.0 _ll_register_callbacks cb 0x55b2660ef180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6000640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 -- 172.21.15.59:0/176778314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 msgr2=0x55b2661dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661dfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 -- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 msgr2=0x55b2661e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a7400640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a6a00640 1 -- 172.21.15.59:0/176778314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b265e453f0 con 0x55b2661d5190 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.611+0000 7f23a7400640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f23a6a00640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2398009830 tx=0x7f23980bda90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ce30920e76117b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f2390c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980bdf50 con 0x55b2661d5190 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2398032e30 con 0x55b2661d5190 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a00108c0 con 0x55b2661d5190 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23980c2300 con 0x55b2661d5190 2024-04-02T17:14:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f23b05c2740 10 client.15540 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.612+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b265e050f0 con 0x55b2661d5190 2024-04-02T17:14:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.613+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a00108c0 con 0x55b2661d5190 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f239803c030 con 0x55b2661d5190 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 10 client.15540 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 20 client.15540 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b2661ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 4 client.15540 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2660d2630 con 0x55b2661eb0e0 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.614+0000 7f23a6000640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b2661ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.615+0000 7f23a6000640 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b2661ed530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80857fed773649d server_cookie=914b8b3da81918db in_seq=0 out_seq=0 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.615+0000 7f2390c00640 10 client.15540 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f2390c00640 1 -- 172.21.15.59:0/176778314 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2660d2630 con 0x55b2661eb0e0 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f2390c00640 10 client.15540 handle_command_reply: tid=0 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 client.15540 shutdown 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 msgr2=0x55b2661df670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2398009830 tx=0x7f23980bda90 comp rx=0 tx=0).stop 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 shutdown_connections 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b2661ed530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661dacf0 0x55b2661e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f2390c00640 0 client.15540 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 --2- 172.21.15.59:0/176778314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661df670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 >> 172.21.15.59:0/176778314 conn(0x55b2661c7ef0 msgr2=0x55b2661c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 shutdown_connections 2024-04-02T17:14:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.616+0000 7f23b05c2740 1 -- 172.21.15.59:0/176778314 wait complete. 2024-04-02T17:14:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.617+0000 7f23b05c2740 20 client.15540 trim_cache size 0 max 16384 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 Processor -- start 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 -- start start 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5b80 0x55b2661e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661dacf0 0x55b2661de100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a6d30 con 0x55b2661d5190 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a8870 con 0x55b2661d5b80 2024-04-02T17:14:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.651+0000 7f23b05c2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a7400 con 0x55b2661dacf0 2024-04-02T17:14:08.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46148/0 (socket says 172.21.15.59:46148) 2024-04-02T17:14:08.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 -- 172.21.15.59:0/2472740301 learned_addr learned my addr 172.21.15.59:0/2472740301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a7400640 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661dacf0 0x55b2661de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6000640 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5b80 0x55b2661e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 -- 172.21.15.59:0/2472740301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661dacf0 msgr2=0x55b2661de100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661dacf0 0x55b2661de100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 -- 172.21.15.59:0/2472740301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5b80 msgr2=0x55b2661e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5b80 0x55b2661e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.652+0000 7f23a6a00640 1 -- 172.21.15.59:0/2472740301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2661c8c60 con 0x55b2661d5190 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.653+0000 7f23a6a00640 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23980bdee0 tx=0x7f2398004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dcfc570f03df84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.653+0000 7f2393400640 1 -- 172.21.15.59:0/2472740301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980c6bc0 con 0x55b2661d5190 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.653+0000 7f2393400640 1 -- 172.21.15.59:0/2472740301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2398032e30 con 0x55b2661d5190 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.653+0000 7f2393400640 1 -- 172.21.15.59:0/2472740301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980bd600 con 0x55b2661d5190 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- 172.21.15.59:0/2472740301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 msgr2=0x55b2661c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23980bdee0 tx=0x7f2398004550 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- 172.21.15.59:0/2472740301 shutdown_connections 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661dacf0 0x55b2661de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5b80 0x55b2661e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 --2- 172.21.15.59:0/2472740301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661d5190 0x55b2661c7190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- 172.21.15.59:0/2472740301 >> 172.21.15.59:0/2472740301 conn(0x55b2661c7ef0 msgr2=0x55b2661c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- 172.21.15.59:0/2472740301 shutdown_connections 2024-04-02T17:14:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- 172.21.15.59:0/2472740301 wait complete. 2024-04-02T17:14:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 -- start start 2024-04-02T17:14:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.654+0000 7f23b05c2740 1 Processor -- start 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 0x55b2661e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 0x55b2661e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a81a0 con 0x55b2661dacf0 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a7ad0 con 0x55b2661d5190 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81a4d10 con 0x55b2661d5b80 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23b05c2740 10 client.0 _ll_register_callbacks cb 0x7f2398005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 0x55b2661e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23a6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 0x55b2661e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 0x55b2661e5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46156/0 (socket says 172.21.15.59:46156) 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23a7400640 1 -- 172.21.15.59:0/1234658098 learned_addr learned my addr 172.21.15.59:0/1234658098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.655+0000 7f23a6000640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 -- 172.21.15.59:0/1234658098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 msgr2=0x55b2661e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 -- 172.21.15.59:0/1234658098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 msgr2=0x55b2661e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 0x55b2661e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 -- 172.21.15.59:0/1234658098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2661e6170 con 0x55b2661d5190 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f23a6a00640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 0x55b2661e4630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f239802cbc0 tx=0x7f2398032900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1559cc401f7a220d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.656+0000 7f2390c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980a7060 con 0x55b2661d5190 2024-04-02T17:14:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f23980a7200 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2398032e30 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23980bd3a0 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f23b05c2740 10 client.25637 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2661e6170 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.657+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2398032e30 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f23980b8b50 con 0x55b2661d5190 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 10 client.25637 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 20 client.25637 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b266253ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 4 client.25637 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b26613ecb0 con 0x55b2661eb0e0 2024-04-02T17:14:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.658+0000 7f23a6000640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b266253ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23a6000640 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b266253ea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15dbd72b50a7cd26 server_cookie=c3d8dcb915a5d1db in_seq=0 out_seq=0 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f2390c00640 10 client.25637 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f2390c00640 1 -- 172.21.15.59:0/1234658098 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b26613ecb0 con 0x55b2661eb0e0 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f2390c00640 10 client.25637 handle_command_reply: tid=0 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 client.25637 shutdown 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 msgr2=0x55b2661e4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 0x55b2661e4630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f239802cbc0 tx=0x7f2398032900 comp rx=0 tx=0).stop 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 shutdown_connections 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b2661eb0e0 0x55b266253ea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b2661dacf0 0x55b2661e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b2661d5b80 0x55b2661e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 --2- 172.21.15.59:0/1234658098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b2661d5190 0x55b2661e4630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f2390c00640 0 client.25637 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 >> 172.21.15.59:0/1234658098 conn(0x55b2661c7ef0 msgr2=0x55b2661c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 shutdown_connections 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.659+0000 7f23b05c2740 1 -- 172.21.15.59:0/1234658098 wait complete. 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23b05c2740 20 client.25637 trim_cache size 0 max 16384 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-02T17:14:08.663 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 msgr2=0x7f238c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 0x7f238c0a60f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23a000ece0 tx=0x7f23a001e000 comp rx=0 tx=0).stop 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 msgr2=0x7f23a8197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f239802c0b0 tx=0x7f2398032430 comp rx=0 tx=0).stop 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 shutdown_connections 2024-04-02T17:14:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f238c0a3c30 0x7f238c0a60f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a8102bf0 0x7f23a8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23a81035b0 0x7f23a819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.660+0000 7f23a7e00640 1 --2- 172.21.15.59:0/2905699908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23a8101a00 0x7f23a8197cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.661+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 >> 172.21.15.59:0/2905699908 conn(0x7f23a80fcf70 msgr2=0x7f23a8105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.661+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 shutdown_connections 2024-04-02T17:14:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.661+0000 7f23a7e00640 1 -- 172.21.15.59:0/2905699908 wait complete. 2024-04-02T17:14:08.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T17:14:08.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 -- 172.21.15.59:0/362323003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc4103d40 msgr2=0x7f8bc4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/362323003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc4103d40 0x7f8bc4104160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009900 tx=0x7f8bb402bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 -- 172.21.15.59:0/362323003 shutdown_connections 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/362323003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc41058f0 0x7f8bc410c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/362323003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 0x7f8bc41053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/362323003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc4103d40 0x7f8bc4104160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.866+0000 7f8bc8c00640 1 -- 172.21.15.59:0/362323003 >> 172.21.15.59:0/362323003 conn(0x7f8bc40ff2f0 msgr2=0x7f8bc4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.867+0000 7f8bc8c00640 1 -- 172.21.15.59:0/362323003 shutdown_connections 2024-04-02T17:14:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.867+0000 7f8bc8c00640 1 -- 172.21.15.59:0/362323003 wait complete. 2024-04-02T17:14:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.868+0000 7f8bc8c00640 1 Processor -- start 2024-04-02T17:14:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.868+0000 7f8bc8c00640 1 -- start start 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.868+0000 7f8bc8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc4103d40 0x7f8bc419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc8c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 0x7f8bc419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410f9e0 con 0x7f8bc4103d40 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc8c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410f860 con 0x7f8bc4104f30 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc8c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc410fb60 con 0x7f8bc41058f0 2024-04-02T17:14:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc4103d40 0x7f8bc419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56334/0 (socket says 172.21.15.59:56334) 2024-04-02T17:14:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 -- 172.21.15.59:0/3430858111 learned_addr learned my addr 172.21.15.59:0/3430858111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc1400640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 0x7f8bc419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 -- 172.21.15.59:0/3430858111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 msgr2=0x7f8bc419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 0x7f8bc419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 -- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc4103d40 msgr2=0x7f8bc419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.869+0000 7f8bc2800640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc4103d40 0x7f8bc419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bc2800640 1 -- 172.21.15.59:0/3430858111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc41a1c70 con 0x7f8bc41058f0 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bc2800640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00eaa0 tx=0x7f8bbc00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00cdb0 con 0x7f8bc41058f0 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc005020 con 0x7f8bc41058f0 2024-04-02T17:14:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc41a1f60 con 0x7f8bc41058f0 2024-04-02T17:14:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0116b0 con 0x7f8bc41058f0 2024-04-02T17:14:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.870+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41a2420 con 0x7f8bc41058f0 2024-04-02T17:14:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.872+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc410b060 con 0x7f8bc41058f0 2024-04-02T17:14:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.874+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bbc005560 con 0x7f8bc41058f0 2024-04-02T17:14:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.876+0000 7f8bbaa00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 0x7f8ba80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.876+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8bbc0d0960 con 0x7f8bc41058f0 2024-04-02T17:14:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.876+0000 7f8bc1e00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 0x7f8ba80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.877+0000 7f8bc1e00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 0x7f8ba80a60f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8bb402c170 tx=0x7f8bb402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:08.878+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8bbc016030 con 0x7f8bc41058f0 2024-04-02T17:14:09.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.119+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bc402d5c0 con 0x7f8bc41058f0 2024-04-02T17:14:09.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.120+0000 7f8bbaa00640 1 -- 172.21.15.59:0/3430858111 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f8bbc09ccb0 con 0x7f8bc41058f0 2024-04-02T17:14:09.120 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:14:09.120 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:12:54.846637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24529,24530,24547,24560,24584]},"id":1}]} 2024-04-02T17:14:09.121 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 msgr2=0x7f8ba80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 0x7f8ba80a60f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8bb402c170 tx=0x7f8bb402df90 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 msgr2=0x7f8bc41a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00eaa0 tx=0x7f8bbc00ef60 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 shutdown_connections 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8bc41058f0 0x7f8bc41a1600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bc4104f30 0x7f8bc419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ba80a3c30 0x7f8ba80a60f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 --2- 172.21.15.59:0/3430858111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bc4103d40 0x7f8bc419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.122+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 >> 172.21.15.59:0/3430858111 conn(0x7f8bc40ff2f0 msgr2=0x7f8bc4108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.123+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 shutdown_connections 2024-04-02T17:14:09.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.123+0000 7f8bc8c00640 1 -- 172.21.15.59:0/3430858111 wait complete. 2024-04-02T17:14:09.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 -- 172.21.15.59:0/3587586625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff450101a00 msgr2=0x7ff450101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 --2- 172.21.15.59:0/3587586625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff450101a00 0x7ff450101e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff44402bc40 comp rx=0 tx=0).stop 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 -- 172.21.15.59:0/3587586625 shutdown_connections 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 --2- 172.21.15.59:0/3587586625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4501035b0 0x7ff450109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 --2- 172.21.15.59:0/3587586625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 0x7ff450103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 --2- 172.21.15.59:0/3587586625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff450101a00 0x7ff450101e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 -- 172.21.15.59:0/3587586625 >> 172.21.15.59:0/3587586625 conn(0x7ff4500fcf70 msgr2=0x7ff4500ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 -- 172.21.15.59:0/3587586625 shutdown_connections 2024-04-02T17:14:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.343+0000 7ff456a00640 1 -- 172.21.15.59:0/3587586625 wait complete. 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.344+0000 7ff456a00640 1 Processor -- start 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.344+0000 7ff456a00640 1 -- start start 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 0x7ff450199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 0x7ff45019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 0x7ff4501a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d6a0 con 0x7ff450101a00 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d520 con 0x7ff4501035b0 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d820 con 0x7ff450102bf0 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff455600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 0x7ff450199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff455600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 0x7ff450199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46194/0 (socket says 172.21.15.59:46194) 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 0x7ff4501a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff455600640 1 -- 172.21.15.59:0/851433445 learned_addr learned my addr 172.21.15.59:0/851433445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff454c00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 0x7ff45019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.345+0000 7ff456000640 1 -- 172.21.15.59:0/851433445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 msgr2=0x7ff45019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456000640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 0x7ff45019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456000640 1 -- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 msgr2=0x7ff450199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456000640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 0x7ff450199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456000640 1 -- 172.21.15.59:0/851433445 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4501a1b50 con 0x7ff4501035b0 2024-04-02T17:14:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456000640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 0x7ff4501a14e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff448009fd0 tx=0x7ff44800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff44e000640 1 -- 172.21.15.59:0/851433445 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44800ce20 con 0x7ff4501035b0 2024-04-02T17:14:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff44e000640 1 -- 172.21.15.59:0/851433445 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff448005020 con 0x7ff4501035b0 2024-04-02T17:14:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4501a1e40 con 0x7ff4501035b0 2024-04-02T17:14:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff44e000640 1 -- 172.21.15.59:0/851433445 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448010720 con 0x7ff4501035b0 2024-04-02T17:14:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.346+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4501a2300 con 0x7ff4501035b0 2024-04-02T17:14:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.350+0000 7ff44e000640 1 -- 172.21.15.59:0/851433445 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff448005540 con 0x7ff4501035b0 2024-04-02T17:14:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.351+0000 7ff45ad66740 1 Processor -- start 2024-04-02T17:14:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.351+0000 7ff44e000640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 0x7ff4380a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.351+0000 7ff45ad66740 1 -- start start 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.351+0000 7ff44e000640 1 -- 172.21.15.59:0/851433445 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff4480d0bd0 con 0x7ff4501035b0 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.351+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c5020 0x564ebf7c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 0x564ebf7cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff45ad66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564ebf6fd400 con 0x564ebf7c4630 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff45ad66740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564ebf6e1f10 con 0x564ebf7c5020 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff45ad66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ebf7216f0 con 0x564ebf7ca190 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff454c00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 0x7ff4380a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46196/0 (socket says 172.21.15.59:46196) 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 -- 172.21.15.59:0/3256878477 learned_addr learned my addr 172.21.15.59:0/3256878477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff456000640 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 0x564ebf7cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 -- 172.21.15.59:0/3256878477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 msgr2=0x564ebf7cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 0x564ebf7cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 -- 172.21.15.59:0/3256878477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c5020 msgr2=0x564ebf7c9b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c5020 0x564ebf7c9b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff455600640 1 -- 172.21.15.59:0/3256878477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebf4353f0 con 0x564ebf7c4630 2024-04-02T17:14:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.352+0000 7ff456000640 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 0x564ebf7cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.353+0000 7ff455600640 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff44402c8f0 tx=0x7ff444009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a10c0bb5b805ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.353+0000 7ff442600640 1 -- 172.21.15.59:0/3256878477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444042070 con 0x564ebf7c4630 2024-04-02T17:14:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.353+0000 7ff442600640 1 -- 172.21.15.59:0/3256878477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44402cb90 con 0x564ebf7c4630 2024-04-02T17:14:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.353+0000 7ff442600640 1 -- 172.21.15.59:0/3256878477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004920 con 0x564ebf7c4630 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 -- 172.21.15.59:0/3256878477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 msgr2=0x564ebf7c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff454c00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 0x7ff4380a6170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff45019b440 tx=0x7ff43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff44402c8f0 tx=0x7ff444009560 comp rx=0 tx=0).stop 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 -- 172.21.15.59:0/3256878477 shutdown_connections 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7ca190 0x564ebf7cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c5020 0x564ebf7c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3256878477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7c4630 0x564ebf7c4a50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.354+0000 7ff45ad66740 1 -- 172.21.15.59:0/3256878477 >> 172.21.15.59:0/3256878477 conn(0x564ebf783a50 msgr2=0x564ebf7b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.355+0000 7ff45ad66740 1 -- 172.21.15.59:0/3256878477 shutdown_connections 2024-04-02T17:14:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.355+0000 7ff45ad66740 1 -- 172.21.15.59:0/3256878477 wait complete. 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.355+0000 7ff45ad66740 1 -- start start 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.355+0000 7ff45ad66740 1 Processor -- start 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564ebf71d6a0 con 0x564ebf7d3f00 2024-04-02T17:14:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564ebf6c9980 con 0x564ebf7ca190 2024-04-02T17:14:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ebf6dccb0 con 0x564ebf7c4630 2024-04-02T17:14:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff45ad66740 10 client.0 _ll_register_callbacks cb 0x564ebf6dc6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff456000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff456000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46206/0 (socket says 172.21.15.59:46206) 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff456000640 1 -- 172.21.15.59:0/1352000480 learned_addr learned my addr 172.21.15.59:0/1352000480 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff454c00640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.356+0000 7ff455600640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 -- 172.21.15.59:0/1352000480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 msgr2=0x564ebf7ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 -- 172.21.15.59:0/1352000480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 msgr2=0x564ebf7d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 -- 172.21.15.59:0/1352000480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebf4ff7d0 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff456000640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff455600640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7ce990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff44402c250 tx=0x7ff4440045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82c31d23a489cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff436a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.357+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444042050 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44402cb90 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff444003a00 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff444003c00 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff45ad66740 10 client.25678 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ebf7d9660 con 0x564ebf7c4630 2024-04-02T17:14:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.358+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44402cb90 con 0x564ebf7c4630 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff444057020 con 0x564ebf7c4630 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 10 client.25678 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 10 client.25678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 20 client.25678 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf7da9b0 0x564ebf7dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 4 client.25678 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ebf6e1040 con 0x564ebf7da9b0 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.359+0000 7ff454c00640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf7da9b0 0x564ebf7dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.360+0000 7ff454c00640 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf7da9b0 0x564ebf7dcda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6236337ba004c12f server_cookie=8be1daa7022f044d in_seq=0 out_seq=0 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.360+0000 7ff436a00640 10 client.25678 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.360+0000 7ff436a00640 1 -- 172.21.15.59:0/1352000480 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ebf6e1040 con 0x564ebf7da9b0 2024-04-02T17:14:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.360+0000 7ff436a00640 10 client.25678 handle_command_reply: tid=0 2024-04-02T17:14:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 client.25678 shutdown 2024-04-02T17:14:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 msgr2=0x564ebf7ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7ce990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff44402c250 tx=0x7ff4440045e0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 shutdown_connections 2024-04-02T17:14:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf7da9b0 0x564ebf7dcda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 --2- 172.21.15.59:0/1352000480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7ce990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff436a00640 0 client.25678 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 >> 172.21.15.59:0/1352000480 conn(0x564ebf783a50 msgr2=0x564ebf7b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 shutdown_connections 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 1 -- 172.21.15.59:0/1352000480 wait complete. 2024-04-02T17:14:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.361+0000 7ff45ad66740 20 client.25678 trim_cache size 0 max 16384 2024-04-02T17:14:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 Processor -- start 2024-04-02T17:14:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 -- start start 2024-04-02T17:14:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a6d10 con 0x564ebf7d3f00 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a8850 con 0x564ebf7ca190 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff45ad66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a73e0 con 0x564ebf7c4630 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff455600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.376+0000 7ff454c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37414/0 (socket says 172.21.15.59:37414) 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff456000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 -- 172.21.15.59:0/2166815996 learned_addr learned my addr 172.21.15.59:0/2166815996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff455600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56376/0 (socket says 172.21.15.59:56376) 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 -- 172.21.15.59:0/2166815996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 msgr2=0x564ebf7b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 -- 172.21.15.59:0/2166815996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 msgr2=0x564ebf7ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 -- 172.21.15.59:0/2166815996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ebf7b84e0 con 0x564ebf7ca190 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff456000640 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff455600640 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.377+0000 7ff454c00640 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x564ebf7c5420 tx=0x7ff43c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff8ac383e068a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.378+0000 7ff441c00640 1 -- 172.21.15.59:0/2166815996 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c02f070 con 0x564ebf7ca190 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.378+0000 7ff441c00640 1 -- 172.21.15.59:0/2166815996 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff43c029030 con 0x564ebf7ca190 2024-04-02T17:14:09.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.378+0000 7ff441c00640 1 -- 172.21.15.59:0/2166815996 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c01d680 con 0x564ebf7ca190 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- 172.21.15.59:0/2166815996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 msgr2=0x564ebf7cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x564ebf7c5420 tx=0x7ff43c014000 comp rx=0 tx=0).stop 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- 172.21.15.59:0/2166815996 shutdown_connections 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7d3f00 0x564ebf7ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7ca190 0x564ebf7cdfb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 --2- 172.21.15.59:0/2166815996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7c4630 0x564ebf7b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- 172.21.15.59:0/2166815996 >> 172.21.15.59:0/2166815996 conn(0x564ebf783a50 msgr2=0x564ebf7b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- 172.21.15.59:0/2166815996 shutdown_connections 2024-04-02T17:14:09.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- 172.21.15.59:0/2166815996 wait complete. 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.379+0000 7ff45ad66740 1 -- start start 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 Processor -- start 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 0x564ebf7ebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7d3f00 0x564ebf7ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a8180 con 0x564ebf7ca190 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a7ab0 con 0x564ebf7c4630 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501a4d10 con 0x564ebf7d3f00 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.380+0000 7ff45ad66740 10 client.0 _ll_register_callbacks cb 0x7ff444009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff454c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff454c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46222/0 (socket says 172.21.15.59:46222) 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff454c00640 1 -- 172.21.15.59:0/3053160672 learned_addr learned my addr 172.21.15.59:0/3053160672 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 0x564ebf7ebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff456000640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7d3f00 0x564ebf7ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 -- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7d3f00 msgr2=0x564ebf7ed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7d3f00 0x564ebf7ed2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 -- 172.21.15.59:0/3053160672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 msgr2=0x564ebf7ec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff455600640 1 -- 172.21.15.59:0/3053160672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ebf7ed8b0 con 0x564ebf7c4630 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.381+0000 7ff454c00640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff455600640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 0x564ebf7ebdc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff44402cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fce703a752491c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff436a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44405e060 con 0x564ebf7c4630 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff43c029030 con 0x564ebf7c4630 2024-04-02T17:14:09.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff444005240 con 0x564ebf7c4630 2024-04-02T17:14:09.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff444042040 con 0x564ebf7c4630 2024-04-02T17:14:09.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff45ad66740 10 client.25658 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.382+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ebf840e40 con 0x564ebf7c4630 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff43c029030 con 0x564ebf7c4630 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff444062030 con 0x564ebf7c4630 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 10 client.25658 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 10 client.25658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 20 client.25658 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf841730 0x564ebf843b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 4 client.25658 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ebf72ec30 con 0x564ebf841730 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.383+0000 7ff454c00640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf841730 0x564ebf843b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.384+0000 7ff454c00640 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf841730 0x564ebf843b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c042e1e1ccefca6 server_cookie=baf0864c9e430634 in_seq=0 out_seq=0 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.384+0000 7ff436a00640 10 client.25658 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.384+0000 7ff436a00640 1 -- 172.21.15.59:0/3053160672 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ebf72ec30 con 0x564ebf841730 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.384+0000 7ff436a00640 10 client.25658 handle_command_reply: tid=0 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.384+0000 7ff45ad66740 1 client.25658 shutdown 2024-04-02T17:14:09.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 msgr2=0x564ebf7ebdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 0x564ebf7ebdc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff44402cc80 comp rx=0 tx=0).stop 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 shutdown_connections 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564ebf841730 0x564ebf843b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ebf7ca190 0x564ebf7ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ebf7d3f00 0x564ebf7ed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 --2- 172.21.15.59:0/3053160672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ebf7c4630 0x564ebf7ebdc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 >> 172.21.15.59:0/3053160672 conn(0x564ebf783a50 msgr2=0x564ebf7b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff436a00640 0 client.25658 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 shutdown_connections 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 1 -- 172.21.15.59:0/3053160672 wait complete. 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.385+0000 7ff45ad66740 20 client.25658 trim_cache size 0 max 16384 2024-04-02T17:14:09.388 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 msgr2=0x7ff4380a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 0x7ff4380a6170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff45019b440 tx=0x7ff43c005f90 comp rx=0 tx=0).stop 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 msgr2=0x7ff4501a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 0x7ff4501a14e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff448009fd0 tx=0x7ff44800ef10 comp rx=0 tx=0).stop 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 shutdown_connections 2024-04-02T17:14:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4501035b0 0x7ff4501a14e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff450102bf0 0x7ff45019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4380a3cb0 0x7ff4380a6170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 --2- 172.21.15.59:0/851433445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff450101a00 0x7ff450199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 >> 172.21.15.59:0/851433445 conn(0x7ff4500fcf70 msgr2=0x7ff450073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 shutdown_connections 2024-04-02T17:14:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.386+0000 7ff456a00640 1 -- 172.21.15.59:0/851433445 wait complete. 2024-04-02T17:14:09.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:09 smithi059 ceph-mon[31664]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:09.401 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:09 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3430858111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:09.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T17:14:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:09 smithi164 ceph-mon[36499]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:09 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3430858111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 -- 172.21.15.59:0/2407897263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 msgr2=0x7ff888074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 --2- 172.21.15.59:0/2407897263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888074940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff874009900 tx=0x7ff87402bcc0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 -- 172.21.15.59:0/2407897263 shutdown_connections 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 --2- 172.21.15.59:0/2407897263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8881098b0 0x7ff88810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 --2- 172.21.15.59:0/2407897263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888074940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 --2- 172.21.15.59:0/2407897263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff888075ac0 0x7ff888073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.587+0000 7ff88dc00640 1 -- 172.21.15.59:0/2407897263 >> 172.21.15.59:0/2407897263 conn(0x7ff8880fcf70 msgr2=0x7ff8880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.588+0000 7ff88dc00640 1 -- 172.21.15.59:0/2407897263 shutdown_connections 2024-04-02T17:14:09.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.588+0000 7ff88dc00640 1 -- 172.21.15.59:0/2407897263 wait complete. 2024-04-02T17:14:09.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.589+0000 7ff88dc00640 1 Processor -- start 2024-04-02T17:14:09.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.589+0000 7ff88dc00640 1 -- start start 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888075ac0 0x7ff88819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8881098b0 0x7ff8881a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810f880 con 0x7ff888075ac0 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810f700 con 0x7ff8881098b0 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff88dc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810fa00 con 0x7ff8880744e0 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56402/0 (socket says 172.21.15.59:56402) 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 -- 172.21.15.59:0/3673148412 learned_addr learned my addr 172.21.15.59:0/3673148412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887400640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888075ac0 0x7ff88819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 -- 172.21.15.59:0/3673148412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8881098b0 msgr2=0x7ff8881a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8881098b0 0x7ff8881a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 -- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888075ac0 msgr2=0x7ff88819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888075ac0 0x7ff88819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.590+0000 7ff887e00640 1 -- 172.21.15.59:0/3673148412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881a1af0 con 0x7ff8880744e0 2024-04-02T17:14:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff887e00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9f0 tx=0x7ff87800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff884c00640 1 -- 172.21.15.59:0/3673148412 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87800cd80 con 0x7ff8880744e0 2024-04-02T17:14:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff884c00640 1 -- 172.21.15.59:0/3673148412 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff878005500 con 0x7ff8880744e0 2024-04-02T17:14:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8881a1de0 con 0x7ff8880744e0 2024-04-02T17:14:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff884c00640 1 -- 172.21.15.59:0/3673148412 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878010690 con 0x7ff8880744e0 2024-04-02T17:14:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.591+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a2240 con 0x7ff8880744e0 2024-04-02T17:14:09.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.595+0000 7ff884c00640 1 -- 172.21.15.59:0/3673148412 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8780046c0 con 0x7ff8880744e0 2024-04-02T17:14:09.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.596+0000 7ff884c00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 0x7ff86c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.597+0000 7ff884c00640 1 -- 172.21.15.59:0/3673148412 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8780d0990 con 0x7ff8880744e0 2024-04-02T17:14:09.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.597+0000 7ff891e27740 1 Processor -- start 2024-04-02T17:14:09.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.597+0000 7ff887400640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 0x7ff86c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.597+0000 7ff891e27740 1 -- start start 2024-04-02T17:14:09.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 0x5647266ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5647265eb400 con 0x5647266aa810 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5647265cff10 con 0x5647266a9e20 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.598+0000 7ff891e27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56472660f6f0 con 0x5647266b8190 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887400640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 0x7ff86c0a60f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37476/0 (socket says 172.21.15.59:37476) 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 -- 172.21.15.59:0/37298771 learned_addr learned my addr 172.21.15.59:0/37298771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887400640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff88d200640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 0x5647266ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 -- 172.21.15.59:0/37298771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 msgr2=0x5647266ba620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 0x5647266ba620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 -- 172.21.15.59:0/37298771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 msgr2=0x5647266b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887e00640 1 -- 172.21.15.59:0/37298771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647263233f0 con 0x5647266a9e20 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff88d200640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 0x5647266ba620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.599+0000 7ff887400640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.600+0000 7ff887e00640 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff87809cb40 tx=0x7ff87809c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a49471418969ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.600+0000 7ff87d600640 1 -- 172.21.15.59:0/37298771 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780b6070 con 0x5647266a9e20 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.600+0000 7ff87d600640 1 -- 172.21.15.59:0/37298771 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.600+0000 7ff87d600640 1 -- 172.21.15.59:0/37298771 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780bd070 con 0x5647266a9e20 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 -- 172.21.15.59:0/37298771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 msgr2=0x5647266aa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff87809cb40 tx=0x7ff87809c510 comp rx=0 tx=0).stop 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 -- 172.21.15.59:0/37298771 shutdown_connections 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266b8190 0x5647266ba620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 --2- 172.21.15.59:0/37298771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266a9e20 0x5647266aa240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 -- 172.21.15.59:0/37298771 >> 172.21.15.59:0/37298771 conn(0x564726671a50 msgr2=0x5647266a5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 -- 172.21.15.59:0/37298771 shutdown_connections 2024-04-02T17:14:09.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.601+0000 7ff891e27740 1 -- 172.21.15.59:0/37298771 wait complete. 2024-04-02T17:14:09.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.602+0000 7ff891e27740 1 -- start start 2024-04-02T17:14:09.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.602+0000 7ff891e27740 1 Processor -- start 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56472660b6a0 con 0x5647266aa810 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff887400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff887400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46256/0 (socket says 172.21.15.59:46256) 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff887e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff887400640 1 -- 172.21.15.59:0/2937163035 learned_addr learned my addr 172.21.15.59:0/2937163035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5647265b7980 con 0x5647266b8190 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647265cacb0 con 0x5647266a9e20 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff891e27740 10 client.0 _ll_register_callbacks cb 0x5647265ca6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.603+0000 7ff88d200640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 -- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 msgr2=0x5647266b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 -- 172.21.15.59:0/2937163035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 msgr2=0x5647266b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647263ed7d0 con 0x5647266a9e20 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff88d200640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff887e00640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b11a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff87809c8b0 tx=0x7ff87800cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2628a289d68a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff872000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780b6040 con 0x5647266a9e20 2024-04-02T17:14:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.604+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8780a5390 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.605+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.605+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8780a5590 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.605+0000 7ff891e27740 10 client.25693 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.605+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647266cf870 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.605+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8780a20f0 con 0x5647266a9e20 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 10 client.25693 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 10 client.25693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 20 client.25693 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5647266d0bc0 0x5647266d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 4 client.25693 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647265cf040 con 0x5647266d0bc0 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.606+0000 7ff887400640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5647266d0bc0 0x5647266d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.607+0000 7ff887400640 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5647266d0bc0 0x5647266d2fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d81147f74cf880c server_cookie=60a79f4336f4fdc1 in_seq=0 out_seq=0 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.607+0000 7ff872000640 10 client.25693 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff872000640 1 -- 172.21.15.59:0/2937163035 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647265cf040 con 0x5647266d0bc0 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff872000640 10 client.25693 handle_command_reply: tid=0 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 client.25693 shutdown 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 msgr2=0x5647266b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b11a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff87809c8b0 tx=0x7ff87800cf40 comp rx=0 tx=0).stop 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 shutdown_connections 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x5647266d0bc0 0x5647266d2fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff872000640 0 client.25693 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 --2- 172.21.15.59:0/2937163035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b11a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.608+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 >> 172.21.15.59:0/2937163035 conn(0x564726671a50 msgr2=0x5647266a4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.609+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 shutdown_connections 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.609+0000 7ff891e27740 1 -- 172.21.15.59:0/2937163035 wait complete. 2024-04-02T17:14:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.609+0000 7ff891e27740 20 client.25693 trim_cache size 0 max 16384 2024-04-02T17:14:09.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.637+0000 7ff891e27740 1 Processor -- start 2024-04-02T17:14:09.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.637+0000 7ff891e27740 1 -- start start 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a6d00 con 0x5647266aa810 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a8840 con 0x5647266b8190 2024-04-02T17:14:09.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff891e27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a73d0 con 0x5647266a9e20 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff887400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff88d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff88d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37484/0 (socket says 172.21.15.59:37484) 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff88d200640 1 -- 172.21.15.59:0/2386454351 learned_addr learned my addr 172.21.15.59:0/2386454351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.638+0000 7ff887e00640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 -- 172.21.15.59:0/2386454351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 msgr2=0x5647266a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 -- 172.21.15.59:0/2386454351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 msgr2=0x5647266aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 -- 172.21.15.59:0/2386454351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647266a64e0 con 0x5647266a9e20 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff88d200640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887400640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.639+0000 7ff887e00640 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b1640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff878099530 tx=0x7ff87809b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d121a826afbdf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.640+0000 7ff87cc00640 1 -- 172.21.15.59:0/2386454351 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780b6050 con 0x5647266a9e20 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.640+0000 7ff87cc00640 1 -- 172.21.15.59:0/2386454351 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.640+0000 7ff87cc00640 1 -- 172.21.15.59:0/2386454351 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87809ba70 con 0x5647266a9e20 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 -- 172.21.15.59:0/2386454351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 msgr2=0x5647266b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b1640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff878099530 tx=0x7ff87809b0c0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 -- 172.21.15.59:0/2386454351 shutdown_connections 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266aa810 0x5647266aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266b8190 0x5647266a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 --2- 172.21.15.59:0/2386454351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266b1640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 -- 172.21.15.59:0/2386454351 >> 172.21.15.59:0/2386454351 conn(0x564726671a50 msgr2=0x5647266af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 -- 172.21.15.59:0/2386454351 shutdown_connections 2024-04-02T17:14:09.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.641+0000 7ff891e27740 1 -- 172.21.15.59:0/2386454351 wait complete. 2024-04-02T17:14:09.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.642+0000 7ff891e27740 1 -- start start 2024-04-02T17:14:09.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.642+0000 7ff891e27740 1 Processor -- start 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.642+0000 7ff891e27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.642+0000 7ff891e27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 0x5647266e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff891e27740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 0x5647266e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff891e27740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a8170 con 0x5647266b8190 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff891e27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a7aa0 con 0x5647266aa810 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff891e27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a4ce0 con 0x5647266a9e20 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff891e27740 10 client.0 _ll_register_callbacks cb 0x7ff87809ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56438/0 (socket says 172.21.15.59:56438) 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 -- 172.21.15.59:0/1554413203 learned_addr learned my addr 172.21.15.59:0/1554413203 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff88d200640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 0x5647266e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 -- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 msgr2=0x5647266e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887400640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 0x5647266e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 0x5647266e25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 -- 172.21.15.59:0/1554413203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 msgr2=0x5647266e35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 0x5647266e35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887e00640 1 -- 172.21.15.59:0/1554413203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647266e3bd0 con 0x5647266a9e20 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff887400640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 0x5647266e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.643+0000 7ff88d200640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 0x5647266e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff887e00640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9c0 tx=0x7ff87809bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac666c523f1d73e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff872000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:09.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780a5a70 con 0x5647266a9e20 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8780b1300 con 0x5647266a9e20 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff87808c050 con 0x5647266a9e20 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff891e27740 10 client.25699 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.644+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647266e3bd0 con 0x5647266a9e20 2024-04-02T17:14:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.645+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780a0f60 con 0x5647266a9e20 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.645+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff87809c120 con 0x5647266a9e20 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.645+0000 7ff891e27740 10 client.25699 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff891e27740 10 client.25699 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff891e27740 20 client.25699 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564726737760 0x564726739b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff891e27740 4 client.25699 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56472661cc30 con 0x564726737760 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.646+0000 7ff887400640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564726737760 0x564726739b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff872000640 10 client.25699 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff887400640 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564726737760 0x564726739b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a65d64b9e18cc1f5 server_cookie=cf9d88f9a0a570 in_seq=0 out_seq=0 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff872000640 1 -- 172.21.15.59:0/1554413203 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56472661cc30 con 0x564726737760 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff872000640 10 client.25699 handle_command_reply: tid=0 2024-04-02T17:14:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 client.25699 shutdown 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 msgr2=0x5647266e2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9c0 tx=0x7ff87809bd20 comp rx=0 tx=0).stop 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 shutdown_connections 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564726737760 0x564726739b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff872000640 0 client.25699 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5647266b8190 0x5647266e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5647266aa810 0x5647266e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 --2- 172.21.15.59:0/1554413203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647266a9e20 0x5647266e2090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.647+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 >> 172.21.15.59:0/1554413203 conn(0x564726671a50 msgr2=0x5647266b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.648+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 shutdown_connections 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.648+0000 7ff891e27740 1 -- 172.21.15.59:0/1554413203 wait complete. 2024-04-02T17:14:09.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.649+0000 7ff891e27740 20 client.25699 trim_cache size 0 max 16384 2024-04-02T17:14:09.651 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:09.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.649+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 msgr2=0x7ff86c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.649+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 0x7ff86c0a60f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87402df90 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.649+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 msgr2=0x7ff888199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.649+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff87800e9f0 tx=0x7ff87800edf0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 shutdown_connections 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff86c0a3c30 0x7ff86c0a60f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888075ac0 0x7ff88819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8881098b0 0x7ff8881a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 --2- 172.21.15.59:0/3673148412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8880744e0 0x7ff888199ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 >> 172.21.15.59:0/3673148412 conn(0x7ff8880fcf70 msgr2=0x7ff888105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 shutdown_connections 2024-04-02T17:14:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.650+0000 7ff88dc00640 1 -- 172.21.15.59:0/3673148412 wait complete. 2024-04-02T17:14:09.653 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:09 smithi154 ceph-mon[36479]: pgmap v420: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T17:14:09.653 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:09 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3430858111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:09.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T17:14:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 -- 172.21.15.59:0/2145466886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc101a00 msgr2=0x7f53dc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 --2- 172.21.15.59:0/2145466886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc101a00 0x7f53dc101e20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009900 tx=0x7f53cc02bc40 comp rx=0 tx=0).stop 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 -- 172.21.15.59:0/2145466886 shutdown_connections 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 --2- 172.21.15.59:0/2145466886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 --2- 172.21.15.59:0/2145466886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc102bf0 0x7f53dc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 --2- 172.21.15.59:0/2145466886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc101a00 0x7f53dc101e20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 -- 172.21.15.59:0/2145466886 >> 172.21.15.59:0/2145466886 conn(0x7f53dc0fcf70 msgr2=0x7f53dc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 -- 172.21.15.59:0/2145466886 shutdown_connections 2024-04-02T17:14:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.857+0000 7f53db200640 1 -- 172.21.15.59:0/2145466886 wait complete. 2024-04-02T17:14:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.858+0000 7f53db200640 1 Processor -- start 2024-04-02T17:14:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.858+0000 7f53db200640 1 -- start start 2024-04-02T17:14:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc102bf0 0x7f53dc196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10b4f0 con 0x7f53dc102bf0 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10b370 con 0x7f53dc1035b0 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53db200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10b670 con 0x7f53dc101a00 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56464/0 (socket says 172.21.15.59:56464) 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc102bf0 0x7f53dc196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53d9e00640 1 -- 172.21.15.59:0/785049948 learned_addr learned my addr 172.21.15.59:0/785049948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.859+0000 7f53da800640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 -- 172.21.15.59:0/785049948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 msgr2=0x7f53dc19d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc19d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 -- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc102bf0 msgr2=0x7f53dc196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc102bf0 0x7f53dc196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 -- 172.21.15.59:0/785049948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19d730 con 0x7f53dc101a00 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53da800640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d9e00640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53cc02efd0 tx=0x7f53cc02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.860+0000 7f53d2a00640 1 -- 172.21.15.59:0/785049948 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc041070 con 0x7f53dc101a00 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.861+0000 7f53d2a00640 1 -- 172.21.15.59:0/785049948 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc004450 con 0x7f53dc101a00 2024-04-02T17:14:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.861+0000 7f53d2a00640 1 -- 172.21.15.59:0/785049948 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc003e50 con 0x7f53dc101a00 2024-04-02T17:14:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.861+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19d9c0 con 0x7f53dc101a00 2024-04-02T17:14:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.862+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc19de30 con 0x7f53dc101a00 2024-04-02T17:14:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.865+0000 7f53d2a00640 1 -- 172.21.15.59:0/785049948 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f53cc044070 con 0x7f53dc101a00 2024-04-02T17:14:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.866+0000 7f53e37cd740 1 Processor -- start 2024-04-02T17:14:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.866+0000 7f53e37cd740 1 -- start start 2024-04-02T17:14:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.866+0000 7f53d2a00640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 0x7f53c00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c84240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c84810 0x55b3d3c89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53d2a00640 1 -- 172.21.15.59:0/785049948 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f53cc0f1bb0 con 0x7f53dc101a00 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3bc5400 con 0x55b3d3c89980 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3ba9f10 con 0x55b3d3c83e20 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53e37cd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3be96f0 con 0x55b3d3c84810 2024-04-02T17:14:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53da800640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 0x7f53c00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c84240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c8be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46290/0 (socket says 172.21.15.59:46290) 2024-04-02T17:14:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53da800640 1 -- 172.21.15.59:0/2858937553 learned_addr learned my addr 172.21.15.59:0/2858937553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.867+0000 7f53d9400640 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c84810 0x55b3d3c89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 -- 172.21.15.59:0/2858937553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c84810 msgr2=0x55b3d3c89310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c84810 0x55b3d3c89310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 -- 172.21.15.59:0/2858937553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 msgr2=0x55b3d3c8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c8be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 -- 172.21.15.59:0/2858937553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d38fd3f0 con 0x55b3d3c83e20 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c84240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0ef5f0 tx=0x7f53cc02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400dc9b592357744 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53c7000640 1 -- 172.21.15.59:0/2858937553 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0c2180 con 0x55b3d3c83e20 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53c7000640 1 -- 172.21.15.59:0/2858937553 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc0c6a40 con 0x55b3d3c83e20 2024-04-02T17:14:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.868+0000 7f53c7000640 1 -- 172.21.15.59:0/2858937553 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0c65b0 con 0x55b3d3c83e20 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 -- 172.21.15.59:0/2858937553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 msgr2=0x55b3d3c84240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c84240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0ef5f0 tx=0x7f53cc02c890 comp rx=0 tx=0).stop 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53da800640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 0x7f53c00a6290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53d4012d20 tx=0x7f53d4009ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 -- 172.21.15.59:0/2858937553 shutdown_connections 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c84810 0x55b3d3c89310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2858937553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c84240 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 -- 172.21.15.59:0/2858937553 >> 172.21.15.59:0/2858937553 conn(0x55b3d3c4ba50 msgr2=0x55b3d3c7f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 -- 172.21.15.59:0/2858937553 shutdown_connections 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.869+0000 7f53e37cd740 1 -- 172.21.15.59:0/2858937553 wait complete. 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 -- start start 2024-04-02T17:14:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 Processor -- start 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c89980 0x55b3d3c9dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.870+0000 7f53e37cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3be56a0 con 0x55b3d3c84810 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53e37cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3b91980 con 0x55b3d3c83e20 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53e37cd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b3d3ba4cb0 con 0x55b3d3c89980 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53e37cd740 10 client.0 _ll_register_callbacks cb 0x55b3d3ba46a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37530/0 (socket says 172.21.15.59:37530) 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 -- 172.21.15.59:0/626142621 learned_addr learned my addr 172.21.15.59:0/626142621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53da800640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c89980 0x55b3d3c9dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9400640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 -- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c89980 msgr2=0x55b3d3c9dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c89980 0x55b3d3c9dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 -- 172.21.15.59:0/626142621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 msgr2=0x55b3d3c96b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c96b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.871+0000 7f53d9e00640 1 -- 172.21.15.59:0/626142621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d39c77d0 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53d9e00640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0c6510 tx=0x7f53cc009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2df4c9858c0a031 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53bb400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ba440 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53cc0c6a40 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f53cc0d7170 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53cc0c2180 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53e37cd740 10 client.25682 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3d3ca0f40 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.872+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc0c6a40 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f53cc0de260 con 0x55b3d3c83e20 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 10 client.25682 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 10 client.25682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:14:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 20 client.25682 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3ca2160 0x55b3d3ca45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 4 client.25682 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3d3ba9040 con 0x55b3d3ca2160 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.873+0000 7f53d9400640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3ca2160 0x55b3d3ca45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.874+0000 7f53bb400640 10 client.25682 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.874+0000 7f53d9400640 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3ca2160 0x55b3d3ca45b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebd604e83063e71b server_cookie=1eeeb4e89793e7bc in_seq=0 out_seq=0 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53bb400640 1 -- 172.21.15.59:0/626142621 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3d3ba9040 con 0x55b3d3ca2160 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53bb400640 10 client.25682 handle_command_reply: tid=0 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 client.25682 shutdown 2024-04-02T17:14:09.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 msgr2=0x55b3d3c96650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0c6510 tx=0x7f53cc009610 comp rx=0 tx=0).stop 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 shutdown_connections 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3ca2160 0x55b3d3ca45b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c89980 0x55b3d3c9dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 --2- 172.21.15.59:0/626142621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53bb400640 0 client.25682 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.875+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 >> 172.21.15.59:0/626142621 conn(0x55b3d3c4ba50 msgr2=0x55b3d3c7eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.876+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 shutdown_connections 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.876+0000 7f53e37cd740 1 -- 172.21.15.59:0/626142621 wait complete. 2024-04-02T17:14:09.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.876+0000 7f53e37cd740 20 client.25682 trim_cache size 0 max 16384 2024-04-02T17:14:09.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.890+0000 7f53e37cd740 1 Processor -- start 2024-04-02T17:14:09.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.890+0000 7f53e37cd740 1 -- start start 2024-04-02T17:14:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c9db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 0x55b3d3c7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a6da0 con 0x55b3d3c84810 2024-04-02T17:14:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a88e0 con 0x55b3d3c89980 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53e37cd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a7470 con 0x55b3d3c83e20 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56502/0 (socket says 172.21.15.59:56502) 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 -- 172.21.15.59:0/2796610127 learned_addr learned my addr 172.21.15.59:0/2796610127 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9400640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c9db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53da800640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 0x55b3d3c7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 -- 172.21.15.59:0/2796610127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 msgr2=0x55b3d3c7e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 0x55b3d3c7e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 -- 172.21.15.59:0/2796610127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 msgr2=0x55b3d3c9db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c9db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9e00640 1 -- 172.21.15.59:0/2796610127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d3c7af30 con 0x55b3d3c83e20 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53da800640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 0x55b3d3c7e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.891+0000 7f53d9400640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c9db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.892+0000 7f53d9e00640 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006010 tx=0x7f53cc0bc2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2502437bdb305d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.892+0000 7f53c6600640 1 -- 172.21.15.59:0/2796610127 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0c6a40 con 0x55b3d3c83e20 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.892+0000 7f53c6600640 1 -- 172.21.15.59:0/2796610127 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc0ba440 con 0x55b3d3c83e20 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.892+0000 7f53c6600640 1 -- 172.21.15.59:0/2796610127 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0bd480 con 0x55b3d3c83e20 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- 172.21.15.59:0/2796610127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 msgr2=0x55b3d3c96af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53cc006010 tx=0x7f53cc0bc2e0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- 172.21.15.59:0/2796610127 shutdown_connections 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c84810 0x55b3d3c9db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c89980 0x55b3d3c7e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 --2- 172.21.15.59:0/2796610127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c96af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- 172.21.15.59:0/2796610127 >> 172.21.15.59:0/2796610127 conn(0x55b3d3c4ba50 msgr2=0x55b3d3c7e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- 172.21.15.59:0/2796610127 shutdown_connections 2024-04-02T17:14:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- 172.21.15.59:0/2796610127 wait complete. 2024-04-02T17:14:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.893+0000 7f53e37cd740 1 -- start start 2024-04-02T17:14:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 Processor -- start 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 0x55b3d3c9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c9c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a8210 con 0x55b3d3c89980 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a7b40 con 0x55b3d3c84810 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1a4d80 con 0x55b3d3c83e20 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53e37cd740 10 client.0 _ll_register_callbacks cb 0x7f53cc0c6900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c9c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.894+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c9aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56516/0 (socket says 172.21.15.59:56516) 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9e00640 1 -- 172.21.15.59:0/1619351380 learned_addr learned my addr 172.21.15.59:0/1619351380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 0x55b3d3c9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 -- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 msgr2=0x55b3d3c9aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c9aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 -- 172.21.15.59:0/1619351380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 msgr2=0x55b3d3c9c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c9c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 -- 172.21.15.59:0/1619351380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d3c9c8e0 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53da800640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c9c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53d9400640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 0x55b3d3c9b3f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b3d3c97b30 tx=0x7f53c80039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a8ac125ad2b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.895+0000 7f53bb400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c802a060 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f53c8025630 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53cc0ba440 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53c8025830 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53e37cd740 10 client.25685 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3d3c9dbc0 con 0x55b3d3c84810 2024-04-02T17:14:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.896+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc0ba440 con 0x55b3d3c84810 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f53c802e030 con 0x55b3d3c84810 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 10 client.25685 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 10 client.25685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 20 client.25685 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3d08c60 0x55b3d3d0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 4 client.25685 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3d3bf6c30 con 0x55b3d3d08c60 2024-04-02T17:14:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.897+0000 7f53d9e00640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3d08c60 0x55b3d3d0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53d9e00640 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3d08c60 0x55b3d3d0b080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6895c551df45b33a server_cookie=142e26a7f31f7aea in_seq=0 out_seq=0 2024-04-02T17:14:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53bb400640 10 client.25685 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53bb400640 1 -- 172.21.15.59:0/1619351380 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3d3bf6c30 con 0x55b3d3d08c60 2024-04-02T17:14:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53bb400640 10 client.25685 handle_command_reply: tid=0 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 client.25685 shutdown 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 msgr2=0x55b3d3c9b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 0x55b3d3c9b3f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55b3d3c97b30 tx=0x7f53c80039c0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 shutdown_connections 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x55b3d3d08c60 0x55b3d3d0b080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3d3c89980 0x55b3d3c9c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b3d3c84810 0x55b3d3c9b3f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53bb400640 0 client.25685 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 --2- 172.21.15.59:0/1619351380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b3d3c83e20 0x55b3d3c9aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.898+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 >> 172.21.15.59:0/1619351380 conn(0x55b3d3c4ba50 msgr2=0x55b3d3c968d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 shutdown_connections 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53e37cd740 1 -- 172.21.15.59:0/1619351380 wait complete. 2024-04-02T17:14:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53e37cd740 20 client.25685 trim_cache size 0 max 16384 2024-04-02T17:14:09.902 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 msgr2=0x7f53c00a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 0x7f53c00a6290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53d4012d20 tx=0x7f53d4009ed0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 msgr2=0x7f53dc195b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53cc02efd0 tx=0x7f53cc02c660 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 shutdown_connections 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f53c00a3dd0 0x7f53c00a6290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53dc102bf0 0x7f53dc196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53dc1035b0 0x7f53dc19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 --2- 172.21.15.59:0/785049948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53dc101a00 0x7f53dc195b10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.899+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 >> 172.21.15.59:0/785049948 conn(0x7f53dc0fcf70 msgr2=0x7f53dc105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.900+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 shutdown_connections 2024-04-02T17:14:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:09.900+0000 7f53db200640 1 -- 172.21.15.59:0/785049948 wait complete. 2024-04-02T17:14:09.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 -- 172.21.15.59:0/3722677862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8105730 msgr2=0x7f50a810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 --2- 172.21.15.59:0/3722677862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8105730 0x7f50a810bfc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009be0 tx=0x7f50a002bef0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 -- 172.21.15.59:0/3722677862 shutdown_connections 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 --2- 172.21.15.59:0/3722677862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8105730 0x7f50a810bfc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 --2- 172.21.15.59:0/3722677862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 --2- 172.21.15.59:0/3722677862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8103b80 0x7f50a8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.067+0000 7f50ad800640 1 -- 172.21.15.59:0/3722677862 >> 172.21.15.59:0/3722677862 conn(0x7f50a80ff110 msgr2=0x7f50a8101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.068+0000 7f50ad800640 1 -- 172.21.15.59:0/3722677862 shutdown_connections 2024-04-02T17:14:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.068+0000 7f50ad800640 1 -- 172.21.15.59:0/3722677862 wait complete. 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 Processor -- start 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 -- start start 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 0x7f50a8199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 0x7f50a81a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50a810f830 con 0x7f50a8105730 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50a810f6b0 con 0x7f50a8103b80 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.069+0000 7f50ad800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a810f9b0 con 0x7f50a8104d70 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 0x7f50a8199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 0x7f50a8199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37558/0 (socket says 172.21.15.59:37558) 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7e00640 1 -- 172.21.15.59:0/3119737365 learned_addr learned my addr 172.21.15.59:0/3119737365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50ace00640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 0x7f50a81a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 -- 172.21.15.59:0/3119737365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 msgr2=0x7f50a8199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 0x7f50a8199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 -- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 msgr2=0x7f50a81a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 0x7f50a81a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50a7400640 1 -- 172.21.15.59:0/3119737365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a81a1a00 con 0x7f50a8104d70 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.070+0000 7f50ace00640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 0x7f50a81a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50a7400640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a819a320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50a4c00640 1 -- 172.21.15.59:0/3119737365 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400ccc0 con 0x7f50a8104d70 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50a4c00640 1 -- 172.21.15.59:0/3119737365 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5094010910 con 0x7f50a8104d70 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50a4c00640 1 -- 172.21.15.59:0/3119737365 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094018960 con 0x7f50a8104d70 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a81a1cf0 con 0x7f50a8104d70 2024-04-02T17:14:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.071+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81a2190 con 0x7f50a8104d70 2024-04-02T17:14:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.075+0000 7f50b1b76740 1 Processor -- start 2024-04-02T17:14:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.076+0000 7f50b1b76740 1 -- start start 2024-04-02T17:14:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.076+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f890b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50b1b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5579f8868d70 con 0x5579f8903100 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50b1b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579f88112a0 con 0x5579f8903af0 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50b1b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579f8864d20 con 0x5579f8908c60 2024-04-02T17:14:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50a4c00640 1 -- 172.21.15.59:0/3119737365 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5094010ac0 con 0x7f50a8104d70 2024-04-02T17:14:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46332/0 (socket says 172.21.15.59:46332) 2024-04-02T17:14:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.077+0000 7f50ace00640 1 -- 172.21.15.59:0/1290291526 learned_addr learned my addr 172.21.15.59:0/1290291526 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50a7400640 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f890b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50a7e00640 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50ace00640 1 -- 172.21.15.59:0/1290291526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 msgr2=0x5579f890b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50ace00640 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f890b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50ace00640 1 -- 172.21.15.59:0/1290291526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 msgr2=0x5579f89085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50ace00640 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.078+0000 7f50ace00640 1 -- 172.21.15.59:0/1290291526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f8825840 con 0x5579f8903100 2024-04-02T17:14:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.079+0000 7f50ace00640 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f50a0034fd0 tx=0x7f50a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ebc88c4bcba760 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.079+0000 7f509d600640 1 -- 172.21.15.59:0/1290291526 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0043070 con 0x5579f8903100 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.079+0000 7f50a4c00640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 0x7f508c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.080+0000 7f509d600640 1 -- 172.21.15.59:0/1290291526 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a0033bc0 con 0x5579f8903100 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.080+0000 7f509d600640 1 -- 172.21.15.59:0/1290291526 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a002caf0 con 0x5579f8903100 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.080+0000 7f50a4c00640 1 -- 172.21.15.59:0/3119737365 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f50940d0840 con 0x7f50a8104d70 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.080+0000 7f50a7e00640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 0x7f508c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.080+0000 7f50b1b76740 1 -- 172.21.15.59:0/1290291526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 msgr2=0x5579f8903520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f50a0034fd0 tx=0x7f50a0009840 comp rx=0 tx=0).stop 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 -- 172.21.15.59:0/1290291526 shutdown_connections 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f890b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1290291526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f8903520 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 -- 172.21.15.59:0/1290291526 >> 172.21.15.59:0/1290291526 conn(0x5579f88fe670 msgr2=0x5579f88fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50a7e00640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 0x7f508c0a60e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5579f8903ef0 tx=0x7f5098009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 -- 172.21.15.59:0/1290291526 shutdown_connections 2024-04-02T17:14:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.081+0000 7f50b1b76740 1 -- 172.21.15.59:0/1290291526 wait complete. 2024-04-02T17:14:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.082+0000 7f50b1b76740 1 -- start start 2024-04-02T17:14:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.082+0000 7f50b1b76740 1 Processor -- start 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 0x5579f890db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f8914b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5579f890bbb0 con 0x5579f8903af0 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579f8824660 con 0x5579f8903100 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579f890bd30 con 0x5579f8908c60 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50b1b76740 10 client.0 _ll_register_callbacks cb 0x5579f8824050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37572/0 (socket says 172.21.15.59:37572) 2024-04-02T17:14:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50a7e00640 1 -- 172.21.15.59:0/1492757032 learned_addr learned my addr 172.21.15.59:0/1492757032 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.083+0000 7f50ace00640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f8914b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7400640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 0x5579f890db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 -- 172.21.15.59:0/1492757032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 msgr2=0x5579f8914b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f8914b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 -- 172.21.15.59:0/1492757032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 msgr2=0x5579f890db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 0x5579f890db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 -- 172.21.15.59:0/1492757032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f8833e80 con 0x5579f8903100 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50ace00640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f8914b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7400640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 0x5579f890db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f50a7e00640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f509801d930 tx=0x7f5098004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38148b9839602068 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f5092000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509802d060 con 0x5579f8903100 2024-04-02T17:14:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.084+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5098018030 con 0x5579f8903100 2024-04-02T17:14:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.085+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0033bc0 con 0x5579f8903100 2024-04-02T17:14:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.085+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5098028420 con 0x5579f8903100 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.085+0000 7f50b1b76740 10 client.25697 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.085+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579f857d210 con 0x5579f8903100 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.085+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a0033bc0 con 0x5579f8903100 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5098031030 con 0x5579f8903100 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 10 client.25697 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 20 client.25697 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f89194e0 0x5579f891b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 4 client.25697 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579f8818cd0 con 0x5579f89194e0 2024-04-02T17:14:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.086+0000 7f50a7400640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f89194e0 0x5579f891b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.087+0000 7f50a7400640 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f89194e0 0x5579f891b950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6cb3cbb76f76e8 server_cookie=16f0731fac6b006f in_seq=0 out_seq=0 2024-04-02T17:14:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.087+0000 7f5092000640 10 client.25697 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f5092000640 1 -- 172.21.15.59:0/1492757032 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579f8818cd0 con 0x5579f89194e0 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f5092000640 10 client.25697 handle_command_reply: tid=0 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 client.25697 shutdown 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 msgr2=0x5579f890d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f509801d930 tx=0x7f5098004730 comp rx=0 tx=0).stop 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 shutdown_connections 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f89194e0 0x5579f891b950 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903af0 0x5579f890db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8908c60 0x5579f8914b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 --2- 172.21.15.59:0/1492757032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903100 0x5579f890d610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f5092000640 0 client.25697 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 >> 172.21.15.59:0/1492757032 conn(0x5579f88fe670 msgr2=0x5579f88fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 shutdown_connections 2024-04-02T17:14:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 1 -- 172.21.15.59:0/1492757032 wait complete. 2024-04-02T17:14:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.088+0000 7f50b1b76740 20 client.25697 trim_cache size 0 max 16384 2024-04-02T17:14:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.104+0000 7f50b1b76740 1 Processor -- start 2024-04-02T17:14:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 -- start start 2024-04-02T17:14:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f890d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8908c60 0x5579f88fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ab310 con 0x5579f8903100 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ace50 con 0x5579f8908c60 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50b1b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ab9e0 con 0x5579f8903af0 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f890d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50a7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50a7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56556/0 (socket says 172.21.15.59:56556) 2024-04-02T17:14:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50a7400640 1 -- 172.21.15.59:0/2888869994 learned_addr learned my addr 172.21.15.59:0/2888869994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.105+0000 7f50ace00640 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8908c60 0x5579f88fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7400640 1 -- 172.21.15.59:0/2888869994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8908c60 msgr2=0x5579f88fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7400640 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8908c60 0x5579f88fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7400640 1 -- 172.21.15.59:0/2888869994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 msgr2=0x5579f890d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7400640 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f890d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7400640 1 -- 172.21.15.59:0/2888869994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f8649620 con 0x5579f8903af0 2024-04-02T17:14:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.106+0000 7f50a7e00640 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f890d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.107+0000 7f50a7400640 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f509409c9f0 tx=0x7f50940bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7abfc9a79fdd3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.107+0000 7f509cc00640 1 -- 172.21.15.59:0/2888869994 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509409d160 con 0x5579f8903af0 2024-04-02T17:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.107+0000 7f509cc00640 1 -- 172.21.15.59:0/2888869994 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50940d6030 con 0x5579f8903af0 2024-04-02T17:14:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.107+0000 7f509cc00640 1 -- 172.21.15.59:0/2888869994 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940a5240 con 0x5579f8903af0 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- 172.21.15.59:0/2888869994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 msgr2=0x5579f890d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f509409c9f0 tx=0x7f50940bd040 comp rx=0 tx=0).stop 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- 172.21.15.59:0/2888869994 shutdown_connections 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8908c60 0x5579f88fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903af0 0x5579f890d760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2888869994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8903100 0x5579f890d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- 172.21.15.59:0/2888869994 >> 172.21.15.59:0/2888869994 conn(0x5579f88fe670 msgr2=0x5579f8904ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- 172.21.15.59:0/2888869994 shutdown_connections 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- 172.21.15.59:0/2888869994 wait complete. 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.108+0000 7f50b1b76740 1 -- start start 2024-04-02T17:14:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 Processor -- start 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903100 0x5579f8911390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ac780 con 0x5579f8908c60 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ac0b0 con 0x5579f8903af0 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81a92f0 con 0x5579f8903100 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50b1b76740 10 client.0 _ll_register_callbacks cb 0x7f509801d7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50a7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46360/0 (socket says 172.21.15.59:46360) 2024-04-02T17:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.109+0000 7f50ace00640 1 -- 172.21.15.59:0/2184976986 learned_addr learned my addr 172.21.15.59:0/2184976986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7e00640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903100 0x5579f8911390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7400640 1 -- 172.21.15.59:0/2184976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903100 msgr2=0x5579f8911390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7400640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903100 0x5579f8911390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7400640 1 -- 172.21.15.59:0/2184976986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 msgr2=0x5579f8912870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7400640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50a7400640 1 -- 172.21.15.59:0/2184976986 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f8912e60 con 0x5579f8903af0 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.110+0000 7f50ace00640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f50a7400640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89118d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f509409c9f0 tx=0x7f50940ce500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f7a4476aa97b08 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f5092000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50940a56d0 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f50940a5240 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50940d6030 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f509409d160 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f50b1b76740 10 client.25703 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579f8914170 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.111+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50940d6030 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5094010430 con 0x5579f8903af0 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 10 client.25703 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 20 client.25703 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f897ff30 0x5579f8982350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 4 client.25703 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:14:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579f88762b0 con 0x5579f897ff30 2024-04-02T17:14:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.112+0000 7f50a7e00640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f897ff30 0x5579f8982350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.113+0000 7f50a7e00640 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f897ff30 0x5579f8982350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ace46456d968fcc server_cookie=8cbc2d9ef9bfe8dd in_seq=0 out_seq=0 2024-04-02T17:14:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.113+0000 7f5092000640 10 client.25703 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f5092000640 1 -- 172.21.15.59:0/2184976986 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579f88762b0 con 0x5579f897ff30 2024-04-02T17:14:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f5092000640 10 client.25703 handle_command_reply: tid=0 2024-04-02T17:14:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 client.25703 shutdown 2024-04-02T17:14:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 msgr2=0x5579f89118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89118d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f509409c9f0 tx=0x7f50940ce500 comp rx=0 tx=0).stop 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 shutdown_connections 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5579f897ff30 0x5579f8982350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579f8908c60 0x5579f8912870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579f8903af0 0x5579f89118d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 --2- 172.21.15.59:0/2184976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579f8903100 0x5579f8911390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 >> 172.21.15.59:0/2184976986 conn(0x5579f88fe670 msgr2=0x5579f890d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f5092000640 0 client.25703 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.117+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 shutdown_connections 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.118+0000 7f50b1b76740 1 -- 172.21.15.59:0/2184976986 wait complete. 2024-04-02T17:14:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.118+0000 7f50b1b76740 20 client.25703 trim_cache size 0 max 16384 2024-04-02T17:14:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:14:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:14:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:14:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 msgr2=0x7f508c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 0x7f508c0a60e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5579f8903ef0 tx=0x7f5098009260 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 msgr2=0x7f50a819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a819a320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f509400e9f0 tx=0x7f509400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 shutdown_connections 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f508c0a3c20 0x7f508c0a60e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8105730 0x7f50a81a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a8104d70 0x7f50a819a320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 --2- 172.21.15.59:0/3119737365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a8103b80 0x7f50a8199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 >> 172.21.15.59:0/3119737365 conn(0x7f50a80ff110 msgr2=0x7f50a8107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 shutdown_connections 2024-04-02T17:14:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:10.119+0000 7f50ad800640 1 -- 172.21.15.59:0/3119737365 wait complete. 2024-04-02T17:14:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 -- 172.21.15.59:0/1687449091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac102d00 msgr2=0x7ff3ac105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 --2- 172.21.15.59:0/1687449091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac102d00 0x7ff3ac105170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff398009900 tx=0x7ff39802bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 -- 172.21.15.59:0/1687449091 shutdown_connections 2024-04-02T17:14:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 --2- 172.21.15.59:0/1687449091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac1056b0 0x7ff3ac105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 --2- 172.21.15.59:0/1687449091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac102d00 0x7ff3ac105170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 --2- 172.21.15.59:0/1687449091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac100390 0x7ff3ac1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 -- 172.21.15.59:0/1687449091 >> 172.21.15.59:0/1687449091 conn(0x7ff3ac0fbf90 msgr2=0x7ff3ac0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 -- 172.21.15.59:0/1687449091 shutdown_connections 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.148+0000 7ff3b3000640 1 -- 172.21.15.59:0/1687449091 wait complete. 2024-04-02T17:14:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.149+0000 7ff3b3000640 1 Processor -- start 2024-04-02T17:14:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 -- start start 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac1056b0 0x7ff3ac1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac113e10 con 0x7ff3ac102d00 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac113c90 con 0x7ff3ac100390 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3b3000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac113f90 con 0x7ff3ac1056b0 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.150+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37594/0 (socket says 172.21.15.59:37594) 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 -- 172.21.15.59:0/751807616 learned_addr learned my addr 172.21.15.59:0/751807616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46376/0 (socket says 172.21.15.59:46376) 2024-04-02T17:14:10.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3b1000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac1056b0 0x7ff3ac1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 -- 172.21.15.59:0/751807616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac1056b0 msgr2=0x7ff3ac1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac1056b0 0x7ff3ac1a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 -- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 msgr2=0x7ff3ac19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3abe00640 1 -- 172.21.15.59:0/751807616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac1a6060 con 0x7ff3ac100390 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.151+0000 7ff3ab400640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3abe00640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e9f0 tx=0x7ff39c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3a8c00640 1 -- 172.21.15.59:0/751807616 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00cd80 con 0x7ff3ac100390 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3a8c00640 1 -- 172.21.15.59:0/751807616 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c010910 con 0x7ff3ac100390 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3a8c00640 1 -- 172.21.15.59:0/751807616 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c018990 con 0x7ff3ac100390 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a6350 con 0x7ff3ac100390 2024-04-02T17:14:10.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.152+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a6800 con 0x7ff3ac100390 2024-04-02T17:14:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.156+0000 7ff3a8c00640 1 -- 172.21.15.59:0/751807616 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff39c010ac0 con 0x7ff3ac100390 2024-04-02T17:14:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.157+0000 7ff3b5dd0740 1 Processor -- start 2024-04-02T17:14:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.157+0000 7ff3b5dd0740 1 -- start start 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.157+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.157+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d4709a6400 con 0x55d470a65660 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d47098af10 con 0x55d470a6a7d0 2024-04-02T17:14:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4709ca6f0 con 0x55d470a64c70 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56610/0 (socket says 172.21.15.59:56610) 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3b1000640 1 -- 172.21.15.59:0/4244072194 learned_addr learned my addr 172.21.15.59:0/4244072194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3abe00640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3ab400640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.158+0000 7ff3a8c00640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 0x7ff3900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3a8c00640 1 -- 172.21.15.59:0/751807616 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff39c0d0730 con 0x7ff3ac100390 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 -- 172.21.15.59:0/4244072194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 msgr2=0x55d470a6cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a6cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 -- 172.21.15.59:0/4244072194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 msgr2=0x55d470a6a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a6a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 -- 172.21.15.59:0/4244072194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4706de3f0 con 0x55d470a64c70 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3b1000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 0x7ff3900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.159+0000 7ff3ab400640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.160+0000 7ff3b1000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 0x7ff3900a60f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4019000 tx=0x7ff3a401b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.160+0000 7ff3b1000640 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4011590 tx=0x7ff3a4011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fb0ca98e0c79fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.161+0000 7ff3a1600640 1 -- 172.21.15.59:0/4244072194 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4017050 con 0x55d470a64c70 2024-04-02T17:14:10.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.161+0000 7ff3a1600640 1 -- 172.21.15.59:0/4244072194 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a400f640 con 0x55d470a64c70 2024-04-02T17:14:10.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.161+0000 7ff3a1600640 1 -- 172.21.15.59:0/4244072194 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40053e0 con 0x55d470a64c70 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4244072194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 msgr2=0x55d470a65090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4011590 tx=0x7ff3a4011a50 comp rx=0 tx=0).stop 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4244072194 shutdown_connections 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a6a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4244072194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a65090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4244072194 >> 172.21.15.59:0/4244072194 conn(0x55d470a2ca50 msgr2=0x55d470a605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.162+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4244072194 shutdown_connections 2024-04-02T17:14:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.163+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4244072194 wait complete. 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.163+0000 7ff3b5dd0740 1 -- start start 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.163+0000 7ff3b5dd0740 1 Processor -- start 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a6f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d4709c66a0 con 0x55d470a64c70 2024-04-02T17:14:10.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d470972980 con 0x55d470a6a7d0 2024-04-02T17:14:10.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d470985cb0 con 0x55d470a65660 2024-04-02T17:14:10.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b5dd0740 10 client.0 _ll_register_callbacks cb 0x55d4709856a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46390/0 (socket says 172.21.15.59:46390) 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3b1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3abe00640 1 -- 172.21.15.59:0/2694432757 learned_addr learned my addr 172.21.15.59:0/2694432757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.164+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a6f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 -- 172.21.15.59:0/2694432757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 msgr2=0x55d470a6f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a6f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 -- 172.21.15.59:0/2694432757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 msgr2=0x55d470a764a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a764a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 -- 172.21.15.59:0/2694432757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4707a87d0 con 0x55d470a64c70 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff39c09c990 tx=0x7ff39c0a0d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8727256e19b6321 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.165+0000 7ff396000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c018d30 con 0x55d470a64c70 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff39c09be50 con 0x55d470a64c70 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a400f640 con 0x55d470a64c70 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff39c09c070 con 0x55d470a64c70 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff3b5dd0740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d470a79880 con 0x55d470a64c70 2024-04-02T17:14:10.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a400f640 con 0x55d470a64c70 2024-04-02T17:14:10.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.166+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c010550 con 0x55d470a64c70 2024-04-02T17:14:10.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 10 client.15630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 20 client.15630 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470a7afa0 0x55d470a7d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 4 client.15630 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d47098a040 con 0x55d470a7afa0 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.167+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470a7afa0 0x55d470a7d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.168+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470a7afa0 0x55d470a7d3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=698d46c5ce0f3a1a server_cookie=aea245d884a8ba67 in_seq=0 out_seq=0 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.168+0000 7ff396000640 10 client.15630 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.168+0000 7ff396000640 1 -- 172.21.15.59:0/2694432757 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d47098a040 con 0x55d470a7afa0 2024-04-02T17:14:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.168+0000 7ff396000640 10 client.15630 handle_command_reply: tid=0 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 client.15630 shutdown 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 msgr2=0x55d470a6ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff39c09c990 tx=0x7ff39c0a0d00 comp rx=0 tx=0).stop 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 shutdown_connections 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470a7afa0 0x55d470a7d3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2694432757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a6ef20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 >> 172.21.15.59:0/2694432757 conn(0x55d470a2ca50 msgr2=0x55d470a5f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff396000640 0 client.15630 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.169+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 shutdown_connections 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.170+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2694432757 wait complete. 2024-04-02T17:14:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.170+0000 7ff3b5dd0740 20 client.15630 trim_cache size 0 max 16384 2024-04-02T17:14:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.189+0000 7ff3b5dd0740 1 Processor -- start 2024-04-02T17:14:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 -- start start 2024-04-02T17:14:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a63d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1af880 con 0x55d470a64c70 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b13c0 con 0x55d470a6a7d0 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1aff50 con 0x55d470a65660 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a63820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46394/0 (socket says 172.21.15.59:46394) 2024-04-02T17:14:10.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3abe00640 1 -- 172.21.15.59:0/4096029054 learned_addr learned my addr 172.21.15.59:0/4096029054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56614/0 (socket says 172.21.15.59:56614) 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.190+0000 7ff3b1000640 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a63d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 -- 172.21.15.59:0/4096029054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 msgr2=0x55d470a63d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a63d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 -- 172.21.15.59:0/4096029054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 msgr2=0x55d470a63820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a63820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 -- 172.21.15.59:0/4096029054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d470a609f0 con 0x55d470a65660 2024-04-02T17:14:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3ab400640 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff398036040 tx=0x7ff39802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a390cfb86233238 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.191+0000 7ff3a0c00640 1 -- 172.21.15.59:0/4096029054 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398004100 con 0x55d470a65660 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.192+0000 7ff3a0c00640 1 -- 172.21.15.59:0/4096029054 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398050030 con 0x55d470a65660 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.192+0000 7ff3a0c00640 1 -- 172.21.15.59:0/4096029054 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398004b20 con 0x55d470a65660 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.192+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4096029054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 msgr2=0x55d470a76460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.192+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff398036040 tx=0x7ff39802df90 comp rx=0 tx=0).stop 2024-04-02T17:14:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4096029054 shutdown_connections 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a63d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a65660 0x55d470a76460 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/4096029054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a64c70 0x55d470a63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4096029054 >> 172.21.15.59:0/4096029054 conn(0x55d470a2ca50 msgr2=0x55d470a6d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4096029054 shutdown_connections 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/4096029054 wait complete. 2024-04-02T17:14:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.193+0000 7ff3b5dd0740 1 -- start start 2024-04-02T17:14:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 Processor -- start 2024-04-02T17:14:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b0cf0 con 0x55d470a65660 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b0620 con 0x55d470a6a7d0 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ad860 con 0x55d470a64c70 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.194+0000 7ff3b5dd0740 10 client.0 _ll_register_callbacks cb 0x7ff39c09c3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46400/0 (socket says 172.21.15.59:46400) 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3ab400640 1 -- 172.21.15.59:0/2196971503 learned_addr learned my addr 172.21.15.59:0/2196971503 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3b1000640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 -- 172.21.15.59:0/2196971503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 msgr2=0x55d470a74db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a74db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 -- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 msgr2=0x55d470a73df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3abe00640 1 -- 172.21.15.59:0/2196971503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d470a753a0 con 0x55d470a64c70 2024-04-02T17:14:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.195+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff3abe00640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a738b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff39c098e60 tx=0x7ff39c09ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df957e87a0f9a125 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff396000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c010640 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff398050030 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff39c087110 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff39c0b5050 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff3b5dd0740 10 client.25753 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.196+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d470a76710 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398050030 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c018b30 con 0x55d470a64c70 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 10 client.25753 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 10 client.25753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:14:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 20 client.25753 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470ae1aa0 0x55d470ae3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 4 client.25753 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4709d7c30 con 0x55d470ae1aa0 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.197+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470ae1aa0 0x55d470ae3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff3ab400640 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470ae1aa0 0x55d470ae3e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4bc34bbed66289 server_cookie=f3112a198b850355 in_seq=0 out_seq=0 2024-04-02T17:14:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff396000640 10 client.25753 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff396000640 1 -- 172.21.15.59:0/2196971503 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4709d7c30 con 0x55d470ae1aa0 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff396000640 10 client.25753 handle_command_reply: tid=0 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff3b5dd0740 1 client.25753 shutdown 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 msgr2=0x55d470a738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.198+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a738b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff39c098e60 tx=0x7ff39c09ca00 comp rx=0 tx=0).stop 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 shutdown_connections 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x55d470ae1aa0 0x55d470ae3e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d470a65660 0x55d470a73df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d470a6a7d0 0x55d470a74db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 --2- 172.21.15.59:0/2196971503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d470a64c70 0x55d470a738b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 >> 172.21.15.59:0/2196971503 conn(0x55d470a2ca50 msgr2=0x55d470a63820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff396000640 0 client.25753 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 shutdown_connections 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 1 -- 172.21.15.59:0/2196971503 wait complete. 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.199+0000 7ff3b5dd0740 20 client.25753 trim_cache size 0 max 16384 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 msgr2=0x7ff3900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 0x7ff3900a60f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4019000 tx=0x7ff3a401b000 comp rx=0 tx=0).stop 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 msgr2=0x7ff3ac19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e9f0 tx=0x7ff39c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 shutdown_connections 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff3900a3c30 0x7ff3900a60f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ac102d00 0x7ff3ac19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3ac1056b0 0x7ff3ac1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 --2- 172.21.15.59:0/751807616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3ac100390 0x7ff3ac19e440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 >> 172.21.15.59:0/751807616 conn(0x7ff3ac0fbf90 msgr2=0x7ff3ac100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 shutdown_connections 2024-04-02T17:14:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.202+0000 7ff3b3000640 1 -- 172.21.15.59:0/751807616 wait complete. 2024-04-02T17:14:10.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T17:14:10.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.456+0000 7f3463000640 1 -- 172.21.15.59:0/2695975733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 msgr2=0x7f345c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.456+0000 7f3463000640 1 --2- 172.21.15.59:0/2695975733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c103070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3448009900 tx=0x7f344802bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:10.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 -- 172.21.15.59:0/2695975733 shutdown_connections 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 --2- 172.21.15.59:0/2695975733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c103070 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 --2- 172.21.15.59:0/2695975733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c1035b0 0x7f345c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 --2- 172.21.15.59:0/2695975733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c101a00 0x7f345c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 -- 172.21.15.59:0/2695975733 >> 172.21.15.59:0/2695975733 conn(0x7f345c0fcf70 msgr2=0x7f345c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 -- 172.21.15.59:0/2695975733 shutdown_connections 2024-04-02T17:14:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.457+0000 7f3463000640 1 -- 172.21.15.59:0/2695975733 wait complete. 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.458+0000 7f3463000640 1 Processor -- start 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 -- start start 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c1035b0 0x7f345c074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10d6d0 con 0x7f345c102bf0 2024-04-02T17:14:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10d550 con 0x7f345c1035b0 2024-04-02T17:14:10.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3463000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10d850 con 0x7f345c101a00 2024-04-02T17:14:10.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3461200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3461c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3461c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56634/0 (socket says 172.21.15.59:56634) 2024-04-02T17:14:10.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3461200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46430/0 (socket says 172.21.15.59:46430) 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.459+0000 7f3461c00640 1 -- 172.21.15.59:0/1733883996 learned_addr learned my addr 172.21.15.59:0/1733883996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3462600640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c1035b0 0x7f345c074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 -- 172.21.15.59:0/1733883996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c1035b0 msgr2=0x7f345c074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c1035b0 0x7f345c074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 -- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 msgr2=0x7f345c073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 -- 172.21.15.59:0/1733883996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c074cc0 con 0x7f345c101a00 2024-04-02T17:14:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.460+0000 7f3461c00640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f345000e9c0 tx=0x7f345000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.461+0000 7f345a000640 1 -- 172.21.15.59:0/1733883996 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345000cde0 con 0x7f345c101a00 2024-04-02T17:14:10.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.461+0000 7f345a000640 1 -- 172.21.15.59:0/1733883996 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3450010910 con 0x7f345c101a00 2024-04-02T17:14:10.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.461+0000 7f345a000640 1 -- 172.21.15.59:0/1733883996 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450018a30 con 0x7f345c101a00 2024-04-02T17:14:10.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.461+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c1a1dd0 con 0x7f345c101a00 2024-04-02T17:14:10.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.461+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c1a2180 con 0x7f345c101a00 2024-04-02T17:14:10.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.464+0000 7f345a000640 1 -- 172.21.15.59:0/1733883996 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f345001f020 con 0x7f345c101a00 2024-04-02T17:14:10.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.466+0000 7f345a000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 0x7f34440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.467+0000 7f345a000640 1 -- 172.21.15.59:0/1733883996 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f34500d0720 con 0x7f345c101a00 2024-04-02T17:14:10.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.467+0000 7f34673cc740 1 Processor -- start 2024-04-02T17:14:10.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.467+0000 7f3461200640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 0x7f34440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.467+0000 7f34673cc740 1 -- start start 2024-04-02T17:14:10.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b685b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b688620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55956b5b9400 con 0x55956b680630 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55956b59df10 con 0x55956b686190 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f34673cc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55956b5dd6f0 con 0x55956b681020 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f3461200640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 0x7f34440a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f344802c240 tx=0x7f344802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46444/0 (socket says 172.21.15.59:46444) 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f3461c00640 1 -- 172.21.15.59:0/2918242720 learned_addr learned my addr 172.21.15.59:0/2918242720 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.468+0000 7f3462600640 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b688620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461200640 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b685b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 -- 172.21.15.59:0/2918242720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 msgr2=0x55956b685b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b685b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 -- 172.21.15.59:0/2918242720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 msgr2=0x55956b688620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b688620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 -- 172.21.15.59:0/2918242720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b2f13f0 con 0x55956b680630 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.469+0000 7f3461c00640 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3450099310 tx=0x7f345009c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786df9c4969b8413 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.470+0000 7f344ea00640 1 -- 172.21.15.59:0/2918242720 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500a0cf0 con 0x55956b680630 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.470+0000 7f344ea00640 1 -- 172.21.15.59:0/2918242720 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34500a55b0 con 0x55956b680630 2024-04-02T17:14:10.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.470+0000 7f344ea00640 1 -- 172.21.15.59:0/2918242720 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500a5380 con 0x55956b680630 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 -- 172.21.15.59:0/2918242720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 msgr2=0x55956b680a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3450099310 tx=0x7f345009c3d0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 -- 172.21.15.59:0/2918242720 shutdown_connections 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b688620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b685b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 --2- 172.21.15.59:0/2918242720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b680a50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 -- 172.21.15.59:0/2918242720 >> 172.21.15.59:0/2918242720 conn(0x55956b63fa50 msgr2=0x55956b67bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 -- 172.21.15.59:0/2918242720 shutdown_connections 2024-04-02T17:14:10.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.471+0000 7f34673cc740 1 -- 172.21.15.59:0/2918242720 wait complete. 2024-04-02T17:14:10.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.472+0000 7f34673cc740 1 -- start start 2024-04-02T17:14:10.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.472+0000 7f34673cc740 1 Processor -- start 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b686190 0x55956b6a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55956b5d96a0 con 0x55956b680630 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55956b585980 con 0x55956b681020 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55956b598cb0 con 0x55956b686190 2024-04-02T17:14:10.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f34673cc740 10 client.0 _ll_register_callbacks cb 0x55956b5986a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46456/0 (socket says 172.21.15.59:46456) 2024-04-02T17:14:10.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f3461c00640 1 -- 172.21.15.59:0/2432791702 learned_addr learned my addr 172.21.15.59:0/2432791702 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f3461200640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.473+0000 7f3462600640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b686190 0x55956b6a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 -- 172.21.15.59:0/2432791702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b686190 msgr2=0x55956b6a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b686190 0x55956b6a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 -- 172.21.15.59:0/2432791702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 msgr2=0x55956b69b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 -- 172.21.15.59:0/2432791702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b3bb7d0 con 0x55956b681020 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461c00640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3461200640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69bda0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3448036040 tx=0x7f3448009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5a71588e26b61c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3443400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448052070 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34500a55b0 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.474+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3448004990 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f344805d050 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 10 client.25733 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55956b6a61c0 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34500a55b0 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f344804b050 con 0x55956b681020 2024-04-02T17:14:10.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 10 client.25733 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 20 client.25733 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b6a7840 0x55956b6a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 4 client.25733 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.475+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956b59d040 con 0x55956b6a7840 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.476+0000 7f3461c00640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b6a7840 0x55956b6a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.476+0000 7f3461c00640 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b6a7840 0x55956b6a9c50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a71b13852c9c40 server_cookie=d9e212a0aae53586 in_seq=0 out_seq=0 2024-04-02T17:14:10.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.476+0000 7f3443400640 10 client.25733 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.477+0000 7f3443400640 1 -- 172.21.15.59:0/2432791702 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55956b59d040 con 0x55956b6a7840 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.477+0000 7f3443400640 10 client.25733 handle_command_reply: tid=0 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.477+0000 7f34673cc740 1 client.25733 shutdown 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 msgr2=0x55956b69bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69bda0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3448036040 tx=0x7f3448009260 comp rx=0 tx=0).stop 2024-04-02T17:14:10.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 shutdown_connections 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b6a7840 0x55956b6a9c50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b686190 0x55956b6a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69bda0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 --2- 172.21.15.59:0/2432791702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b69b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 >> 172.21.15.59:0/2432791702 conn(0x55956b63fa50 msgr2=0x55956b67d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f3443400640 0 client.25733 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 shutdown_connections 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 1 -- 172.21.15.59:0/2432791702 wait complete. 2024-04-02T17:14:10.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.478+0000 7f34673cc740 20 client.25733 trim_cache size 0 max 16384 2024-04-02T17:14:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 Processor -- start 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 -- start start 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b6a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b67c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f345c06fe30 con 0x55956b680630 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f345c071970 con 0x55956b686190 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.507+0000 7f34673cc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f345c070500 con 0x55956b681020 2024-04-02T17:14:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46468/0 (socket says 172.21.15.59:46468) 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 -- 172.21.15.59:0/2769140075 learned_addr learned my addr 172.21.15.59:0/2769140075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461200640 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b6a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3462600640 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b67c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 -- 172.21.15.59:0/2769140075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 msgr2=0x55956b6a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b6a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 -- 172.21.15.59:0/2769140075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 msgr2=0x55956b67c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b67c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.508+0000 7f3461c00640 1 -- 172.21.15.59:0/2769140075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b67ccf0 con 0x55956b680630 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.509+0000 7f3461c00640 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3450099310 tx=0x7f345000efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e19a22e518ecd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.509+0000 7f344e000640 1 -- 172.21.15.59:0/2769140075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450098fb0 con 0x55956b680630 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.509+0000 7f344e000640 1 -- 172.21.15.59:0/2769140075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345009ac40 con 0x55956b680630 2024-04-02T17:14:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.509+0000 7f344e000640 1 -- 172.21.15.59:0/2769140075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345009bd90 con 0x55956b680630 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 -- 172.21.15.59:0/2769140075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 msgr2=0x55956b67bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3450099310 tx=0x7f345000efb0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 -- 172.21.15.59:0/2769140075 shutdown_connections 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b686190 0x55956b67c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b681020 0x55956b6a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 --2- 172.21.15.59:0/2769140075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b680630 0x55956b67bac0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 -- 172.21.15.59:0/2769140075 >> 172.21.15.59:0/2769140075 conn(0x55956b63fa50 msgr2=0x55956b69b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 -- 172.21.15.59:0/2769140075 shutdown_connections 2024-04-02T17:14:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.510+0000 7f34673cc740 1 -- 172.21.15.59:0/2769140075 wait complete. 2024-04-02T17:14:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.511+0000 7f34673cc740 1 -- start start 2024-04-02T17:14:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.511+0000 7f34673cc740 1 Processor -- start 2024-04-02T17:14:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.511+0000 7f34673cc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.511+0000 7f34673cc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f34673cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 0x55956b70bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f34673cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f345c0712a0 con 0x55956b686190 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f34673cc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f345c070bd0 con 0x55956b681020 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f34673cc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f345c06de30 con 0x55956b680630 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f34673cc740 10 client.0 _ll_register_callbacks cb 0x7f34480040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56668/0 (socket says 172.21.15.59:56668) 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 -- 172.21.15.59:0/2709898732 learned_addr learned my addr 172.21.15.59:0/2709898732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461200640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 -- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 msgr2=0x55956b69fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3461c00640 1 -- 172.21.15.59:0/2709898732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 msgr2=0x55956b70bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.512+0000 7f3462600640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 0x55956b70bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3461c00640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 0x55956b70bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3461c00640 1 -- 172.21.15.59:0/2709898732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b6a2eb0 con 0x55956b680630 2024-04-02T17:14:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3462600640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 0x55956b70bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3461c00640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3450098da0 tx=0x7f345000c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbe72e3e3d999b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3443400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500871b0 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f345009bd90 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.513+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345009ac40 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3450087350 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 10 client.25771 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55956b70d190 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345009ac40 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3450098690 con 0x55956b680630 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 10 client.25771 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 10 client.25771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 20 client.25771 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b70d570 0x55956b70f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 4 client.25771 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.514+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55956b5eac30 con 0x55956b70d570 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.515+0000 7f3461200640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b70d570 0x55956b70f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.515+0000 7f3461200640 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b70d570 0x55956b70f990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f46682bd1b6790 server_cookie=ce758c1cd4b0bba6 in_seq=0 out_seq=0 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.515+0000 7f3443400640 10 client.25771 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f3443400640 1 -- 172.21.15.59:0/2709898732 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55956b5eac30 con 0x55956b70d570 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f3443400640 10 client.25771 handle_command_reply: tid=0 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 client.25771 shutdown 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 msgr2=0x55956b69f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3450098da0 tx=0x7f345000c860 comp rx=0 tx=0).stop 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 shutdown_connections 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55956b70d570 0x55956b70f990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55956b686190 0x55956b70bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55956b681020 0x55956b69fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 --2- 172.21.15.59:0/2709898732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55956b680630 0x55956b69f8a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 >> 172.21.15.59:0/2709898732 conn(0x55956b63fa50 msgr2=0x55956b672260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f3443400640 0 client.25771 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 shutdown_connections 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.516+0000 7f34673cc740 1 -- 172.21.15.59:0/2709898732 wait complete. 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f34673cc740 20 client.25771 trim_cache size 0 max 16384 2024-04-02T17:14:10.519 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 msgr2=0x7f34440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 0x7f34440a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f344802c240 tx=0x7f344802df90 comp rx=0 tx=0).stop 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 msgr2=0x7f345c0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f345000e9c0 tx=0x7f345000ee80 comp rx=0 tx=0).stop 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 shutdown_connections 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f34440a3c30 0x7f34440a60f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f345c102bf0 0x7f345c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f345c1035b0 0x7f345c074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 --2- 172.21.15.59:0/1733883996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f345c101a00 0x7f345c0758b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:10.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.517+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 >> 172.21.15.59:0/1733883996 conn(0x7f345c0fcf70 msgr2=0x7f345c105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:10.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.518+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 shutdown_connections 2024-04-02T17:14:10.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:10.518+0000 7f3463000640 1 -- 172.21.15.59:0/1733883996 wait complete. 2024-04-02T17:14:11.534 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T17:14:11.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:14:11.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T17:14:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:11 smithi059 ceph-mon[31664]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:11 smithi164 ceph-mon[36499]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:11 smithi154 ceph-mon[36479]: pgmap v421: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 -- 172.21.15.59:0/3136112109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e01058f0 msgr2=0x7f00e010c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 --2- 172.21.15.59:0/3136112109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e01058f0 0x7f00e010c180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009be0 tx=0x7f00dc02bef0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 -- 172.21.15.59:0/3136112109 shutdown_connections 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 --2- 172.21.15.59:0/3136112109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0104f30 0x7f00e01053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 --2- 172.21.15.59:0/3136112109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e01058f0 0x7f00e010c180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 --2- 172.21.15.59:0/3136112109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0103d40 0x7f00e0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 -- 172.21.15.59:0/3136112109 >> 172.21.15.59:0/3136112109 conn(0x7f00e00ff2b0 msgr2=0x7f00e01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 -- 172.21.15.59:0/3136112109 shutdown_connections 2024-04-02T17:14:11.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.728+0000 7f00e8800640 1 -- 172.21.15.59:0/3136112109 wait complete. 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.729+0000 7f00e8800640 1 Processor -- start 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.730+0000 7f00e8800640 1 -- start start 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.730+0000 7f00e8800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 0x7f00e019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.730+0000 7f00e8800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.730+0000 7f00e8800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 0x7f00e01a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.730+0000 7f00e8800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00e010f9c0 con 0x7f00e0103d40 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e8800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e010f840 con 0x7f00e0104f30 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 0x7f00e019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54006/0 (socket says 172.21.15.59:54006) 2024-04-02T17:14:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5400640 1 -- 172.21.15.59:0/2253740112 learned_addr learned my addr 172.21.15.59:0/2253740112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e010fb40 con 0x7f00e01058f0 2024-04-02T17:14:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 0x7f00e01a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 -- 172.21.15.59:0/2253740112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 msgr2=0x7f00e019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 -- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 msgr2=0x7f00e019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 0x7f00e019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e6800640 1 -- 172.21.15.59:0/2253740112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e01a5ff0 con 0x7f00e01058f0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5400640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.731+0000 7f00e5e00640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 0x7f00e019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.732+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 0x7f00e01a5980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02c430 tx=0x7f00dc02cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.732+0000 7f00daa00640 1 -- 172.21.15.59:0/2253740112 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc043020 con 0x7f00e01058f0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.732+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01a6280 con 0x7f00e01058f0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.732+0000 7f00daa00640 1 -- 172.21.15.59:0/2253740112 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc003e50 con 0x7f00e01058f0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.732+0000 7f00daa00640 1 -- 172.21.15.59:0/2253740112 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc034690 con 0x7f00e01058f0 2024-04-02T17:14:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.733+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01a6820 con 0x7f00e01058f0 2024-04-02T17:14:11.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.736+0000 7f00daa00640 1 -- 172.21.15.59:0/2253740112 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f00dc046020 con 0x7f00e01058f0 2024-04-02T17:14:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.737+0000 7f00eb576740 1 Processor -- start 2024-04-02T17:14:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.737+0000 7f00eb576740 1 -- start start 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f489cca310 con 0x55f489d6ae10 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f489c72690 con 0x55f489d652b0 2024-04-02T17:14:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00eb576740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f489cc62c0 con 0x55f489d65ca0 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00daa00640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00e5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00e5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54020/0 (socket says 172.21.15.59:54020) 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00e5400640 1 -- 172.21.15.59:0/450715270 learned_addr learned my addr 172.21.15.59:0/450715270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00e5e00640 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00e5400640 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.738+0000 7f00daa00640 1 -- 172.21.15.59:0/2253740112 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f00dc0f29e0 con 0x7f00e01058f0 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 -- 172.21.15.59:0/450715270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 msgr2=0x55f489d656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 -- 172.21.15.59:0/450715270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 msgr2=0x55f489d6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 -- 172.21.15.59:0/450715270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f489aa67e0 con 0x55f489d65ca0 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.739+0000 7f00e5e00640 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009fa0 tx=0x7f00d40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557a81176579b592 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.740+0000 7f00e6800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009540 tx=0x7f00dc02bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.740+0000 7f00cf000640 1 -- 172.21.15.59:0/450715270 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004030 con 0x55f489d65ca0 2024-04-02T17:14:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.740+0000 7f00cf000640 1 -- 172.21.15.59:0/450715270 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d40041d0 con 0x55f489d65ca0 2024-04-02T17:14:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.740+0000 7f00cf000640 1 -- 172.21.15.59:0/450715270 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400f910 con 0x55f489d65ca0 2024-04-02T17:14:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 -- 172.21.15.59:0/450715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 msgr2=0x55f489d6a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009fa0 tx=0x7f00d40118d0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 -- 172.21.15.59:0/450715270 shutdown_connections 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 --2- 172.21.15.59:0/450715270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 -- 172.21.15.59:0/450715270 >> 172.21.15.59:0/450715270 conn(0x55f489cfa270 msgr2=0x55f489d60c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 -- 172.21.15.59:0/450715270 shutdown_connections 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.741+0000 7f00eb576740 1 -- 172.21.15.59:0/450715270 wait complete. 2024-04-02T17:14:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.742+0000 7f00eb576740 1 -- start start 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.742+0000 7f00eb576740 1 Processor -- start 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.742+0000 7f00eb576740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d6f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.742+0000 7f00eb576740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00eb576740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00eb576740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f489d6ddc0 con 0x55f489d6ae10 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00eb576740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f489c85a60 con 0x55f489d652b0 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00eb576740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f489d6df40 con 0x55f489d65ca0 2024-04-02T17:14:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00eb576740 10 client.0 _ll_register_callbacks cb 0x55f489c8b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60990/0 (socket says 172.21.15.59:60990) 2024-04-02T17:14:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00e5400640 1 -- 172.21.15.59:0/61299464 learned_addr learned my addr 172.21.15.59:0/61299464 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00e5e00640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.743+0000 7f00e5400640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d6f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 -- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 msgr2=0x55f489d6f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d6f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 -- 172.21.15.59:0/61299464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 msgr2=0x55f489d76d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 -- 172.21.15.59:0/61299464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f489b06f70 con 0x55f489d65ca0 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5400640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00e5e00640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6fd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f489d61b70 tx=0x7f00d4009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf4f2fc77284180 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00c3400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.744+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4027070 con 0x55f489d65ca0 2024-04-02T17:14:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d40041d0 con 0x55f489d65ca0 2024-04-02T17:14:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f00d40039b0 con 0x55f489d65ca0 2024-04-02T17:14:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00d4033050 con 0x55f489d65ca0 2024-04-02T17:14:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00eb576740 10 client.25783 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f48999e170 con 0x55f489d65ca0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.745+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d40041d0 con 0x55f489d65ca0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f00d4004c30 con 0x55f489d65ca0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 10 client.25783 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 20 client.25783 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489d7b4a0 0x55f489d7d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 4 client.25783 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f489cd7850 con 0x55f489d7b4a0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.746+0000 7f00e5400640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489d7b4a0 0x55f489d7d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.747+0000 7f00e5400640 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489d7b4a0 0x55f489d7d8c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27611c7893a49006 server_cookie=84190cc280c78ac4 in_seq=0 out_seq=0 2024-04-02T17:14:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.747+0000 7f00c3400640 10 client.25783 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00c3400640 1 -- 172.21.15.59:0/61299464 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f489cd7850 con 0x55f489d7b4a0 2024-04-02T17:14:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00c3400640 10 client.25783 handle_command_reply: tid=0 2024-04-02T17:14:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 client.25783 shutdown 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 msgr2=0x55f489d6fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6fd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f489d61b70 tx=0x7f00d4009c80 comp rx=0 tx=0).stop 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 shutdown_connections 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489d7b4a0 0x55f489d7d8c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d6ae10 0x55f489d76d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d65ca0 0x55f489d6fd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 --2- 172.21.15.59:0/61299464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d652b0 0x55f489d6f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 >> 172.21.15.59:0/61299464 conn(0x55f489cfa270 msgr2=0x55f489d5fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00c3400640 0 client.25783 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 shutdown_connections 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.748+0000 7f00eb576740 1 -- 172.21.15.59:0/61299464 wait complete. 2024-04-02T17:14:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.749+0000 7f00eb576740 20 client.25783 trim_cache size 0 max 16384 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.767+0000 7f00eb576740 1 Processor -- start 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.767+0000 7f00eb576740 1 -- start start 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d652b0 0x55f489d5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d6ae10 0x55f489d76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01af5c0 con 0x55f489d65ca0 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b1100 con 0x55f489d6ae10 2024-04-02T17:14:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00eb576740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01afc90 con 0x55f489d652b0 2024-04-02T17:14:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60994/0 (socket says 172.21.15.59:60994) 2024-04-02T17:14:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5e00640 1 -- 172.21.15.59:0/3629353922 learned_addr learned my addr 172.21.15.59:0/3629353922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5400640 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d652b0 0x55f489d5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5400640 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d6ae10 0x55f489d76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.768+0000 7f00e5e00640 1 -- 172.21.15.59:0/3629353922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d652b0 msgr2=0x55f489d5fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00e5e00640 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d652b0 0x55f489d5fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00e5e00640 1 -- 172.21.15.59:0/3629353922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d6ae10 msgr2=0x55f489d76d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00e5e00640 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d6ae10 0x55f489d76d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00e5e00640 1 -- 172.21.15.59:0/3629353922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f489d66080 con 0x55f489d65ca0 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00e5e00640 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faef8fd31ff1a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00ce600640 1 -- 172.21.15.59:0/3629353922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4027040 con 0x55f489d65ca0 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00ce600640 1 -- 172.21.15.59:0/3629353922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d4004190 con 0x55f489d65ca0 2024-04-02T17:14:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.769+0000 7f00ce600640 1 -- 172.21.15.59:0/3629353922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400fc60 con 0x55f489d65ca0 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 -- 172.21.15.59:0/3629353922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 msgr2=0x55f489d601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4009e20 comp rx=0 tx=0).stop 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 -- 172.21.15.59:0/3629353922 shutdown_connections 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d65ca0 0x55f489d601e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d6ae10 0x55f489d76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 --2- 172.21.15.59:0/3629353922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d652b0 0x55f489d5fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.770+0000 7f00eb576740 1 -- 172.21.15.59:0/3629353922 >> 172.21.15.59:0/3629353922 conn(0x55f489cfa270 msgr2=0x55f489d5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.771+0000 7f00eb576740 1 -- 172.21.15.59:0/3629353922 shutdown_connections 2024-04-02T17:14:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.771+0000 7f00eb576740 1 -- 172.21.15.59:0/3629353922 wait complete. 2024-04-02T17:14:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.771+0000 7f00eb576740 1 -- start start 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.771+0000 7f00eb576740 1 Processor -- start 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 0x55f489d74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d6ae10 0x55f489d757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b0a30 con 0x55f489d652b0 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b0360 con 0x55f489d65ca0 2024-04-02T17:14:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01ad5a0 con 0x55f489d6ae10 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00eb576740 10 client.0 _ll_register_callbacks cb 0x7f00d0011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32774/0 (socket says 172.21.15.59:32774) 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.772+0000 7f00e5400640 1 -- 172.21.15.59:0/1224487036 learned_addr learned my addr 172.21.15.59:0/1224487036 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5e00640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 0x55f489d74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d6ae10 0x55f489d757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 -- 172.21.15.59:0/1224487036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d6ae10 msgr2=0x55f489d757e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d6ae10 0x55f489d757e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 -- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 msgr2=0x55f489d74820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 0x55f489d74820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 -- 172.21.15.59:0/1224487036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f489d75dd0 con 0x55f489d652b0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5e00640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 0x55f489d74820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00c3400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.773+0000 7f00e5400640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f489d7e0d0 tx=0x7f00d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9558bf59990939 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000fe90 con 0x55f489d652b0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d4004190 con 0x55f489d652b0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f00d00285a0 con 0x55f489d652b0 2024-04-02T17:14:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00d003a050 con 0x55f489d652b0 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00eb576740 10 client.15669 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.774+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f489de1a00 con 0x55f489d652b0 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d4004190 con 0x55f489d652b0 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f00d0020040 con 0x55f489d652b0 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 10 client.15669 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 20 client.15669 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489de2380 0x55f489de47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 4 client.15669 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.775+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f489cd1ba0 con 0x55f489de2380 2024-04-02T17:14:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.776+0000 7f00e5e00640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489de2380 0x55f489de47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.777+0000 7f00e5e00640 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489de2380 0x55f489de47a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dac80ce59c95478 server_cookie=d32b3396a5e65619 in_seq=0 out_seq=0 2024-04-02T17:14:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.777+0000 7f00c3400640 10 client.15669 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00c3400640 1 -- 172.21.15.59:0/1224487036 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f489cd1ba0 con 0x55f489de2380 2024-04-02T17:14:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00c3400640 10 client.15669 handle_command_reply: tid=0 2024-04-02T17:14:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 client.15669 shutdown 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 msgr2=0x55f489d742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f489d7e0d0 tx=0x7f00d0009260 comp rx=0 tx=0).stop 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 shutdown_connections 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f489de2380 0x55f489de47a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f489d6ae10 0x55f489d757e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f489d65ca0 0x55f489d74820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 --2- 172.21.15.59:0/1224487036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f489d652b0 0x55f489d742e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 >> 172.21.15.59:0/1224487036 conn(0x55f489cfa270 msgr2=0x55f489d5fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.778+0000 7f00c3400640 0 client.15669 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 shutdown_connections 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00eb576740 1 -- 172.21.15.59:0/1224487036 wait complete. 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00eb576740 20 client.15669 trim_cache size 0 max 16384 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "b86b2dc5-faaa-4985-9b44-eb5caa7fb410", 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-02T17:14:11.780 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 msgr2=0x7f00c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009540 tx=0x7f00dc02bef0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 msgr2=0x7f00e01a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.779+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 0x7f00e01a5980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02c430 tx=0x7f00dc02cb90 comp rx=0 tx=0).stop 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 shutdown_connections 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e01058f0 0x7f00e01a5980 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00e0104f30 0x7f00e019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f00c80a3c30 0x7f00c80a60f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 --2- 172.21.15.59:0/2253740112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00e0103d40 0x7f00e019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 >> 172.21.15.59:0/2253740112 conn(0x7f00e00ff2b0 msgr2=0x7f00e0077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 shutdown_connections 2024-04-02T17:14:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:11.780+0000 7f00e8800640 1 -- 172.21.15.59:0/2253740112 wait complete. 2024-04-02T17:14:11.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:14:11.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:14:12.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.044+0000 7f17e2e00640 1 -- 172.21.15.59:0/2496072048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc074560 msgr2=0x7f17dc0749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.044+0000 7f17e2e00640 1 --2- 172.21.15.59:0/2496072048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc074560 0x7f17dc0749e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009860 tx=0x7f17c802bb70 comp rx=0 tx=0).stop 2024-04-02T17:14:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 -- 172.21.15.59:0/2496072048 shutdown_connections 2024-04-02T17:14:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 --2- 172.21.15.59:0/2496072048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1098c0 0x7f17dc10bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 --2- 172.21.15.59:0/2496072048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc074560 0x7f17dc0749e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 --2- 172.21.15.59:0/2496072048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 -- 172.21.15.59:0/2496072048 >> 172.21.15.59:0/2496072048 conn(0x7f17dc0fcf70 msgr2=0x7f17dc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 -- 172.21.15.59:0/2496072048 shutdown_connections 2024-04-02T17:14:12.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.046+0000 7f17e2e00640 1 -- 172.21.15.59:0/2496072048 wait complete. 2024-04-02T17:14:12.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.047+0000 7f17e2e00640 1 Processor -- start 2024-04-02T17:14:12.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.047+0000 7f17e2e00640 1 -- start start 2024-04-02T17:14:12.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 0x7f17dc19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc1098c0 0x7f17dc1a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10f840 con 0x7f17dc074560 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10f6c0 con 0x7f17dc1098c0 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10f9c0 con 0x7f17dc075b40 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 0x7f17dc19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 0x7f17dc19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32794/0 (socket says 172.21.15.59:32794) 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1a00640 1 -- 172.21.15.59:0/3831627252 learned_addr learned my addr 172.21.15.59:0/3831627252 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e2400640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc1098c0 0x7f17dc1a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 -- 172.21.15.59:0/3831627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc1098c0 msgr2=0x7f17dc1a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc1098c0 0x7f17dc1a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 -- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 msgr2=0x7f17dc19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 0x7f17dc19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.048+0000 7f17e1000640 1 -- 172.21.15.59:0/3831627252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc1a5f30 con 0x7f17dc075b40 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17e1000640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc19e850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17c802c0b0 tx=0x7f17c8032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17da000640 1 -- 172.21.15.59:0/3831627252 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8041020 con 0x7f17dc075b40 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17da000640 1 -- 172.21.15.59:0/3831627252 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c802c9f0 con 0x7f17dc075b40 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1a61c0 con 0x7f17dc075b40 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17da000640 1 -- 172.21.15.59:0/3831627252 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8039040 con 0x7f17dc075b40 2024-04-02T17:14:12.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.049+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a66d0 con 0x7f17dc075b40 2024-04-02T17:14:12.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.053+0000 7f17da000640 1 -- 172.21.15.59:0/3831627252 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f17c8044020 con 0x7f17dc075b40 2024-04-02T17:14:12.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 Processor -- start 2024-04-02T17:14:12.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 -- start start 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa6f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5629aa655770 con 0x5629aa6f64e0 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629aa5fd700 con 0x5629aa6f0980 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.054+0000 7f17e7184740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5629aa651720 con 0x5629aa6f1370 2024-04-02T17:14:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e2400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e2400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54088/0 (socket says 172.21.15.59:54088) 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e2400640 1 -- 172.21.15.59:0/2241037529 learned_addr learned my addr 172.21.15.59:0/2241037529 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17da000640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 0x7f17c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e1a00640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa6f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17da000640 1 -- 172.21.15.59:0/3831627252 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f17c80f1bc0 con 0x7f17dc075b40 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e1000640 1 -- 172.21.15.59:0/2241037529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 msgr2=0x5629aa6f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.055+0000 7f17e1000640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa6f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e1000640 1 -- 172.21.15.59:0/2241037529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 msgr2=0x5629aa6f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e1000640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e1000640 1 -- 172.21.15.59:0/2241037529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aa4337d0 con 0x5629aa6f64e0 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e2400640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e2400640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 0x7f17c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e1000640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6f8970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17c80ef5a0 tx=0x7f17c802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3eefc41ee39d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17e1a00640 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa6f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17cea00640 1 -- 172.21.15.59:0/2241037529 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8032e50 con 0x5629aa6f64e0 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17cea00640 1 -- 172.21.15.59:0/2241037529 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c80be4e0 con 0x5629aa6f64e0 2024-04-02T17:14:12.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.056+0000 7f17cea00640 1 -- 172.21.15.59:0/2241037529 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80042c0 con 0x5629aa6f64e0 2024-04-02T17:14:12.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 -- 172.21.15.59:0/2241037529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 msgr2=0x5629aa6f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6f8970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17c80ef5a0 tx=0x7f17c802bb70 comp rx=0 tx=0).stop 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e2400640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 0x7f17c40a60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5629aa6f1bd0 tx=0x7f17d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 -- 172.21.15.59:0/2241037529 shutdown_connections 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6f8970 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa6f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 --2- 172.21.15.59:0/2241037529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 -- 172.21.15.59:0/2241037529 >> 172.21.15.59:0/2241037529 conn(0x5629aa6ebef0 msgr2=0x5629aa6ec2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 -- 172.21.15.59:0/2241037529 shutdown_connections 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.057+0000 7f17e7184740 1 -- 172.21.15.59:0/2241037529 wait complete. 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.058+0000 7f17e7184740 1 -- start start 2024-04-02T17:14:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.058+0000 7f17e7184740 1 Processor -- start 2024-04-02T17:14:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f0980 0x5629aa6fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 0x5629aa702110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5629aa655770 con 0x5629aa6f0980 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629aa610ac0 con 0x5629aa6f1370 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5629aa6f9250 con 0x5629aa6f64e0 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e7184740 10 client.0 _ll_register_callbacks cb 0x5629aa613180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54096/0 (socket says 172.21.15.59:54096) 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e1000640 1 -- 172.21.15.59:0/141134013 learned_addr learned my addr 172.21.15.59:0/141134013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e1a00640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f0980 0x5629aa6fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.059+0000 7f17e2400640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 0x5629aa702110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 -- 172.21.15.59:0/141134013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 msgr2=0x5629aa702110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 0x5629aa702110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 -- 172.21.15.59:0/141134013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f0980 msgr2=0x5629aa6fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f0980 0x5629aa6fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 -- 172.21.15.59:0/141134013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aa3693f0 con 0x5629aa6f1370 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e2400640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 0x5629aa702110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17e1000640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009780 tx=0x7f17c80f51a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2b7fa864e7da7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17c3400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80c1070 con 0x5629aa6f1370 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.060+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f17c80ba450 con 0x5629aa6f1370 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c80be4e0 con 0x5629aa6f1370 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17c80bdde0 con 0x5629aa6f1370 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17e7184740 10 client.25769 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629aa3290f0 con 0x5629aa6f1370 2024-04-02T17:14:12.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c80be4e0 con 0x5629aa6f1370 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.061+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17c803c030 con 0x5629aa6f1370 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 10 client.25769 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 20 client.25769 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa7061e0 0x5629aa7085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 4 client.25769 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629aa5f6630 con 0x5629aa7061e0 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.062+0000 7f17e1a00640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa7061e0 0x5629aa7085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.063+0000 7f17e1a00640 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa7061e0 0x5629aa7085d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28907de6aa320509 server_cookie=5a03246e208e765f in_seq=0 out_seq=0 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.063+0000 7f17c3400640 10 client.25769 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.063+0000 7f17c3400640 1 -- 172.21.15.59:0/141134013 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629aa5f6630 con 0x5629aa7061e0 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.063+0000 7f17c3400640 10 client.25769 handle_command_reply: tid=0 2024-04-02T17:14:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.063+0000 7f17e7184740 1 client.25769 shutdown 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 msgr2=0x5629aa6fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009780 tx=0x7f17c80f51a0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 shutdown_connections 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa7061e0 0x5629aa7085d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f64e0 0x5629aa702110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f1370 0x5629aa6fb0d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 --2- 172.21.15.59:0/141134013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f0980 0x5629aa6fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 >> 172.21.15.59:0/141134013 conn(0x5629aa6ebef0 msgr2=0x5629aa6ed140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17c3400640 0 client.25769 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 shutdown_connections 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 1 -- 172.21.15.59:0/141134013 wait complete. 2024-04-02T17:14:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.064+0000 7f17e7184740 20 client.25769 trim_cache size 0 max 16384 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.101+0000 7f17e7184740 1 Processor -- start 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.101+0000 7f17e7184740 1 -- start start 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa7020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1af590 con 0x5629aa6f64e0 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b10d0 con 0x5629aa6f0980 2024-04-02T17:14:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e7184740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1afc60 con 0x5629aa6f1370 2024-04-02T17:14:12.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32820/0 (socket says 172.21.15.59:32820) 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa7020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e2400640 1 -- 172.21.15.59:0/3312866815 learned_addr learned my addr 172.21.15.59:0/3312866815 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.102+0000 7f17e1a00640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 -- 172.21.15.59:0/3312866815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 msgr2=0x5629aa6ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 -- 172.21.15.59:0/3312866815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 msgr2=0x5629aa6eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 -- 172.21.15.59:0/3312866815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aa6f3020 con 0x5629aa6f1370 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e2400640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1a00640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.103+0000 7f17e1000640 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa7020d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17c80bc090 tx=0x7f17c802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88f69835c75e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.104+0000 7f17ce000640 1 -- 172.21.15.59:0/3312866815 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80c1070 con 0x5629aa6f1370 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.104+0000 7f17ce000640 1 -- 172.21.15.59:0/3312866815 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c80bc0e0 con 0x5629aa6f1370 2024-04-02T17:14:12.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.104+0000 7f17ce000640 1 -- 172.21.15.59:0/3312866815 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c80c2190 con 0x5629aa6f1370 2024-04-02T17:14:12.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- 172.21.15.59:0/3312866815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 msgr2=0x5629aa7020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa7020d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17c80bc090 tx=0x7f17c802bb70 comp rx=0 tx=0).stop 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- 172.21.15.59:0/3312866815 shutdown_connections 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f64e0 0x5629aa6eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f1370 0x5629aa7020d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 --2- 172.21.15.59:0/3312866815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f0980 0x5629aa6ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- 172.21.15.59:0/3312866815 >> 172.21.15.59:0/3312866815 conn(0x5629aa6ebef0 msgr2=0x5629aa6f95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- 172.21.15.59:0/3312866815 shutdown_connections 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- 172.21.15.59:0/3312866815 wait complete. 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.105+0000 7f17e7184740 1 -- start start 2024-04-02T17:14:12.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 Processor -- start 2024-04-02T17:14:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f64e0 0x5629aa700860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0a00 con 0x5629aa6f1370 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0330 con 0x5629aa6f64e0 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ad570 con 0x5629aa6f0980 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.106+0000 7f17e7184740 10 client.0 _ll_register_callbacks cb 0x7f17c80f5020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46260/0 (socket says 172.21.15.59:46260) 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 -- 172.21.15.59:0/1724920515 learned_addr learned my addr 172.21.15.59:0/1724920515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32832/0 (socket says 172.21.15.59:32832) 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e2400640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f64e0 0x5629aa700860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 -- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f64e0 msgr2=0x5629aa700860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f64e0 0x5629aa700860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 -- 172.21.15.59:0/1724920515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 msgr2=0x5629aa6ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1a00640 1 -- 172.21.15.59:0/1724920515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aa700e50 con 0x5629aa6f0980 2024-04-02T17:14:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.107+0000 7f17e1000640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.108+0000 7f17e1a00640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5629aa6f1de0 tx=0x7f17d00044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483b19f51f5f3082 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.108+0000 7f17c3400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.108+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d001c310 con 0x5629aa6f0980 2024-04-02T17:14:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.108+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f17d001c4b0 con 0x5629aa6f0980 2024-04-02T17:14:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.108+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629aa662cb0 con 0x5629aa6f0980 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17d001c6b0 con 0x5629aa6f0980 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17e7184740 10 client.25804 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629aa702230 con 0x5629aa6f0980 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629aa662cb0 con 0x5629aa6f0980 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17d002e4b0 con 0x5629aa6f0980 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17e7184740 10 client.25804 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17e7184740 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.109+0000 7f17e7184740 20 client.25804 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.110+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa76cca0 0x5629aa76f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.110+0000 7f17e7184740 4 client.25804 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.110+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629aa65d000 con 0x5629aa76cca0 2024-04-02T17:14:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.110+0000 7f17e1000640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa76cca0 0x5629aa76f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.111+0000 7f17e1000640 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa76cca0 0x5629aa76f0c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255025eccc93a56f server_cookie=40220e3af72154ca in_seq=0 out_seq=0 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.111+0000 7f17c3400640 10 client.25804 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.111+0000 7f17c3400640 1 -- 172.21.15.59:0/1724920515 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5629aa65d000 con 0x5629aa76cca0 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.111+0000 7f17c3400640 10 client.25804 handle_command_reply: tid=0 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.111+0000 7f17e7184740 1 client.25804 shutdown 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 msgr2=0x5629aa6ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5629aa6f1de0 tx=0x7f17d00044d0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 shutdown_connections 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5629aa76cca0 0x5629aa76f0c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17c3400640 0 client.25804 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5629aa6f1370 0x5629aa6ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629aa6f64e0 0x5629aa700860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 --2- 172.21.15.59:0/1724920515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629aa6f0980 0x5629aa6ff310 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 >> 172.21.15.59:0/1724920515 conn(0x5629aa6ebef0 msgr2=0x5629aa6ef150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 shutdown_connections 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 1 -- 172.21.15.59:0/1724920515 wait complete. 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.112+0000 7f17e7184740 20 client.25804 trim_cache size 0 max 16384 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout: "b86b2dc5-faaa-4985-9b44-eb5caa7fb410": { 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-02T17:14:12.114 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "b86b2dc5-faaa-4985-9b44-eb5caa7fb410", 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.113+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 msgr2=0x7f17c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.113+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 0x7f17c40a60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5629aa6f1bd0 tx=0x7f17d4004a40 comp rx=0 tx=0).stop 2024-04-02T17:14:12.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.113+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 msgr2=0x7f17dc19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.113+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc19e850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17c802c0b0 tx=0x7f17c8032430 comp rx=0 tx=0).stop 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 shutdown_connections 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc1098c0 0x7f17dc1a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc075b40 0x7f17dc19e850 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f17c40a3c30 0x7f17c40a60f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 --2- 172.21.15.59:0/3831627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc074560 0x7f17dc19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 >> 172.21.15.59:0/3831627252 conn(0x7f17dc0fcf70 msgr2=0x7f17dc1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 shutdown_connections 2024-04-02T17:14:12.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:12.114+0000 7f17e2e00640 1 -- 172.21.15.59:0/3831627252 wait complete. 2024-04-02T17:14:12.138 INFO:tasks.cephfs.filesystem:scrub status for tag:b86b2dc5-faaa-4985-9b44-eb5caa7fb410 - {'path': '/', 'tag': 'b86b2dc5-faaa-4985-9b44-eb5caa7fb410', 'options': 'recursive,force'} 2024-04-02T17:14:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:12 smithi059 ceph-mon[31664]: scrub queued for path: / 2024-04-02T17:14:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:12 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:12 smithi059 ceph-mon[31664]: scrub summary: active paths [/] 2024-04-02T17:14:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:12 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:12 smithi164 ceph-mon[36499]: scrub queued for path: / 2024-04-02T17:14:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:12 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:12 smithi164 ceph-mon[36499]: scrub summary: active paths [/] 2024-04-02T17:14:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:12 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:12.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:12 smithi154 ceph-mon[36479]: scrub queued for path: / 2024-04-02T17:14:12.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:12 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:12.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:12 smithi154 ceph-mon[36479]: scrub summary: active paths [/] 2024-04-02T17:14:12.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:12 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:13 smithi059 ceph-mon[31664]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:14:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:13 smithi164 ceph-mon[36499]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:14:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:13 smithi154 ceph-mon[36479]: pgmap v422: 97 pgs: 97 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T17:14:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:15 smithi059 ceph-mon[31664]: pgmap v423: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:15 smithi059 ceph-mon[31664]: scrub summary: idle 2024-04-02T17:14:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:15 smithi164 ceph-mon[36499]: pgmap v423: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:15 smithi164 ceph-mon[36499]: scrub summary: idle 2024-04-02T17:14:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:15 smithi154 ceph-mon[36479]: pgmap v423: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:15 smithi154 ceph-mon[36479]: scrub summary: idle 2024-04-02T17:14:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:17 smithi059 ceph-mon[31664]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:17 smithi164 ceph-mon[36499]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:17 smithi154 ceph-mon[36479]: pgmap v424: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:19 smithi059 ceph-mon[31664]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:19 smithi164 ceph-mon[36499]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:19 smithi154 ceph-mon[36479]: pgmap v425: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T17:14:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:21 smithi059 ceph-mon[31664]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:21 smithi164 ceph-mon[36499]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:21 smithi154 ceph-mon[36479]: pgmap v426: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 -- 172.21.15.59:0/2036118332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff630075ac0 msgr2=0x7ff630073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 --2- 172.21.15.59:0/2036118332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff630075ac0 0x7ff630073f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff62002bcd0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 -- 172.21.15.59:0/2036118332 shutdown_connections 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 --2- 172.21.15.59:0/2036118332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630109830 0x7ff63010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 --2- 172.21.15.59:0/2036118332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300744e0 0x7ff630074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 --2- 172.21.15.59:0/2036118332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff630075ac0 0x7ff630073f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 -- 172.21.15.59:0/2036118332 >> 172.21.15.59:0/2036118332 conn(0x7ff6300fcf10 msgr2=0x7ff6300ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 -- 172.21.15.59:0/2036118332 shutdown_connections 2024-04-02T17:14:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.245+0000 7ff62fe00640 1 -- 172.21.15.59:0/2036118332 wait complete. 2024-04-02T17:14:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.246+0000 7ff62fe00640 1 Processor -- start 2024-04-02T17:14:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.246+0000 7ff62fe00640 1 -- start start 2024-04-02T17:14:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630075ac0 0x7ff63019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff630109830 0x7ff6301a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010f830 con 0x7ff6300744e0 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010f6b0 con 0x7ff630075ac0 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010f9b0 con 0x7ff630109830 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38214/0 (socket says 172.21.15.59:38214) 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62ea00640 1 -- 172.21.15.59:0/398599337 learned_addr learned my addr 172.21.15.59:0/398599337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62e000640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630075ac0 0x7ff63019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.247+0000 7ff62f400640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff630109830 0x7ff6301a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 -- 172.21.15.59:0/398599337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff630109830 msgr2=0x7ff6301a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff630109830 0x7ff6301a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 -- 172.21.15.59:0/398599337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630075ac0 msgr2=0x7ff63019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630075ac0 0x7ff63019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 -- 172.21.15.59:0/398599337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301a1aa0 con 0x7ff6300744e0 2024-04-02T17:14:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.248+0000 7ff62ea00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff620000c00 tx=0x7ff62002c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.249+0000 7ff627400640 1 -- 172.21.15.59:0/398599337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620043020 con 0x7ff6300744e0 2024-04-02T17:14:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.249+0000 7ff627400640 1 -- 172.21.15.59:0/398599337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6200054e0 con 0x7ff6300744e0 2024-04-02T17:14:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.249+0000 7ff627400640 1 -- 172.21.15.59:0/398599337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620003c20 con 0x7ff6300744e0 2024-04-02T17:14:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.249+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301a1d30 con 0x7ff6300744e0 2024-04-02T17:14:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.249+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301a21d0 con 0x7ff6300744e0 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.252+0000 7ff627400640 1 -- 172.21.15.59:0/398599337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff620046020 con 0x7ff6300744e0 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.252+0000 7ff638846740 1 Processor -- start 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 -- start start 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb228c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb228cc60 0x55edb228f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55edb21ecd70 con 0x55edb228cc60 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55edb21952a0 con 0x55edb2287af0 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.253+0000 7ff638846740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edb21e8d20 con 0x55edb2287100 2024-04-02T17:14:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39228/0 (socket says 172.21.15.59:39228) 2024-04-02T17:14:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 -- 172.21.15.59:0/529892219 learned_addr learned my addr 172.21.15.59:0/529892219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62ea00640 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb228c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 -- 172.21.15.59:0/529892219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 msgr2=0x55edb228c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb228c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 -- 172.21.15.59:0/529892219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb228cc60 msgr2=0x55edb228f0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb228cc60 0x55edb228f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff62f400640 1 -- 172.21.15.59:0/529892219 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb21a9840 con 0x55edb2287100 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.254+0000 7ff627400640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 0x7ff6140a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff627400640 1 -- 172.21.15.59:0/398599337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff6200f2520 con 0x7ff6300744e0 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff62f400640 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff628009fd0 tx=0x7ff6280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4815239da78391c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff62ea00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 0x7ff6140a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff61be00640 1 -- 172.21.15.59:0/529892219 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628004d20 con 0x55edb2287100 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff61be00640 1 -- 172.21.15.59:0/529892219 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628004050 con 0x55edb2287100 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.255+0000 7ff61be00640 1 -- 172.21.15.59:0/529892219 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628005260 con 0x55edb2287100 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 -- 172.21.15.59:0/529892219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 msgr2=0x55edb2287520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff62ea00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 0x7ff6140a60e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55edb2287ef0 tx=0x7ff6200f5f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff628009fd0 tx=0x7ff6280118d0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 -- 172.21.15.59:0/529892219 shutdown_connections 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb228cc60 0x55edb228f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb228c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 --2- 172.21.15.59:0/529892219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287100 0x55edb2287520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 -- 172.21.15.59:0/529892219 >> 172.21.15.59:0/529892219 conn(0x55edb2282670 msgr2=0x55edb2282a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 -- 172.21.15.59:0/529892219 shutdown_connections 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.256+0000 7ff638846740 1 -- 172.21.15.59:0/529892219 wait complete. 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.257+0000 7ff638846740 1 -- start start 2024-04-02T17:14:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.257+0000 7ff638846740 1 Processor -- start 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.257+0000 7ff638846740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.257+0000 7ff638846740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 0x55edb22a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff638846740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb228cc60 0x55edb22a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff638846740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55edb22a0c20 con 0x55edb2287100 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff638846740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55edb21a8660 con 0x55edb228cc60 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff638846740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edb22a0da0 con 0x55edb2287af0 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff638846740 10 client.0 _ll_register_callbacks cb 0x55edb21a8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff62e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 0x55edb22a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff62f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff62f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38222/0 (socket says 172.21.15.59:38222) 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff62f400640 1 -- 172.21.15.59:0/2102382153 learned_addr learned my addr 172.21.15.59:0/2102382153 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.258+0000 7ff62e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 0x55edb22a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39232/0 (socket says 172.21.15.59:39232) 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 -- 172.21.15.59:0/2102382153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 msgr2=0x55edb22a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 0x55edb22a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 -- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb228cc60 msgr2=0x55edb22a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb228cc60 0x55edb22a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 -- 172.21.15.59:0/2102382153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb21b7e80 con 0x55edb2287100 2024-04-02T17:14:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff62f400640 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff62801b040 tx=0x7ff628004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b00fdb8b3594e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff618c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628028020 con 0x55edb2287100 2024-04-02T17:14:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.259+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628004050 con 0x55edb2287100 2024-04-02T17:14:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff628004610 con 0x55edb2287100 2024-04-02T17:14:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff62800f4e0 con 0x55edb2287100 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff638846740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55edb1f01210 con 0x55edb2287100 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628004050 con 0x55edb2287100 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.260+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff628005460 con 0x55edb2287100 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 20 client.15696 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb22ae2a0 0x55edb22b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 4 client.15696 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb219ccd0 con 0x55edb22ae2a0 2024-04-02T17:14:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.261+0000 7ff62e000640 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb22ae2a0 0x55edb22b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.262+0000 7ff62e000640 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb22ae2a0 0x55edb22b06e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64da9b0384854935 server_cookie=4c23c93e52547c44 in_seq=0 out_seq=0 2024-04-02T17:14:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.262+0000 7ff618c00640 10 client.15696 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff618c00640 1 -- 172.21.15.59:0/2102382153 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edb219ccd0 con 0x55edb22ae2a0 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff618c00640 10 client.15696 handle_command_reply: tid=0 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 client.15696 shutdown 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 msgr2=0x55edb22a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff62801b040 tx=0x7ff628004db0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 shutdown_connections 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb22ae2a0 0x55edb22b06e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb228cc60 0x55edb22a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb2287af0 0x55edb22a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff618c00640 0 client.15696 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 --2- 172.21.15.59:0/2102382153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a26b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 >> 172.21.15.59:0/2102382153 conn(0x55edb2282670 msgr2=0x55edb2281870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.263+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 shutdown_connections 2024-04-02T17:14:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.264+0000 7ff638846740 1 -- 172.21.15.59:0/2102382153 wait complete. 2024-04-02T17:14:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.264+0000 7ff638846740 20 client.15696 trim_cache size 0 max 16384 2024-04-02T17:14:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.293+0000 7ff638846740 1 Processor -- start 2024-04-02T17:14:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 -- start start 2024-04-02T17:14:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb22a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb2280fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301a2a00 con 0x55edb2287100 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301a4540 con 0x55edb2287af0 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff638846740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301a30d0 con 0x55edb228cc60 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff62f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff62f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38228/0 (socket says 172.21.15.59:38228) 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.294+0000 7ff62f400640 1 -- 172.21.15.59:0/3925249382 learned_addr learned my addr 172.21.15.59:0/3925249382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb22a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62f400640 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb2280fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 -- 172.21.15.59:0/3925249382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 msgr2=0x55edb2280fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb2280fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 -- 172.21.15.59:0/3925249382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 msgr2=0x55edb22a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 -- 172.21.15.59:0/3925249382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb1fcd620 con 0x55edb2287af0 2024-04-02T17:14:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.295+0000 7ff62e000640 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb22a9bc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55edb22b0ef0 tx=0x7ff61c014840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352e5d141908dda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.296+0000 7ff61b400640 1 -- 172.21.15.59:0/3925249382 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c005030 con 0x55edb2287af0 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.296+0000 7ff61b400640 1 -- 172.21.15.59:0/3925249382 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c004730 con 0x55edb2287af0 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.296+0000 7ff61b400640 1 -- 172.21.15.59:0/3925249382 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c01e2b0 con 0x55edb2287af0 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- 172.21.15.59:0/3925249382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 msgr2=0x55edb22a9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb22a9bc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55edb22b0ef0 tx=0x7ff61c014840 comp rx=0 tx=0).stop 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- 172.21.15.59:0/3925249382 shutdown_connections 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb2280fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287af0 0x55edb22a9bc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 --2- 172.21.15.59:0/3925249382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287100 0x55edb22a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- 172.21.15.59:0/3925249382 >> 172.21.15.59:0/3925249382 conn(0x55edb2282670 msgr2=0x55edb22a22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- 172.21.15.59:0/3925249382 shutdown_connections 2024-04-02T17:14:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- 172.21.15.59:0/3925249382 wait complete. 2024-04-02T17:14:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.297+0000 7ff638846740 1 -- start start 2024-04-02T17:14:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 Processor -- start 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 0x55edb22a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb22a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301a3e70 con 0x55edb2287af0 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff6301a37a0 con 0x55edb2287100 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff630072370 con 0x55edb228cc60 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.298+0000 7ff638846740 10 client.0 _ll_register_callbacks cb 0x7ff628004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 0x55edb22a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55368/0 (socket says 172.21.15.59:55368) 2024-04-02T17:14:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 -- 172.21.15.59:0/1964305867 learned_addr learned my addr 172.21.15.59:0/1964305867 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb22a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 -- 172.21.15.59:0/1964305867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 msgr2=0x55edb22a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb22a82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 -- 172.21.15.59:0/1964305867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 msgr2=0x55edb22a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 0x55edb22a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62f400640 1 -- 172.21.15.59:0/1964305867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb22a88e0 con 0x55edb2287100 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.299+0000 7ff62e000640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 0x55edb22a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff62f400640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff628005a20 tx=0x7ff628009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2335fee2cf0a3c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff618c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628028040 con 0x55edb2287100 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff628004000 con 0x55edb2287100 2024-04-02T17:14:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff61c004730 con 0x55edb2287100 2024-04-02T17:14:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff628005aa0 con 0x55edb2287100 2024-04-02T17:14:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff638846740 10 client.25790 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.300+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55edb22a9bf0 con 0x55edb2287100 2024-04-02T17:14:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.301+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c004730 con 0x55edb2287100 2024-04-02T17:14:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.301+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff62800f680 con 0x55edb2287100 2024-04-02T17:14:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 20 client.25790 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb2314b90 0x55edb2316fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 4 client.25790 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edb21fa2b0 con 0x55edb2314b90 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.302+0000 7ff62e000640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb2314b90 0x55edb2316fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.304+0000 7ff62e000640 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb2314b90 0x55edb2316fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676f9117fd80eb48 server_cookie=5c3086957ae6cce0 in_seq=0 out_seq=0 2024-04-02T17:14:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.304+0000 7ff618c00640 10 client.25790 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.310+0000 7ff618c00640 1 -- 172.21.15.59:0/1964305867 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edb21fa2b0 con 0x55edb2314b90 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.310+0000 7ff618c00640 10 client.25790 handle_command_reply: tid=0 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.310+0000 7ff638846740 1 client.25790 shutdown 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 msgr2=0x55edb22a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff628005a20 tx=0x7ff628009500 comp rx=0 tx=0).stop 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 shutdown_connections 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55edb2314b90 0x55edb2316fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb2287af0 0x55edb22a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edb228cc60 0x55edb22a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 --2- 172.21.15.59:0/1964305867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55edb2287100 0x55edb22a6da0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff618c00640 0 client.25790 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 >> 172.21.15.59:0/1964305867 conn(0x55edb2282670 msgr2=0x55edb22858d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 shutdown_connections 2024-04-02T17:14:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 1 -- 172.21.15.59:0/1964305867 wait complete. 2024-04-02T17:14:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.311+0000 7ff638846740 20 client.25790 trim_cache size 0 max 16384 2024-04-02T17:14:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:14:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:14:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:14:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:14:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 msgr2=0x7ff6140a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 0x7ff6140a60e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55edb2287ef0 tx=0x7ff6200f5f60 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 msgr2=0x7ff630199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff620000c00 tx=0x7ff62002c6f0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 shutdown_connections 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff630109830 0x7ff6301a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff630075ac0 0x7ff63019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff6140a3c20 0x7ff6140a60e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 --2- 172.21.15.59:0/398599337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6300744e0 0x7ff630199e80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.312+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 >> 172.21.15.59:0/398599337 conn(0x7ff6300fcf10 msgr2=0x7ff6301057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.313+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 shutdown_connections 2024-04-02T17:14:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:23.313+0000 7ff62fe00640 1 -- 172.21.15.59:0/398599337 wait complete. 2024-04-02T17:14:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:23 smithi059 ceph-mon[31664]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:23 smithi164 ceph-mon[36499]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:23 smithi154 ceph-mon[36479]: pgmap v427: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T17:14:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:25 smithi059 ceph-mon[31664]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:25 smithi164 ceph-mon[36499]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:25.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:25 smithi154 ceph-mon[36479]: pgmap v428: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T17:14:27.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:27 smithi059 ceph-mon[31664]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-02T17:14:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:27 smithi164 ceph-mon[36499]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-02T17:14:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:27 smithi154 ceph-mon[36479]: pgmap v429: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-02T17:14:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:29 smithi059 ceph-mon[31664]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:14:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:29 smithi164 ceph-mon[36499]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:14:29.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:29 smithi154 ceph-mon[36479]: pgmap v430: 97 pgs: 97 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T17:14:30.982 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 61% (7591/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:Note: switching to '191d963ac9de7322da8aff20b567b3fa458e2ef6'. 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.429 INFO:tasks.workunit.client.0.smithi059.stderr: git switch -c 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr:Or undo this operation with: 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr: git switch - 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-04-02T17:14:31.430 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at 191d963ac9d Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 2024-04-02T17:14:31.442 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T17:14:31.509 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T17:14:31.517 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T17:14:31.517 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T17:14:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:31 smithi059 ceph-mon[31664]: pgmap v431: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:31 smithi164 ceph-mon[36499]: pgmap v431: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:31 smithi154 ceph-mon[36479]: pgmap v431: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:32.449 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T17:14:32.581 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T17:14:32.674 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T17:14:32.680 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T17:14:32.680 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T17:14:32.768 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T17:14:32.779 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:14:32.779 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T17:14:32.833 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-02T17:14:32.834 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-02T17:14:32.834 DEBUG:teuthology.orchestra.run.smithi059:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-02T17:14:32.898 INFO:tasks.workunit.client.0.smithi059.stderr:+ dbench 1 2024-04-02T17:14:32.917 INFO:tasks.workunit.client.0.smithi059.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-02T17:14:32.917 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:14:32.917 INFO:tasks.workunit.client.0.smithi059.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-02T17:14:32.927 INFO:tasks.workunit.client.0.smithi059.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-02T17:14:33.028 INFO:tasks.workunit.client.0.smithi059.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-02T17:14:33.028 INFO:tasks.workunit.client.0.smithi059.stdout:releasing clients 2024-04-02T17:14:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:33 smithi059 ceph-mon[31664]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T17:14:33.575 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:33 smithi164 ceph-mon[36499]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T17:14:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:33 smithi154 ceph-mon[36479]: pgmap v432: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T17:14:34.028 INFO:tasks.workunit.client.0.smithi059.stdout: 1 1653 26.20 MB/sec warmup 1 sec latency 13.367 ms 2024-04-02T17:14:35.029 INFO:tasks.workunit.client.0.smithi059.stdout: 1 3926 20.03 MB/sec warmup 2 sec latency 11.170 ms 2024-04-02T17:14:35.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:35 smithi059 ceph-mon[31664]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:35 smithi164 ceph-mon[36499]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:35 smithi154 ceph-mon[36479]: pgmap v433: 97 pgs: 97 active+clean; 672 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-02T17:14:36.029 INFO:tasks.workunit.client.0.smithi059.stdout: 1 5855 15.62 MB/sec warmup 3 sec latency 13.319 ms 2024-04-02T17:14:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 -- 172.21.15.59:0/3775125841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 msgr2=0x7fa1a4109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 --2- 172.21.15.59:0/3775125841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4109c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ca0 tx=0x7fa19c02c100 comp rx=0 tx=0).stop 2024-04-02T17:14:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 -- 172.21.15.59:0/3775125841 shutdown_connections 2024-04-02T17:14:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 --2- 172.21.15.59:0/3775125841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4109c50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 --2- 172.21.15.59:0/3775125841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 0x7fa1a41072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 --2- 172.21.15.59:0/3775125841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 0x7fa1a4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.471+0000 7fa1aa000640 1 -- 172.21.15.59:0/3775125841 >> 172.21.15.59:0/3775125841 conn(0x7fa1a40fbf90 msgr2=0x7fa1a40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.472+0000 7fa1aa000640 1 -- 172.21.15.59:0/3775125841 shutdown_connections 2024-04-02T17:14:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.472+0000 7fa1aa000640 1 -- 172.21.15.59:0/3775125841 wait complete. 2024-04-02T17:14:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 Processor -- start 2024-04-02T17:14:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 -- start start 2024-04-02T17:14:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 0x7fa1a4101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 0x7fa1a4100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410d7c0 con 0x7fa1a41025a0 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410d640 con 0x7fa1a4107800 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1aa000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410d940 con 0x7fa1a4104ed0 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1a3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 0x7fa1a4101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1a3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34252/0 (socket says 172.21.15.59:34252) 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 0x7fa1a4100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.473+0000 7fa1a3e00640 1 -- 172.21.15.59:0/699898476 learned_addr learned my addr 172.21.15.59:0/699898476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 -- 172.21.15.59:0/699898476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 msgr2=0x7fa1a4100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 0x7fa1a4100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 -- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 msgr2=0x7fa1a4101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 0x7fa1a4101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 -- 172.21.15.59:0/699898476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4100fd0 con 0x7fa1a4107800 2024-04-02T17:14:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02c640 tx=0x7fa19c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.474+0000 7fa19be00640 1 -- 172.21.15.59:0/699898476 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c039040 con 0x7fa1a4107800 2024-04-02T17:14:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.475+0000 7fa19be00640 1 -- 172.21.15.59:0/699898476 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c044070 con 0x7fa1a4107800 2024-04-02T17:14:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.475+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a4101260 con 0x7fa1a4107800 2024-04-02T17:14:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.475+0000 7fa19be00640 1 -- 172.21.15.59:0/699898476 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0043c0 con 0x7fa1a4107800 2024-04-02T17:14:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.475+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a6860 con 0x7fa1a4107800 2024-04-02T17:14:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.478+0000 7fa19be00640 1 -- 172.21.15.59:0/699898476 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa19c004560 con 0x7fa1a4107800 2024-04-02T17:14:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.478+0000 7fa1acc37740 1 Processor -- start 2024-04-02T17:14:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.478+0000 7fa1acc37740 1 -- start start 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.478+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e8085f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.478+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 0x55e8085fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1acc37740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e80855ad70 con 0x55e8085f4ed0 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1acc37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e8085032a0 con 0x55e8085faa30 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1acc37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e808556d20 con 0x55e8085f58c0 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51128/0 (socket says 172.21.15.59:51128) 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 -- 172.21.15.59:0/1004299248 learned_addr learned my addr 172.21.15.59:0/1004299248 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e8085f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 0x55e8085fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 -- 172.21.15.59:0/1004299248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 msgr2=0x55e8085fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 0x55e8085fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 -- 172.21.15.59:0/1004299248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 msgr2=0x55e8085f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e8085f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a3400640 1 -- 172.21.15.59:0/1004299248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e808517840 con 0x55e8085f58c0 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.479+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 0x55e8085fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa1a3400640 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa194009cf0 tx=0x7fa194011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e66e9f0d391303 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa18fe00640 1 -- 172.21.15.59:0/1004299248 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194004230 con 0x55e8085f58c0 2024-04-02T17:14:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa18fe00640 1 -- 172.21.15.59:0/1004299248 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1940043d0 con 0x55e8085f58c0 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa18fe00640 1 -- 172.21.15.59:0/1004299248 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194019960 con 0x55e8085f58c0 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa19be00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 0x7fa1880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.480+0000 7fa19be00640 1 -- 172.21.15.59:0/699898476 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa19c0f27e0 con 0x7fa1a4107800 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 0x7fa1880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 -- 172.21.15.59:0/1004299248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 msgr2=0x55e8085fa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa194009cf0 tx=0x7fa194011810 comp rx=0 tx=0).stop 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 -- 172.21.15.59:0/1004299248 shutdown_connections 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085faa30 0x55e8085fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085f58c0 0x55e8085fa3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 --2- 172.21.15.59:0/1004299248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e8085f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 -- 172.21.15.59:0/1004299248 >> 172.21.15.59:0/1004299248 conn(0x55e8085f0440 msgr2=0x55e8085f0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 -- 172.21.15.59:0/1004299248 shutdown_connections 2024-04-02T17:14:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1acc37740 1 -- 172.21.15.59:0/1004299248 wait complete. 2024-04-02T17:14:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.481+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 0x7fa1880a6190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1900058c0 tx=0x7fa190015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 -- start start 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 Processor -- start 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e8085ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e8085ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808606990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e8085fd9e0 con 0x55e8085f58c0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e808516660 con 0x55e8085f4ed0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e8085fdb60 con 0x55e8085faa30 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.482+0000 7fa1acc37740 10 client.0 _ll_register_callbacks cb 0x55e808516050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e8085ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808606990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e8085ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34264/0 (socket says 172.21.15.59:34264) 2024-04-02T17:14:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3400640 1 -- 172.21.15.59:0/2215428763 learned_addr learned my addr 172.21.15.59:0/2215428763 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a2a00640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e8085ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 msgr2=0x55e8085ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e8085ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2215428763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 msgr2=0x55e8085ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e8085ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2215428763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e808525e80 con 0x55e8085faa30 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808606990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0f0220 tx=0x7fa19c0be600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89a98abd8d29e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.483+0000 7fa18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0bee60 con 0x55e8085faa30 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa19c0c71e0 con 0x55e8085faa30 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1940043d0 con 0x55e8085faa30 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa19c032890 con 0x55e8085faa30 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa1acc37740 10 client.25834 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e80826f210 con 0x55e8085faa30 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1940043d0 con 0x55e8085faa30 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.484+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa19c03c030 con 0x55e8085faa30 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 10 client.25834 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 20 client.25834 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e80860b220 0x55e80860d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 4 client.25834 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80850acd0 con 0x55e80860b220 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1a3400640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e80860b220 0x55e80860d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa1a3400640 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e80860b220 0x55e80860d6b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88bd6920f38e9e58 server_cookie=4b850f38eb41e58a in_seq=0 out_seq=0 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.485+0000 7fa18cc00640 10 client.25834 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa18cc00640 1 -- 172.21.15.59:0/2215428763 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e80850acd0 con 0x55e80860b220 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa18cc00640 10 client.25834 handle_command_reply: tid=0 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 client.25834 shutdown 2024-04-02T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 msgr2=0x55e808606990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808606990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0f0220 tx=0x7fa19c0be600 comp rx=0 tx=0).stop 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 shutdown_connections 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e80860b220 0x55e80860d6b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e8085ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808606990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa18cc00640 0 client.25834 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2215428763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e8085ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 >> 172.21.15.59:0/2215428763 conn(0x55e8085f0440 msgr2=0x55e8085ef640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.487+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 shutdown_connections 2024-04-02T17:14:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.488+0000 7fa1acc37740 1 -- 172.21.15.59:0/2215428763 wait complete. 2024-04-02T17:14:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.488+0000 7fa1acc37740 20 client.25834 trim_cache size 0 max 16384 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 Processor -- start 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 -- start start 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e808606950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e8086124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41af7b0 con 0x55e8085f58c0 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b12f0 con 0x55e8085f4ed0 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.506+0000 7fa1acc37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41afe80 con 0x55e8085faa30 2024-04-02T17:14:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e808606950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34276/0 (socket says 172.21.15.59:34276) 2024-04-02T17:14:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 -- 172.21.15.59:0/478230833 learned_addr learned my addr 172.21.15.59:0/478230833 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e8086124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 -- 172.21.15.59:0/478230833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 msgr2=0x55e8086124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e8086124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 -- 172.21.15.59:0/478230833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 msgr2=0x55e808606950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e808606950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3400640 1 -- 172.21.15.59:0/478230833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80833b620 con 0x55e8085f4ed0 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.507+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e8086124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.508+0000 7fa1a3400640 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1940058c0 tx=0x7fa194005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24efb15f554a10b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.508+0000 7fa18f400640 1 -- 172.21.15.59:0/478230833 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194031070 con 0x55e8085f4ed0 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.508+0000 7fa18f400640 1 -- 172.21.15.59:0/478230833 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19402bd90 con 0x55e8085f4ed0 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.508+0000 7fa18f400640 1 -- 172.21.15.59:0/478230833 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19402b830 con 0x55e8085f4ed0 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- 172.21.15.59:0/478230833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 msgr2=0x55e808611f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1940058c0 tx=0x7fa194005850 comp rx=0 tx=0).stop 2024-04-02T17:14:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- 172.21.15.59:0/478230833 shutdown_connections 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f58c0 0x55e808606950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e8086124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 --2- 172.21.15.59:0/478230833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f4ed0 0x55e808611f80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- 172.21.15.59:0/478230833 >> 172.21.15.59:0/478230833 conn(0x55e8085f0440 msgr2=0x55e8085f36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- 172.21.15.59:0/478230833 shutdown_connections 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- 172.21.15.59:0/478230833 wait complete. 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.509+0000 7fa1acc37740 1 -- start start 2024-04-02T17:14:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 Processor -- start 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e808604440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f58c0 0x55e808604980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b0c20 con 0x55e8085f4ed0 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b0550 con 0x55e8085f58c0 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ad790 con 0x55e8085faa30 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1acc37740 10 client.0 _ll_register_callbacks cb 0x7fa19c0bad90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51160/0 (socket says 172.21.15.59:51160) 2024-04-02T17:14:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2306589900 learned_addr learned my addr 172.21.15.59:0/2306589900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.510+0000 7fa1a2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f58c0 0x55e808604980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3400640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e808604440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f58c0 msgr2=0x55e808604980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f58c0 0x55e808604980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2306589900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 msgr2=0x55e808604440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e808604440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 -- 172.21.15.59:0/2306589900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e808605f80 con 0x55e8085faa30 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1a3e00640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0be990 tx=0x7fa19c0bcf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db0ce348ea3279e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004d90 con 0x55e8085faa30 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa19c0bde50 con 0x55e8085faa30 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.511+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19402bd90 con 0x55e8085faa30 2024-04-02T17:14:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa19c0bf100 con 0x55e8085faa30 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 10 client.25840 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e808671c20 con 0x55e8085faa30 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c0d9020 con 0x55e8085faa30 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa19c0a90b0 con 0x55e8085faa30 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 10 client.25840 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 20 client.25840 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e808672420 0x55e808674840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 4 client.25840 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8085682b0 con 0x55e808672420 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.512+0000 7fa1a3400640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e808672420 0x55e808674840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.513+0000 7fa1a3400640 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e808672420 0x55e808674840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93cd9420ec64ddd server_cookie=e642b69f32bfe549 in_seq=0 out_seq=0 2024-04-02T17:14:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.513+0000 7fa18cc00640 10 client.25840 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa18cc00640 1 -- 172.21.15.59:0/2306589900 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8085682b0 con 0x55e808672420 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa18cc00640 10 client.25840 handle_command_reply: tid=0 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 client.25840 shutdown 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 msgr2=0x55e808605990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0be990 tx=0x7fa19c0bcf10 comp rx=0 tx=0).stop 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 shutdown_connections 2024-04-02T17:14:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e808672420 0x55e808674840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8085faa30 0x55e808605990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa18cc00640 0 client.25840 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8085f58c0 0x55e808604980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 --2- 172.21.15.59:0/2306589900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8085f4ed0 0x55e808604440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.582+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 >> 172.21.15.59:0/2306589900 conn(0x55e8085f0440 msgr2=0x55e8085eedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.583+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 shutdown_connections 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.583+0000 7fa1acc37740 1 -- 172.21.15.59:0/2306589900 wait complete. 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.583+0000 7fa1acc37740 20 client.25840 trim_cache size 0 max 16384 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:14:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 msgr2=0x7fa1880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 0x7fa1880a6190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1900058c0 tx=0x7fa190015040 comp rx=0 tx=0).stop 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 msgr2=0x7fa1a4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02c640 tx=0x7fa19c035d10 comp rx=0 tx=0).stop 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 shutdown_connections 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1a4107800 0x7fa1a4100a90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a4104ed0 0x7fa1a4100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1880a3cd0 0x7fa1880a6190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 --2- 172.21.15.59:0/699898476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41025a0 0x7fa1a4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 >> 172.21.15.59:0/699898476 conn(0x7fa1a40fbf90 msgr2=0x7fa1a4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 shutdown_connections 2024-04-02T17:14:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:36.584+0000 7fa1aa000640 1 -- 172.21.15.59:0/699898476 wait complete. 2024-04-02T17:14:37.029 INFO:tasks.workunit.client.0.smithi059.stdout: 1 7549 14.79 MB/sec warmup 4 sec latency 14.681 ms 2024-04-02T17:14:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:37 smithi059 ceph-mon[31664]: pgmap v434: 97 pgs: 97 active+clean; 1.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-02T17:14:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:37 smithi164 ceph-mon[36499]: pgmap v434: 97 pgs: 97 active+clean; 1.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-02T17:14:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:37 smithi154 ceph-mon[36479]: pgmap v434: 97 pgs: 97 active+clean; 1.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-02T17:14:38.029 INFO:tasks.workunit.client.0.smithi059.stdout: 1 9540 13.27 MB/sec warmup 5 sec latency 15.683 ms 2024-04-02T17:14:39.029 INFO:tasks.workunit.client.0.smithi059.stdout: 1 11204 13.05 MB/sec warmup 6 sec latency 14.215 ms 2024-04-02T17:14:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:39 smithi059 ceph-mon[31664]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 122 KiB/s wr, 28 op/s 2024-04-02T17:14:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:39 smithi164 ceph-mon[36499]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 122 KiB/s wr, 28 op/s 2024-04-02T17:14:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:39 smithi154 ceph-mon[36479]: pgmap v435: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 122 KiB/s wr, 28 op/s 2024-04-02T17:14:40.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 13143 12.29 MB/sec warmup 7 sec latency 13.786 ms 2024-04-02T17:14:41.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 14748 12.18 MB/sec warmup 8 sec latency 9.505 ms 2024-04-02T17:14:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:41 smithi059 ceph-mon[31664]: pgmap v436: 97 pgs: 97 active+clean; 5.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 457 KiB/s wr, 96 op/s 2024-04-02T17:14:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:41 smithi164 ceph-mon[36499]: pgmap v436: 97 pgs: 97 active+clean; 5.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 457 KiB/s wr, 96 op/s 2024-04-02T17:14:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:41 smithi154 ceph-mon[36479]: pgmap v436: 97 pgs: 97 active+clean; 5.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 457 KiB/s wr, 96 op/s 2024-04-02T17:14:42.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 16754 11.71 MB/sec warmup 9 sec latency 18.830 ms 2024-04-02T17:14:42.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:14:42.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 -- 172.21.15.59:0/1575374441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 msgr2=0x7fe994103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 --2- 172.21.15.59:0/1575374441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe994103050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe980009860 tx=0x7fe98002bb90 comp rx=0 tx=0).stop 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 -- 172.21.15.59:0/1575374441 shutdown_connections 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 --2- 172.21.15.59:0/1575374441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe994103590 0x7fe994109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 --2- 172.21.15.59:0/1575374441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe994103050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 --2- 172.21.15.59:0/1575374441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9941019e0 0x7fe994101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 -- 172.21.15.59:0/1575374441 >> 172.21.15.59:0/1575374441 conn(0x7fe9940fcf70 msgr2=0x7fe9940ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 -- 172.21.15.59:0/1575374441 shutdown_connections 2024-04-02T17:14:42.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.380+0000 7fe993200640 1 -- 172.21.15.59:0/1575374441 wait complete. 2024-04-02T17:14:42.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.381+0000 7fe993200640 1 Processor -- start 2024-04-02T17:14:42.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 -- start start 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 0x7fe994199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe994103590 0x7fe9941a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d640 con 0x7fe9941019e0 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d4c0 con 0x7fe994102bd0 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe993200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410d7c0 con 0x7fe994103590 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 0x7fe994199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe991400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe992800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe994103590 0x7fe9941a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe991400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45046/0 (socket says 172.21.15.59:45046) 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 0x7fe994199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44662/0 (socket says 172.21.15.59:44662) 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.382+0000 7fe991400640 1 -- 172.21.15.59:0/3202222085 learned_addr learned my addr 172.21.15.59:0/3202222085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 -- 172.21.15.59:0/3202222085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe994103590 msgr2=0x7fe9941a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe994103590 0x7fe9941a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 -- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 msgr2=0x7fe994199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 0x7fe994199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 -- 172.21.15.59:0/3202222085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a1af0 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe991400640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9800349e0 tx=0x7fe980032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe98aa00640 1 -- 172.21.15.59:0/3202222085 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980041030 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe98aa00640 1 -- 172.21.15.59:0/3202222085 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98002c530 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.383+0000 7fe98aa00640 1 -- 172.21.15.59:0/3202222085 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980039040 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.384+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a1d80 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.384+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a2260 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.387+0000 7fe99b97c740 1 Processor -- start 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.387+0000 7fe98aa00640 1 -- 172.21.15.59:0/3202222085 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe980044070 con 0x7fe994102bd0 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.387+0000 7fe99b97c740 1 -- start start 2024-04-02T17:14:42.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c808e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c80b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56117c768770 con 0x56117c804370 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56117c710700 con 0x56117c8094e0 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe99b97c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56117c764720 con 0x56117c803980 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39392/0 (socket says 172.21.15.59:39392) 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 -- 172.21.15.59:0/1148323759 learned_addr learned my addr 172.21.15.59:0/1148323759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe991400640 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c80b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe991e00640 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c808e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 -- 172.21.15.59:0/1148323759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 msgr2=0x56117c80b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c80b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 -- 172.21.15.59:0/1148323759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 msgr2=0x56117c808e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c808e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.388+0000 7fe992800640 1 -- 172.21.15.59:0/1148323759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117c5467d0 con 0x56117c803980 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe992800640 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009fd0 tx=0x7fe98c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732721c74f9a9888 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe97f000640 1 -- 172.21.15.59:0/1148323759 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004d20 con 0x56117c803980 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe97f000640 1 -- 172.21.15.59:0/1148323759 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004050 con 0x56117c803980 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe97f000640 1 -- 172.21.15.59:0/1148323759 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c005260 con 0x56117c803980 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe98aa00640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 0x7fe9780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.389+0000 7fe98aa00640 1 -- 172.21.15.59:0/3202222085 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe9800f1920 con 0x7fe994102bd0 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe991e00640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 0x7fe9780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 -- 172.21.15.59:0/1148323759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 msgr2=0x56117c803da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009fd0 tx=0x7fe98c0118d0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 -- 172.21.15.59:0/1148323759 shutdown_connections 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c808e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c80b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1148323759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c803980 0x56117c803da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 -- 172.21.15.59:0/1148323759 >> 172.21.15.59:0/1148323759 conn(0x56117c7feef0 msgr2=0x56117c7ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 -- 172.21.15.59:0/1148323759 shutdown_connections 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe99b97c740 1 -- 172.21.15.59:0/1148323759 wait complete. 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.390+0000 7fe991e00640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 0x7fe9780a60f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56117c7ffe60 tx=0x7fe984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 -- start start 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 Processor -- start 2024-04-02T17:14:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 0x56117c80dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c815460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56117c80c490 con 0x56117c803980 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56117c723ac0 con 0x56117c8094e0 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56117c80c610 con 0x56117c804370 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe99b97c740 10 client.0 _ll_register_callbacks cb 0x56117c726180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 0x56117c80dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.391+0000 7fe991400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39406/0 (socket says 172.21.15.59:39406) 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 -- 172.21.15.59:0/1160895660 learned_addr learned my addr 172.21.15.59:0/1160895660 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 0x56117c80dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44688/0 (socket says 172.21.15.59:44688) 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe992800640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c815460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 -- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 msgr2=0x56117c815460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c815460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 -- 172.21.15.59:0/1160895660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 msgr2=0x56117c80dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 0x56117c80dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 -- 172.21.15.59:0/1160895660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117c47c3f0 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe991400640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9800bdad0 tx=0x7fe9800b9de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bab432d42e328b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe973400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800bd4a0 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe98002ca50 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c004050 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.392+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe98002c6e0 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 10 client.25852 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56117c43c0f0 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004050 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe98004e060 con 0x56117c804370 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 10 client.25852 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 20 client.25852 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c819920 0x56117c81bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 4 client.25852 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.393+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117c709630 con 0x56117c819920 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.394+0000 7fe991e00640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c819920 0x56117c81bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.394+0000 7fe991e00640 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c819920 0x56117c81bdc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e7fec760728600 server_cookie=c0a4f9e71b94c7de in_seq=0 out_seq=0 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.394+0000 7fe973400640 10 client.25852 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe973400640 1 -- 172.21.15.59:0/1160895660 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56117c709630 con 0x56117c819920 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe973400640 10 client.25852 handle_command_reply: tid=0 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 client.25852 shutdown 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 msgr2=0x56117c80e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9800bdad0 tx=0x7fe9800b9de0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 shutdown_connections 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c819920 0x56117c81bdc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c8094e0 0x56117c815460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c80e420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe973400640 0 client.25852 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1160895660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c803980 0x56117c80dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 >> 172.21.15.59:0/1160895660 conn(0x56117c7feef0 msgr2=0x56117c800140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.395+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 shutdown_connections 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.396+0000 7fe99b97c740 1 -- 172.21.15.59:0/1160895660 wait complete. 2024-04-02T17:14:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.396+0000 7fe99b97c740 20 client.25852 trim_cache size 0 max 16384 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.416+0000 7fe99b97c740 1 Processor -- start 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.416+0000 7fe99b97c740 1 -- start start 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c7fda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c7fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ab110 con 0x56117c8094e0 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941acc50 con 0x56117c803980 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe99b97c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ab7e0 con 0x56117c804370 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44700/0 (socket says 172.21.15.59:44700) 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 -- 172.21.15.59:0/3672251452 learned_addr learned my addr 172.21.15.59:0/3672251452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe991400640 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c7fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe991e00640 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c7fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 -- 172.21.15.59:0/3672251452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 msgr2=0x56117c7fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c7fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 -- 172.21.15.59:0/3672251452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 msgr2=0x56117c7fda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c7fda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.417+0000 7fe992800640 1 -- 172.21.15.59:0/3672251452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117c81c440 con 0x56117c8094e0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.418+0000 7fe992800640 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe98c014040 tx=0x7fe98c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8d18f8fea4e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.418+0000 7fe97e600640 1 -- 172.21.15.59:0/3672251452 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003a30 con 0x56117c8094e0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.418+0000 7fe97e600640 1 -- 172.21.15.59:0/3672251452 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004e50 con 0x56117c8094e0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.418+0000 7fe97e600640 1 -- 172.21.15.59:0/3672251452 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0045c0 con 0x56117c8094e0 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- 172.21.15.59:0/3672251452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 msgr2=0x56117c8206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe98c014040 tx=0x7fe98c005bf0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- 172.21.15.59:0/3672251452 shutdown_connections 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c8094e0 0x56117c8206c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c804370 0x56117c7fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 --2- 172.21.15.59:0/3672251452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c7fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- 172.21.15.59:0/3672251452 >> 172.21.15.59:0/3672251452 conn(0x56117c7feef0 msgr2=0x56117c8058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- 172.21.15.59:0/3672251452 shutdown_connections 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- 172.21.15.59:0/3672251452 wait complete. 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.419+0000 7fe99b97c740 1 -- start start 2024-04-02T17:14:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 Processor -- start 2024-04-02T17:14:42.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c8122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c812800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ac580 con 0x56117c804370 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941abeb0 con 0x56117c803980 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941a90f0 con 0x56117c8094e0 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe99b97c740 10 client.0 _ll_register_callbacks cb 0x7fe9800057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe992800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe991e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c8122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe992800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39422/0 (socket says 172.21.15.59:39422) 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.420+0000 7fe992800640 1 -- 172.21.15.59:0/1175140229 learned_addr learned my addr 172.21.15.59:0/1175140229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe991400640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c812800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 -- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 msgr2=0x56117c8122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c8122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 -- 172.21.15.59:0/1175140229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 msgr2=0x56117c812800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c812800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 -- 172.21.15.59:0/1175140229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117c813e00 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe992800640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c011910 tx=0x7fe98c0048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461db0a275f984fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.421+0000 7fe973400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0044e0 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe98c00f4e0 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c004e50 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe98c03e050 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe99b97c740 10 client.25858 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56117c888960 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004e50 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.422+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe98c0051a0 con 0x56117c8094e0 2024-04-02T17:14:42.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 10 client.25858 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 20 client.25858 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c889160 0x56117c88b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 4 client.25858 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56117c775cb0 con 0x56117c889160 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe991e00640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c889160 0x56117c88b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe991e00640 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c889160 0x56117c88b580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60667fec514028e server_cookie=1d06947ec1e44923 in_seq=0 out_seq=0 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe973400640 10 client.25858 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe973400640 1 -- 172.21.15.59:0/1175140229 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56117c775cb0 con 0x56117c889160 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe973400640 10 client.25858 handle_command_reply: tid=0 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.423+0000 7fe99b97c740 1 client.25858 shutdown 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 msgr2=0x56117c813810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c011910 tx=0x7fe98c0048e0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 shutdown_connections 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56117c889160 0x56117c88b580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56117c804370 0x56117c812800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56117c8094e0 0x56117c813810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 --2- 172.21.15.59:0/1175140229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56117c803980 0x56117c8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 >> 172.21.15.59:0/1175140229 conn(0x56117c7feef0 msgr2=0x56117c7fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe973400640 0 client.25858 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 shutdown_connections 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 1 -- 172.21.15.59:0/1175140229 wait complete. 2024-04-02T17:14:42.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.424+0000 7fe99b97c740 20 client.25858 trim_cache size 0 max 16384 2024-04-02T17:14:42.428 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:42.428 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-02T17:14:42.428 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-02T17:14:42.428 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 msgr2=0x7fe9780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 0x7fe9780a60f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x56117c7ffe60 tx=0x7fe984009260 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 msgr2=0x7fe99419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9800349e0 tx=0x7fe980032450 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 shutdown_connections 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe994103590 0x7fe9941a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994102bd0 0x7fe99419a410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9780a3c30 0x7fe9780a60f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 --2- 172.21.15.59:0/3202222085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9941019e0 0x7fe994199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 >> 172.21.15.59:0/3202222085 conn(0x7fe9940fcf70 msgr2=0x7fe994073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 shutdown_connections 2024-04-02T17:14:42.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.425+0000 7fe993200640 1 -- 172.21.15.59:0/3202222085 wait complete. 2024-04-02T17:14:42.444 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T17:14:42.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.669+0000 7f7d6b200640 1 -- 172.21.15.59:0/1212039256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c102b60 msgr2=0x7f7d6c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.669+0000 7f7d6b200640 1 --2- 172.21.15.59:0/1212039256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c102b60 0x7f7d6c102fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009900 tx=0x7f7d5402bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.669+0000 7f7d6b200640 1 -- 172.21.15.59:0/1212039256 shutdown_connections 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.669+0000 7f7d6b200640 1 --2- 172.21.15.59:0/1212039256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c103520 0x7f7d6c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.670+0000 7f7d6b200640 1 --2- 172.21.15.59:0/1212039256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c102b60 0x7f7d6c102fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.670+0000 7f7d6b200640 1 --2- 172.21.15.59:0/1212039256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 0x7f7d6c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.670+0000 7f7d6b200640 1 -- 172.21.15.59:0/1212039256 >> 172.21.15.59:0/1212039256 conn(0x7f7d6c0fcee0 msgr2=0x7f7d6c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.670+0000 7f7d6b200640 1 -- 172.21.15.59:0/1212039256 shutdown_connections 2024-04-02T17:14:42.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.670+0000 7f7d6b200640 1 -- 172.21.15.59:0/1212039256 wait complete. 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 Processor -- start 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 -- start start 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 0x7f7d6c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c103520 0x7f7d6c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10d5f0 con 0x7f7d6c101970 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10d470 con 0x7f7d6c103520 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d6b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10d770 con 0x7f7d6c102b60 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d69400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d69e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 0x7f7d6c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d69400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39448/0 (socket says 172.21.15.59:39448) 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.671+0000 7f7d69400640 1 -- 172.21.15.59:0/3838204060 learned_addr learned my addr 172.21.15.59:0/3838204060 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d6a800640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c103520 0x7f7d6c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 -- 172.21.15.59:0/3838204060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c103520 msgr2=0x7f7d6c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c103520 0x7f7d6c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 -- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 msgr2=0x7f7d6c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 0x7f7d6c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 -- 172.21.15.59:0/3838204060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c19f870 con 0x7f7d6c102b60 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d69400640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009230 tx=0x7f7d5402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.672+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54041070 con 0x7f7d6c102b60 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.673+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d54004450 con 0x7f7d6c102b60 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.673+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54003e20 con 0x7f7d6c102b60 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.673+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c19fb00 con 0x7f7d6c102b60 2024-04-02T17:14:42.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.673+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c19ffa0 con 0x7f7d6c102b60 2024-04-02T17:14:42.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.674+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c108ce0 con 0x7f7d6c102b60 2024-04-02T17:14:42.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.676+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d54044070 con 0x7f7d6c102b60 2024-04-02T17:14:42.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.677+0000 7f7d62a00640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 0x7f7d500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:42.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.677+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7d540f1fe0 con 0x7f7d6c102b60 2024-04-02T17:14:42.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.677+0000 7f7d69e00640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 0x7f7d500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:42.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.678+0000 7f7d69e00640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 0x7f7d500a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c199130 tx=0x7f7d64009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:42.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.682+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d540be310 con 0x7f7d6c102b60 2024-04-02T17:14:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.955+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6c02d370 con 0x7f7d6c102b60 2024-04-02T17:14:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.956+0000 7f7d62a00640 1 -- 172.21.15.59:0/3838204060 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f7d540c2610 con 0x7f7d6c102b60 2024-04-02T17:14:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:14:42.956 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:12:54.846637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24529,24530,24547,24560,24584]},"id":1}]} 2024-04-02T17:14:42.956 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2024-04-02T17:14:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 msgr2=0x7f7d500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 0x7f7d500a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c199130 tx=0x7f7d64009500 comp rx=0 tx=0).stop 2024-04-02T17:14:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 msgr2=0x7f7d6c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009230 tx=0x7f7d5402c650 comp rx=0 tx=0).stop 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 shutdown_connections 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6c103520 0x7f7d6c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d6c102b60 0x7f7d6c1981a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d500a3c30 0x7f7d500a60f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 --2- 172.21.15.59:0/3838204060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d6c101970 0x7f7d6c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 >> 172.21.15.59:0/3838204060 conn(0x7f7d6c0fcee0 msgr2=0x7f7d6c105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 shutdown_connections 2024-04-02T17:14:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:42.959+0000 7f7d6b200640 1 -- 172.21.15.59:0/3838204060 wait complete. 2024-04-02T17:14:42.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T17:14:43.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 18466 11.68 MB/sec warmup 10 sec latency 23.790 ms 2024-04-02T17:14:43.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 -- 172.21.15.59:0/2854070174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934102460 msgr2=0x7f7934104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 --2- 172.21.15.59:0/2854070174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934102460 0x7f7934104850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7924009900 tx=0x7f792402bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 -- 172.21.15.59:0/2854070174 shutdown_connections 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 --2- 172.21.15.59:0/2854070174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7934104d90 0x7f7934107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 --2- 172.21.15.59:0/2854070174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79341076c0 0x7f7934109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 --2- 172.21.15.59:0/2854070174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934102460 0x7f7934104850 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 -- 172.21.15.59:0/2854070174 >> 172.21.15.59:0/2854070174 conn(0x7f79340fbe70 msgr2=0x7f79340fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 -- 172.21.15.59:0/2854070174 shutdown_connections 2024-04-02T17:14:43.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.218+0000 7f7933200640 1 -- 172.21.15.59:0/2854070174 wait complete. 2024-04-02T17:14:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 Processor -- start 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 -- start start 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 0x7f793419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79341076c0 0x7f79341a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7934100de0 con 0x7f79341076c0 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7934100c60 con 0x7f7934102460 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.219+0000 7f7933200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7934100f60 con 0x7f7934104d90 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 0x7f793419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45116/0 (socket says 172.21.15.59:45116) 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 -- 172.21.15.59:0/698449047 learned_addr learned my addr 172.21.15.59:0/698449047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7932800640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79341076c0 0x7f79341a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 -- 172.21.15.59:0/698449047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 msgr2=0x7f793419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 0x7f793419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 -- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79341076c0 msgr2=0x7f79341a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79341076c0 0x7f79341a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931e00640 1 -- 172.21.15.59:0/698449047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79341a1b20 con 0x7f7934102460 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.220+0000 7f7931400640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 0x7f793419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f7931e00640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f792402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f792aa00640 1 -- 172.21.15.59:0/698449047 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924043020 con 0x7f7934102460 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79341a1db0 con 0x7f7934102460 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f792aa00640 1 -- 172.21.15.59:0/698449047 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79240054e0 con 0x7f7934102460 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f792aa00640 1 -- 172.21.15.59:0/698449047 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924003c40 con 0x7f7934102460 2024-04-02T17:14:43.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.221+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79341a2220 con 0x7f7934102460 2024-04-02T17:14:43.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.224+0000 7f792aa00640 1 -- 172.21.15.59:0/698449047 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7924046020 con 0x7f7934102460 2024-04-02T17:14:43.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 Processor -- start 2024-04-02T17:14:43.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 -- start start 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d42310400 con 0x560d423cee20 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560d422f4f10 con 0x560d423d4980 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.225+0000 7f793b322740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d423346f0 con 0x560d423cf810 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7932800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45118/0 (socket says 172.21.15.59:45118) 2024-04-02T17:14:43.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 -- 172.21.15.59:0/2503772232 learned_addr learned my addr 172.21.15.59:0/2503772232 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 -- 172.21.15.59:0/2503772232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 msgr2=0x560d423d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 -- 172.21.15.59:0/2503772232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 msgr2=0x560d423cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.226+0000 7f7931400640 1 -- 172.21.15.59:0/2503772232 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d420483f0 con 0x560d423d4980 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f7931400640 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f791c0114f0 tx=0x7f791c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa91b82eff9520 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f792aa00640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f7923000640 1 -- 172.21.15.59:0/2503772232 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c005020 con 0x560d423d4980 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f792aa00640 1 -- 172.21.15.59:0/698449047 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f79240f2490 con 0x7f7934102460 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f7923000640 1 -- 172.21.15.59:0/2503772232 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c0047a0 con 0x560d423d4980 2024-04-02T17:14:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f7923000640 1 -- 172.21.15.59:0/2503772232 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c00f970 con 0x560d423d4980 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.227+0000 7f7932800640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 -- 172.21.15.59:0/2503772232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 msgr2=0x560d423d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f791c0114f0 tx=0x7f791c0119b0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f7932800640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 0x7f79180a6070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560d423d0070 tx=0x7f792c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 -- 172.21.15.59:0/2503772232 shutdown_connections 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423d6e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 --2- 172.21.15.59:0/2503772232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 -- 172.21.15.59:0/2503772232 >> 172.21.15.59:0/2503772232 conn(0x560d42396a50 msgr2=0x560d423ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 -- 172.21.15.59:0/2503772232 shutdown_connections 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.228+0000 7f793b322740 1 -- 172.21.15.59:0/2503772232 wait complete. 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 -- start start 2024-04-02T17:14:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 Processor -- start 2024-04-02T17:14:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d423306a0 con 0x560d423d4980 2024-04-02T17:14:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560d422dc980 con 0x560d423cee20 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560d422efcb0 con 0x560d423cf810 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f793b322740 10 client.0 _ll_register_callbacks cb 0x560d422ef6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f7931e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f7931400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f7931e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45126/0 (socket says 172.21.15.59:45126) 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 -- 172.21.15.59:0/1185683370 learned_addr learned my addr 172.21.15.59:0/1185683370 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.229+0000 7f7931400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39478/0 (socket says 172.21.15.59:39478) 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7932800640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 -- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 msgr2=0x560d423d95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 -- 172.21.15.59:0/1185683370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 msgr2=0x560d423e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 -- 172.21.15.59:0/1185683370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d421127d0 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7932800640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7931e00640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79240bab10 tx=0x7f79240ba350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62f7478d2551294 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7917400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79240be1f0 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f79240effe0 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.230+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f791c0047a0 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f79240be010 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 10 client.25841 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d423e3960 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c0047a0 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f79240c6e90 con 0x560d423cee20 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 10 client.25841 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 10 client.25841 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:14:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 20 client.25841 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d423e5090 0x560d423e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 4 client.25841 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.231+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d422f4040 con 0x560d423e5090 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.232+0000 7f7931400640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d423e5090 0x560d423e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.232+0000 7f7931400640 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d423e5090 0x560d423e7520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96cb6937ada67869 server_cookie=c147ebb0ac68c944 in_seq=0 out_seq=0 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.232+0000 7f7917400640 10 client.25841 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f7917400640 1 -- 172.21.15.59:0/1185683370 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d422f4040 con 0x560d423e5090 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f7917400640 10 client.25841 handle_command_reply: tid=0 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 client.25841 shutdown 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 msgr2=0x560d423d9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79240bab10 tx=0x7f79240ba350 comp rx=0 tx=0).stop 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 shutdown_connections 2024-04-02T17:14:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d423e5090 0x560d423e7520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f7917400640 0 client.25841 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 --2- 172.21.15.59:0/1185683370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cee20 0x560d423d9070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 >> 172.21.15.59:0/1185683370 conn(0x560d42396a50 msgr2=0x560d423c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 shutdown_connections 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.233+0000 7f793b322740 1 -- 172.21.15.59:0/1185683370 wait complete. 2024-04-02T17:14:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.234+0000 7f793b322740 20 client.25841 trim_cache size 0 max 16384 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.248+0000 7f793b322740 1 Processor -- start 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 -- start start 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 0x560d423c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79341ab190 con 0x560d423d4980 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79341accd0 con 0x560d423cf810 2024-04-02T17:14:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f793b322740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f79341ab860 con 0x560d423cee20 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f7931400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f7931e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 0x560d423c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f7931400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f7931400640 1 -- 172.21.15.59:0/497503859 learned_addr learned my addr 172.21.15.59:0/497503859 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.249+0000 7f7932800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 -- 172.21.15.59:0/497503859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 msgr2=0x560d423c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 0x560d423c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 -- 172.21.15.59:0/497503859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 msgr2=0x560d423d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 -- 172.21.15.59:0/497503859 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d423cad30 con 0x560d423cf810 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931e00640 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 0x560d423c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7931400640 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f791c011eb0 tx=0x7f791c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d7eed047a4389 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7922600640 1 -- 172.21.15.59:0/497503859 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c02d070 con 0x560d423cf810 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7922600640 1 -- 172.21.15.59:0/497503859 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c004cf0 con 0x560d423cf810 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.250+0000 7f7922600640 1 -- 172.21.15.59:0/497503859 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c029c20 con 0x560d423cf810 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- 172.21.15.59:0/497503859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 msgr2=0x560d423e05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f791c011eb0 tx=0x7f791c0096c0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- 172.21.15.59:0/497503859 shutdown_connections 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423d4980 0x560d423d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423cf810 0x560d423e05b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 --2- 172.21.15.59:0/497503859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cee20 0x560d423c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- 172.21.15.59:0/497503859 >> 172.21.15.59:0/497503859 conn(0x560d42396a50 msgr2=0x560d423c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- 172.21.15.59:0/497503859 shutdown_connections 2024-04-02T17:14:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- 172.21.15.59:0/497503859 wait complete. 2024-04-02T17:14:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.251+0000 7f793b322740 1 -- start start 2024-04-02T17:14:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 Processor -- start 2024-04-02T17:14:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423dd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79341ac600 con 0x560d423cee20 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79341abf30 con 0x560d423d4980 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f79341a9170 con 0x560d423cf810 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f793b322740 10 client.0 _ll_register_callbacks cb 0x7f79240f5bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f7932800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f7931e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423dd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f7932800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45142/0 (socket says 172.21.15.59:45142) 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.252+0000 7f7932800640 1 -- 172.21.15.59:0/4021204976 learned_addr learned my addr 172.21.15.59:0/4021204976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7931400640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 -- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 msgr2=0x560d423ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423ddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 -- 172.21.15.59:0/4021204976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 msgr2=0x560d423dd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423dd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 -- 172.21.15.59:0/4021204976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d423df2c0 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7931e00640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423dd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7932800640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f792c004280 tx=0x7f792c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e59a229c3f9534 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.253+0000 7f7917400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c02e060 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f791c004cf0 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f792c00f6f0 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f792c00f8f0 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f793b322740 10 client.25847 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d423e0680 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c004cf0 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.254+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f792c032030 con 0x560d423d4980 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 10 client.25847 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 10 client.25847 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 20 client.25847 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d4244bca0 0x560d4244e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 4 client.25847 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d42341c30 con 0x560d4244bca0 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f7931e00640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d4244bca0 0x560d4244e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f7931e00640 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d4244bca0 0x560d4244e0c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8c5d44d24d6d838 server_cookie=1e10d1b0cc2a90a8 in_seq=0 out_seq=0 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.255+0000 7f7917400640 10 client.25847 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f7917400640 1 -- 172.21.15.59:0/4021204976 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d42341c30 con 0x560d4244bca0 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f7917400640 10 client.25847 handle_command_reply: tid=0 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 client.25847 shutdown 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 msgr2=0x560d423decd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f792c004280 tx=0x7f792c004ab0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 shutdown_connections 2024-04-02T17:14:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x560d4244bca0 0x560d4244e0c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d423d4980 0x560d423decd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560d423cf810 0x560d423ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f7917400640 0 client.25847 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 --2- 172.21.15.59:0/4021204976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d423cee20 0x560d423dd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 >> 172.21.15.59:0/4021204976 conn(0x560d42396a50 msgr2=0x560d423c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 shutdown_connections 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 1 -- 172.21.15.59:0/4021204976 wait complete. 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.256+0000 7f793b322740 20 client.25847 trim_cache size 0 max 16384 2024-04-02T17:14:43.259 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 msgr2=0x7f79180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 0x7f79180a6070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x560d423d0070 tx=0x7f792c009500 comp rx=0 tx=0).stop 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 msgr2=0x7f7934199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f792402c650 comp rx=0 tx=0).stop 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 shutdown_connections 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79341076c0 0x7f79341a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7934104d90 0x7f793419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 --2- 172.21.15.59:0/698449047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7934102460 0x7f7934199f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 >> 172.21.15.59:0/698449047 conn(0x7f79340fbe70 msgr2=0x7f7934107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 shutdown_connections 2024-04-02T17:14:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.257+0000 7f7933200640 1 -- 172.21.15.59:0/698449047 wait complete. 2024-04-02T17:14:43.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T17:14:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.498+0000 7f6c3d800640 1 -- 172.21.15.59:0/2974320529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109310 msgr2=0x7f6c38109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 --2- 172.21.15.59:0/2974320529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109310 0x7f6c38109790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009900 tx=0x7f6c2402bc40 comp rx=0 tx=0).stop 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 -- 172.21.15.59:0/2974320529 shutdown_connections 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 --2- 172.21.15.59:0/2974320529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38109cd0 0x7f6c38110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 --2- 172.21.15.59:0/2974320529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109310 0x7f6c38109790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 --2- 172.21.15.59:0/2974320529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38108120 0x7f6c38108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 -- 172.21.15.59:0/2974320529 >> 172.21.15.59:0/2974320529 conn(0x7f6c380770c0 msgr2=0x7f6c38079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 -- 172.21.15.59:0/2974320529 shutdown_connections 2024-04-02T17:14:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.499+0000 7f6c3d800640 1 -- 172.21.15.59:0/2974320529 wait complete. 2024-04-02T17:14:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 Processor -- start 2024-04-02T17:14:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 -- start start 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 0x7f6c3819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109cd0 0x7f6c381a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38113db0 con 0x7f6c38109310 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38113c30 con 0x7f6c38109cd0 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c3d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38113f30 con 0x7f6c38108120 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39518/0 (socket says 172.21.15.59:39518) 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.500+0000 7f6c36a00640 1 -- 172.21.15.59:0/1187971074 learned_addr learned my addr 172.21.15.59:0/1187971074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c37400640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109cd0 0x7f6c381a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36000640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 0x7f6c3819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 -- 172.21.15.59:0/1187971074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109cd0 msgr2=0x7f6c381a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109cd0 0x7f6c381a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 -- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 msgr2=0x7f6c3819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 0x7f6c3819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 -- 172.21.15.59:0/1187971074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381a6030 con 0x7f6c38108120 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36000640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 0x7f6c3819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800d770 tx=0x7f6c2800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c2f400640 1 -- 172.21.15.59:0/1187971074 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28017070 con 0x7f6c38108120 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c2f400640 1 -- 172.21.15.59:0/1187971074 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28009c30 con 0x7f6c38108120 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381a6320 con 0x7f6c38108120 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.501+0000 7f6c2f400640 1 -- 172.21.15.59:0/1187971074 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28003dd0 con 0x7f6c38108120 2024-04-02T17:14:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.502+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a6840 con 0x7f6c38108120 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.505+0000 7f6c2f400640 1 -- 172.21.15.59:0/1187971074 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c28003fb0 con 0x7f6c38108120 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.505+0000 7f6c4041a740 1 Processor -- start 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.505+0000 7f6c4041a740 1 -- start start 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb39aa6400 con 0x55cb39b6a7d0 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cb39a8af10 con 0x55cb39b65660 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c4041a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cb39aca6f0 con 0x55cb39b64c70 2024-04-02T17:14:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45164/0 (socket says 172.21.15.59:45164) 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c36a00640 1 -- 172.21.15.59:0/1717502167 learned_addr learned my addr 172.21.15.59:0/1717502167 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c37400640 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c36000640 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.506+0000 7f6c2f400640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 0x7f6c1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 -- 172.21.15.59:0/1717502167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 msgr2=0x55cb39b65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b65090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 -- 172.21.15.59:0/1717502167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 msgr2=0x55cb39b6cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b6cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 -- 172.21.15.59:0/1717502167 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb397de3f0 con 0x55cb39b65660 2024-04-02T17:14:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c2f400640 1 -- 172.21.15.59:0/1187971074 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6c280cfe30 con 0x7f6c38108120 2024-04-02T17:14:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c37400640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 0x7f6c1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c28023b70 tx=0x7f6c28022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb8ff59f8253d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c23e00640 1 -- 172.21.15.59:0/1717502167 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2809bb90 con 0x55cb39b65660 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c23e00640 1 -- 172.21.15.59:0/1717502167 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2809c750 con 0x55cb39b65660 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.507+0000 7f6c23e00640 1 -- 172.21.15.59:0/1717502167 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2809b9b0 con 0x55cb39b65660 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 -- 172.21.15.59:0/1717502167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 msgr2=0x55cb39b6a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c28023b70 tx=0x7f6c28022000 comp rx=0 tx=0).stop 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c37400640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 0x7f6c1c0a6070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55cb39b65ec0 tx=0x7f6c30004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 -- 172.21.15.59:0/1717502167 shutdown_connections 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6a160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1717502167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 -- 172.21.15.59:0/1717502167 >> 172.21.15.59:0/1717502167 conn(0x55cb39b2ca50 msgr2=0x55cb39b605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 -- 172.21.15.59:0/1717502167 shutdown_connections 2024-04-02T17:14:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.508+0000 7f6c4041a740 1 -- 172.21.15.59:0/1717502167 wait complete. 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 -- start start 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 Processor -- start 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b64c70 0x55cb39b6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b6a7d0 0x55cb39b762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cb39ac66a0 con 0x55cb39b64c70 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cb39a72980 con 0x55cb39b65660 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cb39a85cb0 con 0x55cb39b6a7d0 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.509+0000 7f6c4041a740 10 client.0 _ll_register_callbacks cb 0x55cb39a856a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b64c70 0x55cb39b6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45172/0 (socket says 172.21.15.59:45172) 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 -- 172.21.15.59:0/3721741277 learned_addr learned my addr 172.21.15.59:0/3721741277 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c37400640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b6a7d0 0x55cb39b762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 -- 172.21.15.59:0/3721741277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b6a7d0 msgr2=0x55cb39b762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b6a7d0 0x55cb39b762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 -- 172.21.15.59:0/3721741277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b64c70 msgr2=0x55cb39b6ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b64c70 0x55cb39b6ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 -- 172.21.15.59:0/3721741277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb398a87d0 con 0x55cb39b65660 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.510+0000 7f6c36000640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c24036040 tx=0x7f6c24004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa4d4ce9e8ef724 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24005230 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6c240053d0 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2809c750 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c240055d0 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c4041a740 10 client.25859 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb39b796e0 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.511+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2809c750 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c2404d020 con 0x55cb39b65660 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 10 client.25859 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 10 client.25859 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 20 client.25859 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39b7ad10 0x55cb39b7d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 4 client.25859 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb39a8a040 con 0x55cb39b7ad10 2024-04-02T17:14:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.512+0000 7f6c36a00640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39b7ad10 0x55cb39b7d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.513+0000 7f6c36a00640 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39b7ad10 0x55cb39b7d190 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62639150e425dfef server_cookie=5ae8ab8fd70dab79 in_seq=0 out_seq=0 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.513+0000 7f6c20c00640 10 client.25859 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.513+0000 7f6c20c00640 1 -- 172.21.15.59:0/3721741277 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb39a8a040 con 0x55cb39b7ad10 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.513+0000 7f6c20c00640 10 client.25859 handle_command_reply: tid=0 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 client.25859 shutdown 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 msgr2=0x55cb39b6f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c24036040 tx=0x7f6c24004270 comp rx=0 tx=0).stop 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 shutdown_connections 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39b7ad10 0x55cb39b7d190 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b6a7d0 0x55cb39b762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b6f270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 --2- 172.21.15.59:0/3721741277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b64c70 0x55cb39b6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 >> 172.21.15.59:0/3721741277 conn(0x55cb39b2ca50 msgr2=0x55cb39b5f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c20c00640 0 client.25859 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 shutdown_connections 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 1 -- 172.21.15.59:0/3721741277 wait complete. 2024-04-02T17:14:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.514+0000 7f6c4041a740 20 client.25859 trim_cache size 0 max 16384 2024-04-02T17:14:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.531+0000 7f6c4041a740 1 Processor -- start 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.531+0000 7f6c4041a740 1 -- start start 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b65660 0x55cb39b76270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b5f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381af760 con 0x55cb39b6a7d0 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381b12a0 con 0x55cb39b64c70 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c4041a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381afe30 con 0x55cb39b65660 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c37400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b5f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45176/0 (socket says 172.21.15.59:45176) 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c36a00640 1 -- 172.21.15.59:0/1480253673 learned_addr learned my addr 172.21.15.59:0/1480253673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.532+0000 7f6c36000640 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b65660 0x55cb39b76270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 -- 172.21.15.59:0/1480253673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b65660 msgr2=0x55cb39b76270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b65660 0x55cb39b76270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 -- 172.21.15.59:0/1480253673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 msgr2=0x55cb39b5f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b5f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 -- 172.21.15.59:0/1480253673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb39b6d660 con 0x55cb39b64c70 2024-04-02T17:14:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c36a00640 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009610 tx=0x7f6c280cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f00ece30f7f636 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c23400640 1 -- 172.21.15.59:0/1480253673 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280b5070 con 0x55cb39b64c70 2024-04-02T17:14:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c23400640 1 -- 172.21.15.59:0/1480253673 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28009de0 con 0x55cb39b64c70 2024-04-02T17:14:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.533+0000 7f6c23400640 1 -- 172.21.15.59:0/1480253673 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a4cc0 con 0x55cb39b64c70 2024-04-02T17:14:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 -- 172.21.15.59:0/1480253673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 msgr2=0x55cb39b5f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009610 tx=0x7f6c280cf040 comp rx=0 tx=0).stop 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 -- 172.21.15.59:0/1480253673 shutdown_connections 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b5f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b65660 0x55cb39b76270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 --2- 172.21.15.59:0/1480253673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b64c70 0x55cb39b5f010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 -- 172.21.15.59:0/1480253673 >> 172.21.15.59:0/1480253673 conn(0x55cb39b2ca50 msgr2=0x55cb39b66ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 -- 172.21.15.59:0/1480253673 shutdown_connections 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.534+0000 7f6c4041a740 1 -- 172.21.15.59:0/1480253673 wait complete. 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 -- start start 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 Processor -- start 2024-04-02T17:14:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b74330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381b0bd0 con 0x55cb39b6a7d0 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381b0500 con 0x55cb39b65660 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c381ad740 con 0x55cb39b64c70 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.535+0000 7f6c4041a740 10 client.0 _ll_register_callbacks cb 0x7f6c24009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39550/0 (socket says 172.21.15.59:39550) 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b74330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c37400640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 -- 172.21.15.59:0/4260423793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 msgr2=0x55cb39b73df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 -- 172.21.15.59:0/4260423793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 msgr2=0x55cb39b75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36000640 1 -- 172.21.15.59:0/4260423793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb39b75930 con 0x55cb39b65660 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c37400640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36a00640 1 -- 172.21.15.59:0/4260423793 learned_addr learned my addr 172.21.15.59:0/4260423793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.536+0000 7f6c36a00640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c36000640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b74330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402c180 tx=0x7f6c24003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20288e050c6640bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2404f990 con 0x55cb39b65660 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6c2404fb30 con 0x55cb39b65660 2024-04-02T17:14:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c28009de0 con 0x55cb39b65660 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c24005850 con 0x55cb39b65660 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c4041a740 10 client.25865 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb39be1750 con 0x55cb39b65660 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.537+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28009de0 con 0x55cb39b65660 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c24057030 con 0x55cb39b65660 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 10 client.25865 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 10 client.25865 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 20 client.25865 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39be20f0 0x55cb39be4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 4 client.25865 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb39ad7c30 con 0x55cb39be20f0 2024-04-02T17:14:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.538+0000 7f6c36a00640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39be20f0 0x55cb39be4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c36a00640 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39be20f0 0x55cb39be4510 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf450d4bbfc6df6b server_cookie=8c8e7ad38a0c175b in_seq=0 out_seq=0 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c20c00640 10 client.25865 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c20c00640 1 -- 172.21.15.59:0/4260423793 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb39ad7c30 con 0x55cb39be20f0 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c20c00640 10 client.25865 handle_command_reply: tid=0 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 client.25865 shutdown 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 msgr2=0x55cb39b74330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b74330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402c180 tx=0x7f6c24003b90 comp rx=0 tx=0).stop 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 shutdown_connections 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x55cb39be20f0 0x55cb39be4510 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cb39b6a7d0 0x55cb39b75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb39b65660 0x55cb39b74330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 --2- 172.21.15.59:0/4260423793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb39b64c70 0x55cb39b73df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c20c00640 0 client.25865 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 >> 172.21.15.59:0/4260423793 conn(0x55cb39b2ca50 msgr2=0x55cb39b63820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 shutdown_connections 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 1 -- 172.21.15.59:0/4260423793 wait complete. 2024-04-02T17:14:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.539+0000 7f6c4041a740 20 client.25865 trim_cache size 0 max 16384 2024-04-02T17:14:43.545 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 msgr2=0x7f6c1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 0x7f6c1c0a6070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55cb39b65ec0 tx=0x7f6c30004ab0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 msgr2=0x7f6c3819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800d770 tx=0x7f6c2800dc30 comp rx=0 tx=0).stop 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 shutdown_connections 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6c1c0a3bb0 0x7f6c1c0a6070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c38109310 0x7f6c3819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c38109cd0 0x7f6c381a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 --2- 172.21.15.59:0/1187971074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c38108120 0x7f6c3819e410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.540+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 >> 172.21.15.59:0/1187971074 conn(0x7f6c380770c0 msgr2=0x7f6c3810c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.541+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 shutdown_connections 2024-04-02T17:14:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.541+0000 7f6c3d800640 1 -- 172.21.15.59:0/1187971074 wait complete. 2024-04-02T17:14:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:43 smithi059 ceph-mon[31664]: pgmap v437: 97 pgs: 97 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 536 KiB/s wr, 121 op/s 2024-04-02T17:14:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:43 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3838204060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:43.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T17:14:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:43 smithi164 ceph-mon[36499]: pgmap v437: 97 pgs: 97 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 536 KiB/s wr, 121 op/s 2024-04-02T17:14:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:43 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3838204060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:43 smithi154 ceph-mon[36479]: pgmap v437: 97 pgs: 97 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 536 KiB/s wr, 121 op/s 2024-04-02T17:14:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:43 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3838204060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 -- 172.21.15.59:0/2315727813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec0075b40 msgr2=0x7f4ec0073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/2315727813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec0075b40 0x7f4ec0073f90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009ca0 tx=0x7f4eb002c100 comp rx=0 tx=0).stop 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 -- 172.21.15.59:0/2315727813 shutdown_connections 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/2315727813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec01099d0 0x7f4ec010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/2315727813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0074560 0x7f4ec00749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/2315727813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec0075b40 0x7f4ec0073f90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 -- 172.21.15.59:0/2315727813 >> 172.21.15.59:0/2315727813 conn(0x7f4ec00fd0a0 msgr2=0x7f4ec00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 -- 172.21.15.59:0/2315727813 shutdown_connections 2024-04-02T17:14:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.777+0000 7f4ec7c00640 1 -- 172.21.15.59:0/2315727813 wait complete. 2024-04-02T17:14:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 Processor -- start 2024-04-02T17:14:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 -- start start 2024-04-02T17:14:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0075b40 0x7f4ec019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010f950 con 0x7f4ec0074560 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010f7d0 con 0x7f4ec01099d0 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec7c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010fad0 con 0x7f4ec0075b40 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.779+0000 7f4ec5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0075b40 0x7f4ec019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44884/0 (socket says 172.21.15.59:44884) 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45204/0 (socket says 172.21.15.59:45204) 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 -- 172.21.15.59:0/1458979227 learned_addr learned my addr 172.21.15.59:0/1458979227 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 -- 172.21.15.59:0/1458979227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0075b40 msgr2=0x7f4ec019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0075b40 0x7f4ec019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 -- 172.21.15.59:0/1458979227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 msgr2=0x7f4ec01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 -- 172.21.15.59:0/1458979227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01a1c40 con 0x7f4ec0074560 2024-04-02T17:14:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.780+0000 7f4ec5200640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009dc0 tx=0x7f4eb0035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.781+0000 7f4ebd600640 1 -- 172.21.15.59:0/1458979227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0039040 con 0x7f4ec0074560 2024-04-02T17:14:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.781+0000 7f4ebd600640 1 -- 172.21.15.59:0/1458979227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0044070 con 0x7f4ec0074560 2024-04-02T17:14:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.781+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01a1ed0 con 0x7f4ec0074560 2024-04-02T17:14:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.781+0000 7f4ebd600640 1 -- 172.21.15.59:0/1458979227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00044a0 con 0x7f4ec0074560 2024-04-02T17:14:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.781+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a22e0 con 0x7f4ec0074560 2024-04-02T17:14:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4ebd600640 1 -- 172.21.15.59:0/1458979227 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4eb0004640 con 0x7f4ec0074560 2024-04-02T17:14:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 Processor -- start 2024-04-02T17:14:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 -- start start 2024-04-02T17:14:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a7404c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a7404ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a73f88400 con 0x560a7404c7d0 2024-04-02T17:14:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a73f6cf10 con 0x560a74046c70 2024-04-02T17:14:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4eca99c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560a73fac6f0 con 0x560a74047660 2024-04-02T17:14:43.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.785+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45212/0 (socket says 172.21.15.59:45212) 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 -- 172.21.15.59:0/2330880682 learned_addr learned my addr 172.21.15.59:0/2330880682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a7404ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5200640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a7404c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 -- 172.21.15.59:0/2330880682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 msgr2=0x560a7404c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a7404c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 -- 172.21.15.59:0/2330880682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 msgr2=0x560a7404ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a7404ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 -- 172.21.15.59:0/2330880682 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a73cc03f0 con 0x560a74046c70 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a7404ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.786+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011540 tx=0x7f4eb8011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2f8ca37ab6b053 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4ebd600640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4eb6000640 1 -- 172.21.15.59:0/2330880682 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800f5c0 con 0x560a74046c70 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4eb6000640 1 -- 172.21.15.59:0/2330880682 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb800fc40 con 0x560a74046c70 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4eb6000640 1 -- 172.21.15.59:0/2330880682 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8019d20 con 0x560a74046c70 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4ebd600640 1 -- 172.21.15.59:0/1458979227 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4eb00f2b00 con 0x7f4ec0074560 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.787+0000 7f4ec5200640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- 172.21.15.59:0/2330880682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 msgr2=0x560a74047090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4ec5200640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bb6f0 tx=0x7f4eb00f6540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011540 tx=0x7f4eb8011a00 comp rx=0 tx=0).stop 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- 172.21.15.59:0/2330880682 shutdown_connections 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a7404ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a7404c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2330880682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74047090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- 172.21.15.59:0/2330880682 >> 172.21.15.59:0/2330880682 conn(0x560a7400ea50 msgr2=0x560a740425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- 172.21.15.59:0/2330880682 shutdown_connections 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- 172.21.15.59:0/2330880682 wait complete. 2024-04-02T17:14:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 -- start start 2024-04-02T17:14:43.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.788+0000 7f4eca99c740 1 Processor -- start 2024-04-02T17:14:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74050ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a74051430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74058470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a73fa86a0 con 0x560a7404c7d0 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a73f54980 con 0x560a74046c70 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560a73f67cb0 con 0x560a74047660 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4eca99c740 10 client.0 _ll_register_callbacks cb 0x560a73f676a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74050ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a74051430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4ec5c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74050ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45226/0 (socket says 172.21.15.59:45226) 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4ec5c00640 1 -- 172.21.15.59:0/305617588 learned_addr learned my addr 172.21.15.59:0/305617588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.789+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74058470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 -- 172.21.15.59:0/305617588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 msgr2=0x560a74050ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74050ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 -- 172.21.15.59:0/305617588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 msgr2=0x560a74058470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74058470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 -- 172.21.15.59:0/305617588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a73d8a7d0 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74058470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a74051430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ec019b530 tx=0x7f4eac012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35cb4f4996584239 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4eaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac004050 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb800fc40 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.790+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4eac0041f0 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.791+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4eac0043f0 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.791+0000 7f4eca99c740 10 client.25909 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.791+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a7405b890 con 0x560a74047660 2024-04-02T17:14:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.791+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb800fc40 con 0x560a74047660 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.791+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4eac025050 con 0x560a74047660 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 10 client.25909 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 10 client.25909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 20 client.25909 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a7405cfc0 0x560a7405f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 4 client.25909 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a73f6c040 con 0x560a7405cfc0 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a7405cfc0 0x560a7405f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.792+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a7405cfc0 0x560a7405f3e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44ed7148ed9c6767 server_cookie=7ecb0d0a8e60307c in_seq=0 out_seq=0 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.793+0000 7f4eaaa00640 10 client.25909 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.793+0000 7f4eaaa00640 1 -- 172.21.15.59:0/305617588 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a73f6c040 con 0x560a7405cfc0 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.793+0000 7f4eaaa00640 10 client.25909 handle_command_reply: tid=0 2024-04-02T17:14:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.793+0000 7f4eca99c740 1 client.25909 shutdown 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 msgr2=0x560a74051430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a74051430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ec019b530 tx=0x7f4eac012b60 comp rx=0 tx=0).stop 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 shutdown_connections 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a7405cfc0 0x560a7405f3e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74058470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74047660 0x560a74051430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 --2- 172.21.15.59:0/305617588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74046c70 0x560a74050ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 >> 172.21.15.59:0/305617588 conn(0x560a7400ea50 msgr2=0x560a740418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eaaa00640 0 client.25909 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:43.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 shutdown_connections 2024-04-02T17:14:43.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 1 -- 172.21.15.59:0/305617588 wait complete. 2024-04-02T17:14:43.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.794+0000 7f4eca99c740 20 client.25909 trim_cache size 0 max 16384 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.810+0000 7f4eca99c740 1 Processor -- start 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.810+0000 7f4eca99c740 1 -- start start 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 0x560a74050510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a7404c7d0 0x560a7404f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01ab440 con 0x560a74046c70 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01acf80 con 0x560a74047660 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4eca99c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01abb10 con 0x560a7404c7d0 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45230/0 (socket says 172.21.15.59:45230) 2024-04-02T17:14:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 -- 172.21.15.59:0/3727297487 learned_addr learned my addr 172.21.15.59:0/3727297487 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 0x560a74050510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a7404c7d0 0x560a7404f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 -- 172.21.15.59:0/3727297487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a7404c7d0 msgr2=0x560a7404f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a7404c7d0 0x560a7404f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 -- 172.21.15.59:0/3727297487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 msgr2=0x560a74050510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 0x560a74050510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.811+0000 7f4ebfe00640 1 -- 172.21.15.59:0/3727297487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a74049490 con 0x560a74047660 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.812+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 0x560a74050510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.812+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003c60 tx=0x7f4eac0105c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae5f568b30f71a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.812+0000 7f4eb5600640 1 -- 172.21.15.59:0/3727297487 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac02b030 con 0x560a74047660 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.812+0000 7f4eb5600640 1 -- 172.21.15.59:0/3727297487 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac003f50 con 0x560a74047660 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.812+0000 7f4eb5600640 1 -- 172.21.15.59:0/3727297487 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac005670 con 0x560a74047660 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- 172.21.15.59:0/3727297487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 msgr2=0x560a74058430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003c60 tx=0x7f4eac0105c0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- 172.21.15.59:0/3727297487 shutdown_connections 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a7404c7d0 0x560a7404f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74058430 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 --2- 172.21.15.59:0/3727297487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a74046c70 0x560a74050510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- 172.21.15.59:0/3727297487 >> 172.21.15.59:0/3727297487 conn(0x560a7400ea50 msgr2=0x560a74050ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- 172.21.15.59:0/3727297487 shutdown_connections 2024-04-02T17:14:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- 172.21.15.59:0/3727297487 wait complete. 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.813+0000 7f4eca99c740 1 -- start start 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 Processor -- start 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74046c70 0x560a74055e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74057360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01ac8b0 con 0x560a7404c7d0 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01ac1e0 con 0x560a74047660 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01a9420 con 0x560a74046c70 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4eca99c740 10 client.0 _ll_register_callbacks cb 0x7f4eac009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45236/0 (socket says 172.21.15.59:45236) 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4ebfe00640 1 -- 172.21.15.59:0/2214574851 learned_addr learned my addr 172.21.15.59:0/2214574851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.814+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74057360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74046c70 0x560a74055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 -- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74046c70 msgr2=0x560a74055e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74046c70 0x560a74055e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 -- 172.21.15.59:0/2214574851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 msgr2=0x560a74057360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74057360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 -- 172.21.15.59:0/2214574851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a74057950 con 0x560a74047660 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4eac012ba0 tx=0x7f4eac003ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f62320b120024bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4eaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.815+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00f790 con 0x560a74047660 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eac003f50 con 0x560a74047660 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4eac00f930 con 0x560a74047660 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4eac00fb30 con 0x560a74047660 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eca99c740 10 client.25883 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:43.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a740c3780 con 0x560a74047660 2024-04-02T17:14:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac003f50 con 0x560a74047660 2024-04-02T17:14:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4eac025040 con 0x560a74047660 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.816+0000 7f4eca99c740 10 client.25883 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eca99c740 10 client.25883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eca99c740 20 client.25883 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a740c4090 0x560a740c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eca99c740 4 client.25883 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a74050ce0 con 0x560a740c4090 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a740c4090 0x560a740c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4ec5c00640 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a740c4090 0x560a740c64b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7573e9b3441a291c server_cookie=eb57ddb97ddfc7a9 in_seq=0 out_seq=0 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eaaa00640 10 client.25883 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.817+0000 7f4eaaa00640 1 -- 172.21.15.59:0/2214574851 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a74050ce0 con 0x560a740c4090 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eaaa00640 10 client.25883 handle_command_reply: tid=0 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 client.25883 shutdown 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 msgr2=0x560a74056350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4eac012ba0 tx=0x7f4eac003ad0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 shutdown_connections 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x560a740c4090 0x560a740c64b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a7404c7d0 0x560a74057360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a74047660 0x560a74056350 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 --2- 172.21.15.59:0/2214574851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a74046c70 0x560a74055e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eaaa00640 0 client.25883 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 >> 172.21.15.59:0/2214574851 conn(0x560a7400ea50 msgr2=0x560a74041010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 shutdown_connections 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 1 -- 172.21.15.59:0/2214574851 wait complete. 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.818+0000 7f4eca99c740 20 client.25883 trim_cache size 0 max 16384 2024-04-02T17:14:43.821 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 msgr2=0x7f4ea40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00bb6f0 tx=0x7f4eb00f6540 comp rx=0 tx=0).stop 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 msgr2=0x7f4ec019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009dc0 tx=0x7f4eb0035d90 comp rx=0 tx=0).stop 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 shutdown_connections 2024-04-02T17:14:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ec01099d0 0x7f4ec01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ec0075b40 0x7f4ec019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 --2- 172.21.15.59:0/1458979227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec0074560 0x7f4ec019a020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 >> 172.21.15.59:0/1458979227 conn(0x7f4ec00fd0a0 msgr2=0x7f4ec01058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 shutdown_connections 2024-04-02T17:14:43.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:43.819+0000 7f4ec7c00640 1 -- 172.21.15.59:0/1458979227 wait complete. 2024-04-02T17:14:43.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T17:14:44.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 21144 11.92 MB/sec warmup 11 sec latency 35.454 ms 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 -- 172.21.15.59:0/2114429621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 msgr2=0x7fa74c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 --2- 172.21.15.59:0/2114429621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c101e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009900 tx=0x7fa73c02bc50 comp rx=0 tx=0).stop 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 -- 172.21.15.59:0/2114429621 shutdown_connections 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 --2- 172.21.15.59:0/2114429621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c1035b0 0x7fa74c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 --2- 172.21.15.59:0/2114429621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c102bf0 0x7fa74c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 --2- 172.21.15.59:0/2114429621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c101e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.058+0000 7fa750c00640 1 -- 172.21.15.59:0/2114429621 >> 172.21.15.59:0/2114429621 conn(0x7fa74c0fcf70 msgr2=0x7fa74c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.059+0000 7fa750c00640 1 -- 172.21.15.59:0/2114429621 shutdown_connections 2024-04-02T17:14:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.059+0000 7fa750c00640 1 -- 172.21.15.59:0/2114429621 wait complete. 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 Processor -- start 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 -- start start 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c102bf0 0x7fa74c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c1035b0 0x7fa74c1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10d6c0 con 0x7fa74c102bf0 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10d540 con 0x7fa74c101a00 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa750c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10d840 con 0x7fa74c1035b0 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.060+0000 7fa74b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45242/0 (socket says 172.21.15.59:45242) 2024-04-02T17:14:44.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 -- 172.21.15.59:0/1123598561 learned_addr learned my addr 172.21.15.59:0/1123598561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74be00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c1035b0 0x7fa74c1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74aa00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c102bf0 0x7fa74c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 -- 172.21.15.59:0/1123598561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c1035b0 msgr2=0x7fa74c1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c1035b0 0x7fa74c1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 -- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c102bf0 msgr2=0x7fa74c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c102bf0 0x7fa74c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 -- 172.21.15.59:0/1123598561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c1a1ba0 con 0x7fa74c101a00 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.061+0000 7fa74b400640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02c260 tx=0x7fa73c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.062+0000 7fa743e00640 1 -- 172.21.15.59:0/1123598561 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c041070 con 0x7fa74c101a00 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.062+0000 7fa743e00640 1 -- 172.21.15.59:0/1123598561 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c004450 con 0x7fa74c101a00 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.062+0000 7fa743e00640 1 -- 172.21.15.59:0/1123598561 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c034440 con 0x7fa74c101a00 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.062+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c1a1e30 con 0x7fa74c101a00 2024-04-02T17:14:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.062+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c1a22f0 con 0x7fa74c101a00 2024-04-02T17:14:44.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa754f86740 1 Processor -- start 2024-04-02T17:14:44.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa743e00640 1 -- 172.21.15.59:0/1123598561 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa73c044070 con 0x7fa74c101a00 2024-04-02T17:14:44.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa754f86740 1 -- start start 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa754f86740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa754f86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.066+0000 7fa754f86740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a8980 0x5584bc6aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa754f86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5584bc5e4400 con 0x5584bc6a3810 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa754f86740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584bc5c8f10 con 0x5584bc6a2e20 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa754f86740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584bc6086f0 con 0x5584bc6a8980 2024-04-02T17:14:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45252/0 (socket says 172.21.15.59:45252) 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 -- 172.21.15.59:0/536939155 learned_addr learned my addr 172.21.15.59:0/536939155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74aa00640 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a8980 0x5584bc6aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 -- 172.21.15.59:0/536939155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a8980 msgr2=0x5584bc6aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a8980 0x5584bc6aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 -- 172.21.15.59:0/536939155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 msgr2=0x5584bc6a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74be00640 1 -- 172.21.15.59:0/536939155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bc31c3f0 con 0x5584bc6a2e20 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.067+0000 7fa74b400640 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa743e00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 0x7fa7300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa74be00640 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa744009ca0 tx=0x7fa744011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15d571b289155a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa743e00640 1 -- 172.21.15.59:0/1123598561 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa73c0f1ea0 con 0x7fa74c101a00 2024-04-02T17:14:44.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa737e00640 1 -- 172.21.15.59:0/536939155 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74401e070 con 0x5584bc6a2e20 2024-04-02T17:14:44.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa74b400640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 0x7fa7300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa737e00640 1 -- 172.21.15.59:0/536939155 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa744009e50 con 0x5584bc6a2e20 2024-04-02T17:14:44.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.068+0000 7fa737e00640 1 -- 172.21.15.59:0/536939155 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400eac0 con 0x5584bc6a2e20 2024-04-02T17:14:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- 172.21.15.59:0/536939155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 msgr2=0x5584bc6a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa74b400640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 0x7fa7300a60f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0bab40 tx=0x7fa73c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa744009ca0 tx=0x7fa744011810 comp rx=0 tx=0).stop 2024-04-02T17:14:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- 172.21.15.59:0/536939155 shutdown_connections 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a8980 0x5584bc6aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 --2- 172.21.15.59:0/536939155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc6a3240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- 172.21.15.59:0/536939155 >> 172.21.15.59:0/536939155 conn(0x5584bc66aa50 msgr2=0x5584bc69e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- 172.21.15.59:0/536939155 shutdown_connections 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- 172.21.15.59:0/536939155 wait complete. 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.069+0000 7fa754f86740 1 -- start start 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 Processor -- start 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a2e20 0x5584bc6ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5584bc6046a0 con 0x5584bc6a2e20 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584bc5b0980 con 0x5584bc6a8980 2024-04-02T17:14:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584bc5c3cb0 con 0x5584bc6a3810 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.070+0000 7fa754f86740 10 client.0 _ll_register_callbacks cb 0x5584bc5c36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39620/0 (socket says 172.21.15.59:39620) 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a2e20 0x5584bc6ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 -- 172.21.15.59:0/1506874768 learned_addr learned my addr 172.21.15.59:0/1506874768 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 -- 172.21.15.59:0/1506874768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 msgr2=0x5584bc6b4610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 -- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a2e20 msgr2=0x5584bc6ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a2e20 0x5584bc6ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 -- 172.21.15.59:0/1506874768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bc3e67d0 con 0x5584bc6a3810 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa74aa00640 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa738011cf0 tx=0x7fa73800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e4f116298c9766 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.071+0000 7fa734c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738003970 con 0x5584bc6a3810 2024-04-02T17:14:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa738003b10 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa744009e50 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa738003d10 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa754f86740 10 client.25927 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584bc6b7aa0 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.072+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa744009e50 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa738019070 con 0x5584bc6a3810 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 10 client.25927 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 10 client.25927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 20 client.25927 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc6b9180 0x5584bc6bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 4 client.25927 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584bc5c8040 con 0x5584bc6b9180 2024-04-02T17:14:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.073+0000 7fa74be00640 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc6b9180 0x5584bc6bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.074+0000 7fa74be00640 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc6b9180 0x5584bc6bb600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d591485a14086392 server_cookie=c1421424ad6bd6c6 in_seq=0 out_seq=0 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.074+0000 7fa734c00640 10 client.25927 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa734c00640 1 -- 172.21.15.59:0/1506874768 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584bc5c8040 con 0x5584bc6b9180 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa734c00640 10 client.25927 handle_command_reply: tid=0 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 client.25927 shutdown 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 msgr2=0x5584bc6ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa738011cf0 tx=0x7fa73800f790 comp rx=0 tx=0).stop 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 shutdown_connections 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc6b9180 0x5584bc6bb600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6ad5d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 --2- 172.21.15.59:0/1506874768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a2e20 0x5584bc6ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa734c00640 0 client.25927 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 >> 172.21.15.59:0/1506874768 conn(0x5584bc66aa50 msgr2=0x5584bc69daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 shutdown_connections 2024-04-02T17:14:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.075+0000 7fa754f86740 1 -- 172.21.15.59:0/1506874768 wait complete. 2024-04-02T17:14:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.076+0000 7fa754f86740 20 client.25927 trim_cache size 0 max 16384 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.091+0000 7fa754f86740 1 Processor -- start 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 -- start start 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc69d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 0x5584bc6abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1ab210 con 0x5584bc6a8980 2024-04-02T17:14:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1acd50 con 0x5584bc6a2e20 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa754f86740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1ab8e0 con 0x5584bc6a3810 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa74aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa74aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39636/0 (socket says 172.21.15.59:39636) 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa74aa00640 1 -- 172.21.15.59:0/2251795553 learned_addr learned my addr 172.21.15.59:0/2251795553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa74be00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc69d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.092+0000 7fa74be00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 0x5584bc6abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 -- 172.21.15.59:0/2251795553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 msgr2=0x5584bc69d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc69d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 -- 172.21.15.59:0/2251795553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 msgr2=0x5584bc6abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 0x5584bc6abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 -- 172.21.15.59:0/2251795553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bc69f4e0 con 0x5584bc6a3810 2024-04-02T17:14:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74be00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 0x5584bc6abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa74aa00640 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa738011cc0 tx=0x7fa73800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e7c7a3182b8e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa737400640 1 -- 172.21.15.59:0/2251795553 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738025470 con 0x5584bc6a3810 2024-04-02T17:14:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa737400640 1 -- 172.21.15.59:0/2251795553 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa738003970 con 0x5584bc6a3810 2024-04-02T17:14:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.093+0000 7fa737400640 1 -- 172.21.15.59:0/2251795553 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380058d0 con 0x5584bc6a3810 2024-04-02T17:14:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 -- 172.21.15.59:0/2251795553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 msgr2=0x5584bc6b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa738011cc0 tx=0x7fa73800fc90 comp rx=0 tx=0).stop 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 -- 172.21.15.59:0/2251795553 shutdown_connections 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a8980 0x5584bc6abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a3810 0x5584bc6b45d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 --2- 172.21.15.59:0/2251795553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a2e20 0x5584bc69d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 -- 172.21.15.59:0/2251795553 >> 172.21.15.59:0/2251795553 conn(0x5584bc66aa50 msgr2=0x5584bc69d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 -- 172.21.15.59:0/2251795553 shutdown_connections 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.094+0000 7fa754f86740 1 -- 172.21.15.59:0/2251795553 wait complete. 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 -- start start 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 Processor -- start 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1ac680 con 0x5584bc6a3810 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1abfb0 con 0x5584bc6a8980 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1a91f0 con 0x5584bc6a2e20 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.095+0000 7fa754f86740 10 client.0 _ll_register_callbacks cb 0x7fa738011900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39652/0 (socket says 172.21.15.59:39652) 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 -- 172.21.15.59:0/3835093989 learned_addr learned my addr 172.21.15.59:0/3835093989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74aa00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 -- 172.21.15.59:0/3835093989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 msgr2=0x5584bc6b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 -- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 msgr2=0x5584bc6b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 -- 172.21.15.59:0/3835093989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bc6b3570 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa74be00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa744005fd0 tx=0x7fa744011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea15492e8eb1328 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.096+0000 7fa734c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74401e040 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7440037f0 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa738003970 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa74403c050 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 10 client.25933 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584bc6b48e0 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa738003970 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa74400e9f0 con 0x5584bc6a2e20 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 10 client.25933 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 10 client.25933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:14:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 20 client.25933 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc71fc20 0x5584bc722010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 4 client.25933 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.097+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584bc615c30 con 0x5584bc71fc20 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.098+0000 7fa74aa00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc71fc20 0x5584bc722010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.098+0000 7fa74aa00640 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc71fc20 0x5584bc722010 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9abf6a38aeb249c1 server_cookie=68df92a338ab7fef in_seq=0 out_seq=0 2024-04-02T17:14:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.098+0000 7fa734c00640 10 client.25933 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa734c00640 1 -- 172.21.15.59:0/3835093989 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584bc615c30 con 0x5584bc71fc20 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa734c00640 10 client.25933 handle_command_reply: tid=0 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 client.25933 shutdown 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 msgr2=0x5584bc6b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa744005fd0 tx=0x7fa744011810 comp rx=0 tx=0).stop 2024-04-02T17:14:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 shutdown_connections 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x5584bc71fc20 0x5584bc722010 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584bc6a3810 0x5584bc6b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584bc6a8980 0x5584bc6b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 --2- 172.21.15.59:0/3835093989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584bc6a2e20 0x5584bc6b1a30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa734c00640 0 client.25933 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 >> 172.21.15.59:0/3835093989 conn(0x5584bc66aa50 msgr2=0x5584bc6a2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 shutdown_connections 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 1 -- 172.21.15.59:0/3835093989 wait complete. 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.099+0000 7fa754f86740 20 client.25933 trim_cache size 0 max 16384 2024-04-02T17:14:44.103 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 msgr2=0x7fa7300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 0x7fa7300a60f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0bab40 tx=0x7fa73c009260 comp rx=0 tx=0).stop 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 msgr2=0x7fa74c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02c260 tx=0x7fa73c02c670 comp rx=0 tx=0).stop 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 shutdown_connections 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7300a3c30 0x7fa7300a60f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74c102bf0 0x7fa74c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa74c1035b0 0x7fa74c1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 --2- 172.21.15.59:0/1123598561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa74c101a00 0x7fa74c199f80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 >> 172.21.15.59:0/1123598561 conn(0x7fa74c0fcf70 msgr2=0x7fa74c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 shutdown_connections 2024-04-02T17:14:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.100+0000 7fa750c00640 1 -- 172.21.15.59:0/1123598561 wait complete. 2024-04-02T17:14:44.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T17:14:44.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 -- 172.21.15.59:0/1391674135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c103c00 msgr2=0x7fcd1c104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 --2- 172.21.15.59:0/1391674135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c103c00 0x7fcd1c104020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009900 tx=0x7fcd0c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 -- 172.21.15.59:0/1391674135 shutdown_connections 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 --2- 172.21.15.59:0/1391674135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c104df0 0x7fcd1c105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 --2- 172.21.15.59:0/1391674135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 --2- 172.21.15.59:0/1391674135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c103c00 0x7fcd1c104020 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 -- 172.21.15.59:0/1391674135 >> 172.21.15.59:0/1391674135 conn(0x7fcd1c0ff170 msgr2=0x7fcd1c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 -- 172.21.15.59:0/1391674135 shutdown_connections 2024-04-02T17:14:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.345+0000 7fcd1be00640 1 -- 172.21.15.59:0/1391674135 wait complete. 2024-04-02T17:14:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 Processor -- start 2024-04-02T17:14:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 -- start start 2024-04-02T17:14:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 0x7fcd1c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.346+0000 7fcd1be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10f8d0 con 0x7fcd1c103c00 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10f750 con 0x7fcd1c1057b0 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c10fa50 con 0x7fcd1c104df0 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 0x7fcd1c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39668/0 (socket says 172.21.15.59:39668) 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1a000640 1 -- 172.21.15.59:0/2049231695 learned_addr learned my addr 172.21.15.59:0/2049231695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 -- 172.21.15.59:0/2049231695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 msgr2=0x7fcd1c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 -- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 msgr2=0x7fcd1c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 0x7fcd1c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1b400640 1 -- 172.21.15.59:0/2049231695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c1a1b00 con 0x7fcd1c1057b0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 0x7fcd1c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.347+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd1b400640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c1a1490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009fd0 tx=0x7fcd1400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd13400640 1 -- 172.21.15.59:0/2049231695 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400ce20 con 0x7fcd1c1057b0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c1a1df0 con 0x7fcd1c1057b0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd13400640 1 -- 172.21.15.59:0/2049231695 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd14005020 con 0x7fcd1c1057b0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd13400640 1 -- 172.21.15.59:0/2049231695 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14010720 con 0x7fcd1c1057b0 2024-04-02T17:14:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.348+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c1a2300 con 0x7fcd1c1057b0 2024-04-02T17:14:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 Processor -- start 2024-04-02T17:14:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd13400640 1 -- 172.21.15.59:0/2049231695 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd14005540 con 0x7fcd1c1057b0 2024-04-02T17:14:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 -- start start 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 0x55a8c2945240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c294ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2886400 con 0x55a8c2944e20 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a8c286af10 con 0x55a8c2945810 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.352+0000 7fcd247b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8c28aa6f0 con 0x55a8c294a980 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 0x55a8c2945240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45306/0 (socket says 172.21.15.59:45306) 2024-04-02T17:14:44.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2968360916 learned_addr learned my addr 172.21.15.59:0/2968360916 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2968360916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 msgr2=0x55a8c294ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c294ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2968360916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 msgr2=0x55a8c2945240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 0x55a8c2945240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2968360916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c25be3f0 con 0x55a8c2945810 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 0x55a8c2945240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.353+0000 7fcd13400640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 0x7fcd000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c240 tx=0x7fcd0c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8ba07a1d7b8348 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd13400640 1 -- 172.21.15.59:0/2049231695 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fcd140d0a50 con 0x7fcd1c1057b0 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 0x7fcd000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd07e00640 1 -- 172.21.15.59:0/2968360916 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c045070 con 0x55a8c2945810 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd07e00640 1 -- 172.21.15.59:0/2968360916 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0c02cd30 con 0x55a8c2945810 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.354+0000 7fcd07e00640 1 -- 172.21.15.59:0/2968360916 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005160 con 0x55a8c2945810 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- 172.21.15.59:0/2968360916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 msgr2=0x55a8c294a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c240 tx=0x7fcd0c009560 comp rx=0 tx=0).stop 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 0x7fcd000a60f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd08009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- 172.21.15.59:0/2968360916 shutdown_connections 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c294ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c294a310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2968360916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2944e20 0x55a8c2945240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- 172.21.15.59:0/2968360916 >> 172.21.15.59:0/2968360916 conn(0x55a8c290ca50 msgr2=0x55a8c2940790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- 172.21.15.59:0/2968360916 shutdown_connections 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- 172.21.15.59:0/2968360916 wait complete. 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.355+0000 7fcd247b1740 1 -- start start 2024-04-02T17:14:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 Processor -- start 2024-04-02T17:14:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c294f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2956590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a8c28a66a0 con 0x55a8c294a980 2024-04-02T17:14:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2852980 con 0x55a8c2944e20 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2865cb0 con 0x55a8c2945810 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c294f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39678/0 (socket says 172.21.15.59:39678) 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd1a000640 1 -- 172.21.15.59:0/3895778684 learned_addr learned my addr 172.21.15.59:0/3895778684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.356+0000 7fcd247b1740 10 client.0 _ll_register_callbacks cb 0x55a8c28656a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1b400640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2956590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 -- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 msgr2=0x55a8c294f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c294f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 -- 172.21.15.59:0/3895778684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 msgr2=0x55a8c2956590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2956590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 -- 172.21.15.59:0/3895778684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c26887d0 con 0x55a8c2945810 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c294f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1b400640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2956590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd1a000640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800fef0 tx=0x7fcd08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3bea7bc04f57c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd04c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08009cd0 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.357+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fcd08018030 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0c02cd30 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd08004020 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 10 client.25942 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8c2959970 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0c02cd30 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcd08003830 con 0x55a8c2945810 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 10 client.25942 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.358+0000 7fcd247b1740 20 client.25942 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c295b120 0x55a8c295d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd247b1740 4 client.25942 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:44.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c286a040 con 0x55a8c295b120 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c295b120 0x55a8c295d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c295b120 0x55a8c295d5a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc99198fdc59e8dd server_cookie=d36ca006b34d4062 in_seq=0 out_seq=0 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.359+0000 7fcd04c00640 10 client.25942 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.360+0000 7fcd04c00640 1 -- 172.21.15.59:0/3895778684 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8c286a040 con 0x55a8c295b120 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.360+0000 7fcd04c00640 10 client.25942 handle_command_reply: tid=0 2024-04-02T17:14:44.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.360+0000 7fcd247b1740 1 client.25942 shutdown 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 msgr2=0x55a8c294f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800fef0 tx=0x7fcd08004730 comp rx=0 tx=0).stop 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 shutdown_connections 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c295b120 0x55a8c295d5a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2956590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd04c00640 0 client.25942 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2945810 0x55a8c294f550 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3895778684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c294f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 >> 172.21.15.59:0/3895778684 conn(0x55a8c290ca50 msgr2=0x55a8c293faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 shutdown_connections 2024-04-02T17:14:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 1 -- 172.21.15.59:0/3895778684 wait complete. 2024-04-02T17:14:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.361+0000 7fcd247b1740 20 client.25942 trim_cache size 0 max 16384 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.376+0000 7fcd247b1740 1 Processor -- start 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.376+0000 7fcd247b1740 1 -- start start 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 0x55a8c293f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2961e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1ab1e0 con 0x55a8c294a980 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1acd20 con 0x55a8c2945810 2024-04-02T17:14:44.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd247b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1ab8b0 con 0x55a8c2944e20 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45320/0 (socket says 172.21.15.59:45320) 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 0x55a8c293f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd1a000640 1 -- 172.21.15.59:0/3104390477 learned_addr learned my addr 172.21.15.59:0/3104390477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.377+0000 7fcd1b400640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2961e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 -- 172.21.15.59:0/3104390477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 msgr2=0x55a8c2956550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 -- 172.21.15.59:0/3104390477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 msgr2=0x55a8c2961e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2961e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 -- 172.21.15.59:0/3104390477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c29414e0 con 0x55a8c2944e20 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1a000640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1b400640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2961e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:44.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 0x55a8c293f970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a8c29416e0 tx=0x7fcd0c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6235504eab830f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd07400640 1 -- 172.21.15.59:0/3104390477 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c052030 con 0x55a8c2944e20 2024-04-02T17:14:44.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd07400640 1 -- 172.21.15.59:0/3104390477 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0c02cd30 con 0x55a8c2944e20 2024-04-02T17:14:44.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.378+0000 7fcd07400640 1 -- 172.21.15.59:0/3104390477 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c045040 con 0x55a8c2944e20 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 -- 172.21.15.59:0/3104390477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 msgr2=0x55a8c293f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 0x55a8c293f970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a8c29416e0 tx=0x7fcd0c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 -- 172.21.15.59:0/3104390477 shutdown_connections 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c294a980 0x55a8c2961e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2945810 0x55a8c2956550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 --2- 172.21.15.59:0/3104390477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c2944e20 0x55a8c293f970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 -- 172.21.15.59:0/3104390477 >> 172.21.15.59:0/3104390477 conn(0x55a8c290ca50 msgr2=0x55a8c293f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 -- 172.21.15.59:0/3104390477 shutdown_connections 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.379+0000 7fcd247b1740 1 -- 172.21.15.59:0/3104390477 wait complete. 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 -- start start 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 Processor -- start 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 0x55a8c2953e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c2954e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1ac650 con 0x55a8c2945810 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1abf80 con 0x55a8c2944e20 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1a91c0 con 0x55a8c294a980 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd247b1740 10 client.0 _ll_register_callbacks cb 0x7fcd0800fd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd1a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 0x55a8c2953e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45332/0 (socket says 172.21.15.59:45332) 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.380+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2703033502 learned_addr learned my addr 172.21.15.59:0/2703033502 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1b400640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c2954e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2703033502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 msgr2=0x55a8c2954e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c2954e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2703033502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 msgr2=0x55a8c2953e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 0x55a8c2953e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 -- 172.21.15.59:0/2703033502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c2955420 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 0x55a8c2953e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1b400640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c2954e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd1aa00640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c7b0 tx=0x7fcd0c004ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7372a53289ae56c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.381+0000 7fcd04c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c052070 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0c02cd30 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fcd0c005540 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd0c045040 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd247b1740 10 client.25916 fetch_fsmap learned FSMap version 28 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8c2956690 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0c02cd30 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcd0c003bb0 con 0x55a8c2944e20 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.382+0000 7fcd247b1740 10 client.25916 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd247b1740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:44.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd247b1740 20 client.25916 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:44.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c29c1880 0x55a8c29c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:44.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd247b1740 4 client.25916 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:14:44.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c28b7c30 con 0x55a8c29c1880 2024-04-02T17:14:44.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c29c1880 0x55a8c29c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd1a000640 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c29c1880 0x55a8c29c3ca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef21ccc76b9ac11 server_cookie=3af09501a5cbd856 in_seq=0 out_seq=0 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.383+0000 7fcd04c00640 10 client.25916 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd04c00640 1 -- 172.21.15.59:0/2703033502 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c28b7c30 con 0x55a8c29c1880 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd04c00640 10 client.25916 handle_command_reply: tid=0 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 client.25916 shutdown 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 msgr2=0x55a8c29538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c7b0 tx=0x7fcd0c004ca0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 shutdown_connections 2024-04-02T17:14:44.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a8c29c1880 0x55a8c29c3ca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a8c2945810 0x55a8c2953e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8c294a980 0x55a8c2954e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd04c00640 0 client.25916 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 --2- 172.21.15.59:0/2703033502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c2944e20 0x55a8c29538e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 >> 172.21.15.59:0/2703033502 conn(0x55a8c290ca50 msgr2=0x55a8c293faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 shutdown_connections 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 1 -- 172.21.15.59:0/2703033502 wait complete. 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.384+0000 7fcd247b1740 20 client.25916 trim_cache size 0 max 16384 2024-04-02T17:14:44.388 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 msgr2=0x7fcd000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 0x7fcd000a60f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd08009c10 comp rx=0 tx=0).stop 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 msgr2=0x7fcd1c1a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c1a1490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009fd0 tx=0x7fcd1400ef10 comp rx=0 tx=0).stop 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 shutdown_connections 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c1057b0 0x7fcd1c1a1490 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd1c104df0 0x7fcd1c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcd000a3c30 0x7fcd000a60f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 --2- 172.21.15.59:0/2049231695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c103c00 0x7fcd1c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:44.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 >> 172.21.15.59:0/2049231695 conn(0x7fcd1c0ff170 msgr2=0x7fcd1c108090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:44.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 shutdown_connections 2024-04-02T17:14:44.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:44.385+0000 7fcd1be00640 1 -- 172.21.15.59:0/2049231695 wait complete. 2024-04-02T17:14:45.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 22383 11.47 MB/sec warmup 12 sec latency 26.967 ms 2024-04-02T17:14:45.405 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T17:14:45.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:14:45.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T17:14:45.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-mon[31664]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 768 KiB/s wr, 155 op/s 2024-04-02T17:14:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:45 smithi164 ceph-mon[36499]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 768 KiB/s wr, 155 op/s 2024-04-02T17:14:45.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 -- 172.21.15.59:0/2544943955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 msgr2=0x7f085c103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 --2- 172.21.15.59:0/2544943955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c103050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0848009860 tx=0x7f084802bb70 comp rx=0 tx=0).stop 2024-04-02T17:14:45.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 -- 172.21.15.59:0/2544943955 shutdown_connections 2024-04-02T17:14:45.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 --2- 172.21.15.59:0/2544943955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 0x7f085c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 --2- 172.21.15.59:0/2544943955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c103050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 --2- 172.21.15.59:0/2544943955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 -- 172.21.15.59:0/2544943955 >> 172.21.15.59:0/2544943955 conn(0x7f085c0fcf70 msgr2=0x7f085c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 -- 172.21.15.59:0/2544943955 shutdown_connections 2024-04-02T17:14:45.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.637+0000 7f085be00640 1 -- 172.21.15.59:0/2544943955 wait complete. 2024-04-02T17:14:45.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.638+0000 7f085be00640 1 Processor -- start 2024-04-02T17:14:45.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.638+0000 7f085be00640 1 -- start start 2024-04-02T17:14:45.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.638+0000 7f085be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.638+0000 7f085be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 0x7f085c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10d660 con 0x7f085c103590 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10d4e0 con 0x7f085c102bd0 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f085c10d7e0 con 0x7f085c1019e0 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39720/0 (socket says 172.21.15.59:39720) 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 -- 172.21.15.59:0/1711200633 learned_addr learned my addr 172.21.15.59:0/1711200633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085b400640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 0x7f085c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085a000640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 -- 172.21.15.59:0/1711200633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 msgr2=0x7f085c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 -- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 msgr2=0x7f085c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 0x7f085c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085aa00640 1 -- 172.21.15.59:0/1711200633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c19f900 con 0x7f085c1019e0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.639+0000 7f085a000640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f085aa00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e9f0 tx=0x7f084c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f0853400640 1 -- 172.21.15.59:0/1711200633 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00ccc0 con 0x7f085c1019e0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c19fbf0 con 0x7f085c1019e0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f0853400640 1 -- 172.21.15.59:0/1711200633 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c0035f0 con 0x7f085c1019e0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f0853400640 1 -- 172.21.15.59:0/1711200633 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00ce60 con 0x7f085c1019e0 2024-04-02T17:14:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.640+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a0140 con 0x7f085c1019e0 2024-04-02T17:14:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f0853400640 1 -- 172.21.15.59:0/1711200633 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f084c003b10 con 0x7f085c1019e0 2024-04-02T17:14:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f08643dd740 1 Processor -- start 2024-04-02T17:14:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f08643dd740 1 -- start start 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f08643dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f08643dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.644+0000 7f08643dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3de7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f08643dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3d40310 con 0x55cfe3de53e0 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f08643dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3ce8690 con 0x55cfe3de0270 2024-04-02T17:14:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f08643dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3d3c2c0 con 0x55cfe3ddf880 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39728/0 (socket says 172.21.15.59:39728) 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45350/0 (socket says 172.21.15.59:45350) 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 -- 172.21.15.59:0/2298781866 learned_addr learned my addr 172.21.15.59:0/2298781866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085a000640 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3de7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 -- 172.21.15.59:0/2298781866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 msgr2=0x55cfe3de4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 -- 172.21.15.59:0/2298781866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 msgr2=0x55cfe3de7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3de7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085b400640 1 -- 172.21.15.59:0/2298781866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe3b1c7e0 con 0x55cfe3ddf880 2024-04-02T17:14:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.645+0000 7f085aa00640 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f085b400640 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0854009fd0 tx=0x7f08540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb11f94745c578d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f0853400640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 0x7f08400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f0847400640 1 -- 172.21.15.59:0/2298781866 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854004d20 con 0x55cfe3ddf880 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f0853400640 1 -- 172.21.15.59:0/1711200633 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f084c0d07a0 con 0x7f085c1019e0 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f0847400640 1 -- 172.21.15.59:0/2298781866 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854004050 con 0x55cfe3ddf880 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f0847400640 1 -- 172.21.15.59:0/2298781866 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854005260 con 0x55cfe3ddf880 2024-04-02T17:14:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.646+0000 7f085aa00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 0x7f08400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- 172.21.15.59:0/2298781866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 msgr2=0x55cfe3ddfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0854009fd0 tx=0x7f08540118d0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f085aa00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 0x7f08400a6070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f084c099340 tx=0x7f084c0d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- 172.21.15.59:0/2298781866 shutdown_connections 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3de7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3de4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 --2- 172.21.15.59:0/2298781866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3ddfca0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- 172.21.15.59:0/2298781866 >> 172.21.15.59:0/2298781866 conn(0x55cfe3d70270 msgr2=0x55cfe3ddb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- 172.21.15.59:0/2298781866 shutdown_connections 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- 172.21.15.59:0/2298781866 wait complete. 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.647+0000 7f08643dd740 1 -- start start 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 Processor -- start 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 0x55cfe3dea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de53e0 0x55cfe3df1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3de8390 con 0x55cfe3de0270 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3cfba60 con 0x55cfe3de53e0 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cfe3de8510 con 0x55cfe3ddf880 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f08643dd740 10 client.0 _ll_register_callbacks cb 0x55cfe3d01030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f085b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.648+0000 7f085a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 0x55cfe3dea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39730/0 (socket says 172.21.15.59:39730) 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085b400640 1 -- 172.21.15.59:0/2059386311 learned_addr learned my addr 172.21.15.59:0/2059386311 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085b400640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de53e0 0x55cfe3df1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 -- 172.21.15.59:0/2059386311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 msgr2=0x55cfe3de9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 -- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de53e0 msgr2=0x55cfe3df1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de53e0 0x55cfe3df1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 -- 172.21.15.59:0/2059386311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe3b7cf70 con 0x55cfe3de0270 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085b400640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f085a000640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 0x55cfe3dea2e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f084802c0b0 tx=0x7f0848005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5d40f84b3711cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.649+0000 7f083be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848045070 con 0x55cfe3de0270 2024-04-02T17:14:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0848004450 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0854004050 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0848051050 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f08643dd740 10 client.15846 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfe3a14170 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.650+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854004050 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f08480328e0 con 0x55cfe3de0270 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 20 client.15846 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3df5c60 0x55cfe3df80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 4 client.15846 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe3d4d850 con 0x55cfe3df5c60 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f085b400640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3df5c60 0x55cfe3df80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f085b400640 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3df5c60 0x55cfe3df80a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1d767a322560e4 server_cookie=2bba56d87e4e943c in_seq=0 out_seq=0 2024-04-02T17:14:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.651+0000 7f083be00640 10 client.15846 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.655 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:45 smithi154 ceph-mon[36479]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 768 KiB/s wr, 155 op/s 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f083be00640 1 -- 172.21.15.59:0/2059386311 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfe3d4d850 con 0x55cfe3df5c60 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f083be00640 10 client.15846 handle_command_reply: tid=0 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 client.15846 shutdown 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 msgr2=0x55cfe3dea2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 0x55cfe3dea2e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f084802c0b0 tx=0x7f0848005a90 comp rx=0 tx=0).stop 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 shutdown_connections 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3df5c60 0x55cfe3df80a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de0270 0x55cfe3dea2e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de53e0 0x55cfe3df1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 --2- 172.21.15.59:0/2059386311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3ddf880 0x55cfe3de9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f083be00640 0 client.15846 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 >> 172.21.15.59:0/2059386311 conn(0x55cfe3d70270 msgr2=0x55cfe3dd5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 shutdown_connections 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 1 -- 172.21.15.59:0/2059386311 wait complete. 2024-04-02T17:14:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.653+0000 7f08643dd740 20 client.15846 trim_cache size 0 max 16384 2024-04-02T17:14:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.674+0000 7f08643dd740 1 Processor -- start 2024-04-02T17:14:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.674+0000 7f08643dd740 1 -- start start 2024-04-02T17:14:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.674+0000 7f08643dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 0x55cfe3dd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f08643dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f08643dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3dfc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f08643dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a6d40 con 0x55cfe3de53e0 2024-04-02T17:14:45.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f08643dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a88b0 con 0x55cfe3ddf880 2024-04-02T17:14:45.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f08643dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a7410 con 0x55cfe3de0270 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 0x55cfe3dd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3dfc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39736/0 (socket says 172.21.15.59:39736) 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3dfc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45110/0 (socket says 172.21.15.59:45110) 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 -- 172.21.15.59:0/3727038098 learned_addr learned my addr 172.21.15.59:0/3727038098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 -- 172.21.15.59:0/3727038098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 msgr2=0x55cfe3dd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 0x55cfe3dd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 -- 172.21.15.59:0/3727038098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 msgr2=0x55cfe3dfc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3dfc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085a000640 1 -- 172.21.15.59:0/3727038098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe3ddf4c0 con 0x55cfe3de0270 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.675+0000 7f085b400640 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 0x55cfe3dd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.676+0000 7f085a000640 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084802c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30df8f8b87149641 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.676+0000 7f0846a00640 1 -- 172.21.15.59:0/3727038098 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084805a070 con 0x55cfe3de0270 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.676+0000 7f0846a00640 1 -- 172.21.15.59:0/3727038098 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848009450 con 0x55cfe3de0270 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.676+0000 7f0846a00640 1 -- 172.21.15.59:0/3727038098 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848045040 con 0x55cfe3de0270 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- 172.21.15.59:0/3727038098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 msgr2=0x55cfe3df12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084802c1f0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- 172.21.15.59:0/3727038098 shutdown_connections 2024-04-02T17:14:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3de53e0 0x55cfe3dfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de0270 0x55cfe3df12e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 --2- 172.21.15.59:0/3727038098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3ddf880 0x55cfe3dd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- 172.21.15.59:0/3727038098 >> 172.21.15.59:0/3727038098 conn(0x55cfe3d70270 msgr2=0x55cfe3dd5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- 172.21.15.59:0/3727038098 shutdown_connections 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- 172.21.15.59:0/3727038098 wait complete. 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 -- start start 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.677+0000 7f08643dd740 1 Processor -- start 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3ddf880 0x55cfe3df0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 0x55cfe3e5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a81b0 con 0x55cfe3ddf880 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a7ae0 con 0x55cfe3de0270 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1a4d00 con 0x55cfe3de53e0 2024-04-02T17:14:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f08643dd740 10 client.0 _ll_register_callbacks cb 0x7f084c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f085a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f085a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45384/0 (socket says 172.21.15.59:45384) 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f085a000640 1 -- 172.21.15.59:0/1252448057 learned_addr learned my addr 172.21.15.59:0/1252448057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f085b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3ddf880 0x55cfe3df0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.678+0000 7f085b400640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 0x55cfe3e5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 -- 172.21.15.59:0/1252448057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 msgr2=0x55cfe3e5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 0x55cfe3e5c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 -- 172.21.15.59:0/1252448057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3ddf880 msgr2=0x55cfe3df0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3ddf880 0x55cfe3df0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 -- 172.21.15.59:0/1252448057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe3e5c760 con 0x55cfe3de0270 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085b400640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 0x55cfe3e5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f085a000640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0848032d00 tx=0x7f0848004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b8861e7df9bfde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f083be00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.679+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084805a070 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0848009450 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0848032850 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0848039040 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f08643dd740 10 client.25934 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfe3e5da00 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848009450 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.680+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f0848068050 con 0x55cfe3de0270 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 10 client.25934 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 20 client.25934 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3e5e320 0x55cfe3e60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 4 client.25934 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T17:14:45.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cfe3d47ba0 con 0x55cfe3e5e320 2024-04-02T17:14:45.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f085b400640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3e5e320 0x55cfe3e60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f085b400640 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3e5e320 0x55cfe3e60740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837af99a6f4846ec server_cookie=89d2c0f12fa97194 in_seq=0 out_seq=0 2024-04-02T17:14:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.681+0000 7f083be00640 10 client.25934 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f083be00640 1 -- 172.21.15.59:0/1252448057 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cfe3d47ba0 con 0x55cfe3e5e320 2024-04-02T17:14:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f083be00640 10 client.25934 handle_command_reply: tid=0 2024-04-02T17:14:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 client.25934 shutdown 2024-04-02T17:14:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 msgr2=0x55cfe3df0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0848032d00 tx=0x7f0848004560 comp rx=0 tx=0).stop 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 shutdown_connections 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cfe3e5e320 0x55cfe3e60740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cfe3de53e0 0x55cfe3e5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfe3de0270 0x55cfe3df0790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 --2- 172.21.15.59:0/1252448057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfe3ddf880 0x55cfe3df0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f083be00640 0 client.25934 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.682+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 >> 172.21.15.59:0/1252448057 conn(0x55cfe3d70270 msgr2=0x55cfe3dd5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.683+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 shutdown_connections 2024-04-02T17:14:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.683+0000 7f08643dd740 1 -- 172.21.15.59:0/1252448057 wait complete. 2024-04-02T17:14:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.683+0000 7f08643dd740 20 client.25934 trim_cache size 0 max 16384 2024-04-02T17:14:45.691 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:45.691 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-02T17:14:45.691 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "224d6c7f-97d2-4abb-9ac8-e4d2eeeab2a5", 2024-04-02T17:14:45.691 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-02T17:14:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 msgr2=0x7f08400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 0x7f08400a6070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f084c099340 tx=0x7f084c0d6530 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 msgr2=0x7f085c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e9f0 tx=0x7f084c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 shutdown_connections 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08400a3bb0 0x7f08400a6070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085c103590 0x7f085c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f085c102bd0 0x7f085c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 --2- 172.21.15.59:0/1711200633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f085c1019e0 0x7f085c197cf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 >> 172.21.15.59:0/1711200633 conn(0x7f085c0fcf70 msgr2=0x7f085c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 shutdown_connections 2024-04-02T17:14:45.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.684+0000 7f085be00640 1 -- 172.21.15.59:0/1711200633 wait complete. 2024-04-02T17:14:45.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:14:45.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:14:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 -- 172.21.15.59:0/376224380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f81036c0 msgr2=0x7f55f8109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 --2- 172.21.15.59:0/376224380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f81036c0 0x7f55f8109f50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ca0 tx=0x7f55f002c100 comp rx=0 tx=0).stop 2024-04-02T17:14:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 -- 172.21.15.59:0/376224380 shutdown_connections 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 --2- 172.21.15.59:0/376224380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 0x7f55f8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 --2- 172.21.15.59:0/376224380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f81036c0 0x7f55f8109f50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 --2- 172.21.15.59:0/376224380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f8101b10 0x7f55f8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 -- 172.21.15.59:0/376224380 >> 172.21.15.59:0/376224380 conn(0x7f55f80fd080 msgr2=0x7f55f80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 -- 172.21.15.59:0/376224380 shutdown_connections 2024-04-02T17:14:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.922+0000 7f55fd400640 1 -- 172.21.15.59:0/376224380 wait complete. 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 Processor -- start 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 -- start start 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 0x7f55f8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 0x7f55f819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55f810d790 con 0x7f55f8102d00 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55f810d610 con 0x7f55f8101b10 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.923+0000 7f55fd400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f810d910 con 0x7f55f81036c0 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45392/0 (socket says 172.21.15.59:45392) 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 -- 172.21.15.59:0/620734668 learned_addr learned my addr 172.21.15.59:0/620734668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f7200640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 0x7f55f819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f5e00640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 0x7f55f8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 -- 172.21.15.59:0/620734668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 msgr2=0x7f55f819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 0x7f55f819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 -- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 msgr2=0x7f55f8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 0x7f55f8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f6800640 1 -- 172.21.15.59:0/620734668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819f9f0 con 0x7f55f8101b10 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.924+0000 7f55f7200640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 0x7f55f819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55f6800640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9c0 tx=0x7f55e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55ef400640 1 -- 172.21.15.59:0/620734668 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800cd80 con 0x7f55f8101b10 2024-04-02T17:14:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55ef400640 1 -- 172.21.15.59:0/620734668 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e80035f0 con 0x7f55f8101b10 2024-04-02T17:14:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55ef400640 1 -- 172.21.15.59:0/620734668 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8010690 con 0x7f55f8101b10 2024-04-02T17:14:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819fce0 con 0x7f55f8101b10 2024-04-02T17:14:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.925+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a0180 con 0x7f55f8101b10 2024-04-02T17:14:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.928+0000 7f55ef400640 1 -- 172.21.15.59:0/620734668 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f55e80037a0 con 0x7f55f8101b10 2024-04-02T17:14:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 Processor -- start 2024-04-02T17:14:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 -- start start 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034edb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558034e36770 con 0x558034ed5be0 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558034dde700 con 0x558034ed65d0 2024-04-02T17:14:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.929+0000 7f560019b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558034e32720 con 0x558034edb740 2024-04-02T17:14:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034edb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39782/0 (socket says 172.21.15.59:39782) 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 -- 172.21.15.59:0/1588145545 learned_addr learned my addr 172.21.15.59:0/1588145545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 -- 172.21.15.59:0/1588145545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 msgr2=0x558034edb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034edb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 -- 172.21.15.59:0/1588145545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 msgr2=0x558034ed6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f5e00640 1 -- 172.21.15.59:0/1588145545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558034c147d0 con 0x558034edb740 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55f6800640 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034edb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.930+0000 7f55ef400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 0x7f55dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55ef400640 1 -- 172.21.15.59:0/620734668 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f55e80d07a0 con 0x7f55f8101b10 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55f7200640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 0x7f55dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009c90 tx=0x7f55e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a8ba8c623bec6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55e3400640 1 -- 172.21.15.59:0/1588145545 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e401e070 con 0x558034edb740 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55e3400640 1 -- 172.21.15.59:0/1588145545 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e4019b10 con 0x558034edb740 2024-04-02T17:14:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.931+0000 7f55e3400640 1 -- 172.21.15.59:0/1588145545 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e400ea50 con 0x558034edb740 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 -- 172.21.15.59:0/1588145545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 msgr2=0x558034eddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009c90 tx=0x7f55e4011810 comp rx=0 tx=0).stop 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f55f7200640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 0x7f55dc0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005e30 tx=0x7f55f002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 -- 172.21.15.59:0/1588145545 shutdown_connections 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eddbd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034edb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 --2- 172.21.15.59:0/1588145545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 -- 172.21.15.59:0/1588145545 >> 172.21.15.59:0/1588145545 conn(0x558034eccd40 msgr2=0x558034ecd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 -- 172.21.15.59:0/1588145545 shutdown_connections 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.932+0000 7f560019b740 1 -- 172.21.15.59:0/1588145545 wait complete. 2024-04-02T17:14:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 -- start start 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 Processor -- start 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed65d0 0x558034ed9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eda3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558034ed16f0 con 0x558034ed65d0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558034df1ac0 con 0x558034ed5be0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558034ed1870 con 0x558034edb740 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.933+0000 7f560019b740 10 client.0 _ll_register_callbacks cb 0x558034df4180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45410/0 (socket says 172.21.15.59:45410) 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 -- 172.21.15.59:0/1002287139 learned_addr learned my addr 172.21.15.59:0/1002287139 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed65d0 0x558034ed9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f7200640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eda3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 -- 172.21.15.59:0/1002287139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 msgr2=0x558034eda3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eda3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 -- 172.21.15.59:0/1002287139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed65d0 msgr2=0x558034ed9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed65d0 0x558034ed9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 -- 172.21.15.59:0/1002287139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558034b4a3f0 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f7200640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eda3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55f6800640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55e8098e20 tx=0x7f55e8098600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103f026ef075d367 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.934+0000 7f55d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e809c790 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f55e80a51a0 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e4019b10 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55e809c320 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 10 client.25946 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558034b0a0f0 con 0x558034ed5be0 2024-04-02T17:14:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e4019b10 con 0x558034ed5be0 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f55e80d0b90 con 0x558034ed5be0 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 10 client.25946 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.935+0000 7f560019b740 20 client.25946 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034ee7690 0x558034ee9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f560019b740 4 client.25946 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558034dd7630 con 0x558034ee7690 2024-04-02T17:14:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034ee7690 0x558034ee9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f55f5e00640 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034ee7690 0x558034ee9b20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596ab56cf3b09ad3 server_cookie=b4352b5fcc936fca in_seq=0 out_seq=0 2024-04-02T17:14:45.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.936+0000 7f55d7e00640 10 client.25946 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f55d7e00640 1 -- 172.21.15.59:0/1002287139 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558034dd7630 con 0x558034ee7690 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f55d7e00640 10 client.25946 handle_command_reply: tid=0 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 client.25946 shutdown 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 msgr2=0x558034ed8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55e8098e20 tx=0x7f55e8098600 comp rx=0 tx=0).stop 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 shutdown_connections 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034ee7690 0x558034ee9b20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed65d0 0x558034ed9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eda3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f55d7e00640 0 client.25946 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 --2- 172.21.15.59:0/1002287139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ed8ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 >> 172.21.15.59:0/1002287139 conn(0x558034eccd40 msgr2=0x558034ecbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 shutdown_connections 2024-04-02T17:14:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 1 -- 172.21.15.59:0/1002287139 wait complete. 2024-04-02T17:14:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.938+0000 7f560019b740 20 client.25946 trim_cache size 0 max 16384 2024-04-02T17:14:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 Processor -- start 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 -- start start 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a6fb0 con 0x558034ed5be0 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a8b20 con 0x558034ed65d0 2024-04-02T17:14:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.959+0000 7f560019b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a7680 con 0x558034edb740 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45426/0 (socket says 172.21.15.59:45426) 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 -- 172.21.15.59:0/182154745 learned_addr learned my addr 172.21.15.59:0/182154745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f6800640 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 -- 172.21.15.59:0/182154745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 msgr2=0x558034eee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 -- 172.21.15.59:0/182154745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 msgr2=0x558034ed9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 -- 172.21.15.59:0/182154745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558034ec73b0 con 0x558034ed65d0 2024-04-02T17:14:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.960+0000 7f55f5e00640 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55e401c040 tx=0x7f55e4009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720c663acfe77b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f55e2a00640 1 -- 172.21.15.59:0/182154745 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e401e050 con 0x558034ed65d0 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f55e2a00640 1 -- 172.21.15.59:0/182154745 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e402d070 con 0x558034ed65d0 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f55e2a00640 1 -- 172.21.15.59:0/182154745 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40196b0 con 0x558034ed65d0 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 -- 172.21.15.59:0/182154745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 msgr2=0x558034ed06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55e401c040 tx=0x7f55e4009850 comp rx=0 tx=0).stop 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 -- 172.21.15.59:0/182154745 shutdown_connections 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034edb740 0x558034eee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed65d0 0x558034ed06b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 --2- 172.21.15.59:0/182154745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034ed5be0 0x558034ed9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.961+0000 7f560019b740 1 -- 172.21.15.59:0/182154745 >> 172.21.15.59:0/182154745 conn(0x558034eccd40 msgr2=0x558034ecffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.962+0000 7f560019b740 1 -- 172.21.15.59:0/182154745 shutdown_connections 2024-04-02T17:14:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.962+0000 7f560019b740 1 -- 172.21.15.59:0/182154745 wait complete. 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.962+0000 7f560019b740 1 -- start start 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.962+0000 7f560019b740 1 Processor -- start 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 0x558034ee2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a8420 con 0x558034edb740 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a7d50 con 0x558034ed5be0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81a4f90 con 0x558034ed65d0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f560019b740 10 client.0 _ll_register_callbacks cb 0x7f55e80d6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45434/0 (socket says 172.21.15.59:45434) 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39808/0 (socket says 172.21.15.59:39808) 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 -- 172.21.15.59:0/4261963093 learned_addr learned my addr 172.21.15.59:0/4261963093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f7200640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 0x558034ee2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 -- 172.21.15.59:0/4261963093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 msgr2=0x558034ee1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 -- 172.21.15.59:0/4261963093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 msgr2=0x558034ee2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 0x558034ee2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f6800640 1 -- 172.21.15.59:0/4261963093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558034ee3490 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f5e00640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.963+0000 7f55f7200640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 0x558034ee2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55f6800640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f55e8099390 tx=0x7f55e8003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cfceee9cf6826c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e809c500 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e402d070 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f55e80a53a0 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55e809c790 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f560019b740 10 client.25952 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558034f4dfd0 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.964+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e402d070 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f55e809bf30 con 0x558034ed5be0 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 10 client.25952 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 20 client.25952 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034f4e8c0 0x558034f50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 4 client.25952 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558034e43cb0 con 0x558034f4e8c0 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f55f5e00640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034f4e8c0 0x558034f50ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f55f5e00640 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034f4e8c0 0x558034f50ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12a10e69cd28f3a server_cookie=46df6faad2bbd614 in_seq=0 out_seq=0 2024-04-02T17:14:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.965+0000 7f55d7e00640 10 client.25952 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f55d7e00640 1 -- 172.21.15.59:0/4261963093 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558034e43cb0 con 0x558034f4e8c0 2024-04-02T17:14:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f55d7e00640 10 client.25952 handle_command_reply: tid=0 2024-04-02T17:14:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 client.25952 shutdown 2024-04-02T17:14:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 msgr2=0x558034ee19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f55e8099390 tx=0x7f55e8003d20 comp rx=0 tx=0).stop 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 shutdown_connections 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558034f4e8c0 0x558034f50ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558034edb740 0x558034ee2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558034ed65d0 0x558034ee1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 --2- 172.21.15.59:0/4261963093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558034ed5be0 0x558034ee19f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f55d7e00640 0 client.25952 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 >> 172.21.15.59:0/4261963093 conn(0x558034eccd40 msgr2=0x558034ed0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 shutdown_connections 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.967+0000 7f560019b740 1 -- 172.21.15.59:0/4261963093 wait complete. 2024-04-02T17:14:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f560019b740 20 client.25952 trim_cache size 0 max 16384 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "224d6c7f-97d2-4abb-9ac8-e4d2eeeab2a5": { 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "224d6c7f-97d2-4abb-9ac8-e4d2eeeab2a5", 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:14:45.971 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 msgr2=0x7f55dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 0x7f55dc0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005e30 tx=0x7f55f002e040 comp rx=0 tx=0).stop 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 msgr2=0x7f55f8197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9c0 tx=0x7f55e800ee80 comp rx=0 tx=0).stop 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 shutdown_connections 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f55dc0a3c30 0x7f55dc0a60f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f8102d00 0x7f55f8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f81036c0 0x7f55f819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 --2- 172.21.15.59:0/620734668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f8101b10 0x7f55f8197dd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.968+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 >> 172.21.15.59:0/620734668 conn(0x7f55f80fd080 msgr2=0x7f55f8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.969+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 shutdown_connections 2024-04-02T17:14:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:14:45.969+0000 7f55fd400640 1 -- 172.21.15.59:0/620734668 wait complete. 2024-04-02T17:14:45.985 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.695+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae04075600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v800>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v4206>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.985 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.695+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae04075b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v800>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v4206>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.696+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae03ef5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v922>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v4373>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.696+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP auth{0=1} v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55ae03e87180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v922>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v4373>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.696+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae04074b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v922>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v4373>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.697+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae04076100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v800>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v4206>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.697+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ae04074580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v922>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v4373>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.986 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:14:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:14:45.697+0000 7f3debc00700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [2,head] /volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA auth{0=1} v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15159=0-4194304@1} caps={15159=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15159 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55ae04075080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v922>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v4373>,<0x10000000008/~dmtmp v58>,<0x10000000007/client0 v42>,<0x10000000006/clients v32>,<0x10000000005/tmp v32>,<0x100000001fb/client.0 v32>,<0x100000001fa/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8 v47>,<0x10000000001/sv_1 v67>,<0x10000000000/qa v25>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-02T17:14:45.987 INFO:tasks.cephfs.filesystem:scrub status for tag:224d6c7f-97d2-4abb-9ac8-e4d2eeeab2a5 - {'path': '/', 'tag': '224d6c7f-97d2-4abb-9ac8-e4d2eeeab2a5', 'options': 'recursive,force'} 2024-04-02T17:14:46.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 24903 11.78 MB/sec warmup 13 sec latency 9.891 ms 2024-04-02T17:14:46.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub queued for path: / 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub summary: active paths [/] 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub summary: active paths [/] 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:46 smithi059 ceph-mon[31664]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub queued for path: / 2024-04-02T17:14:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub summary: active paths [/] 2024-04-02T17:14:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub summary: active paths [/] 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:46 smithi164 ceph-mon[36499]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub queued for path: / 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub summary: active paths [/] 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub summary: active paths [/] 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.g log and `damage ls` output for details 2024-04-02T17:14:46.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:46.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:46 smithi154 ceph-mon[36479]: scrub summary: idle+waiting paths [/] 2024-04-02T17:14:47.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25112 11.04 MB/sec warmup 14 sec latency 753.409 ms 2024-04-02T17:14:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:47 smithi164 ceph-mon[36499]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-02T17:14:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:47 smithi154 ceph-mon[36479]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-02T17:14:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:47 smithi059 ceph-mon[31664]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-02T17:14:48.030 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25112 10.30 MB/sec warmup 15 sec latency 1753.538 ms 2024-04-02T17:14:49.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25199 9.66 MB/sec warmup 16 sec latency 1875.613 ms 2024-04-02T17:14:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:49 smithi164 ceph-mon[36499]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-02T17:14:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:49 smithi154 ceph-mon[36479]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.728+0000 7fc887200640 1 -- 172.21.15.59:0/3777432649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880103d60 msgr2=0x7fc880104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.728+0000 7fc887200640 1 --2- 172.21.15.59:0/3777432649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880103d60 0x7fc880104180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc870009900 tx=0x7fc87002bc40 comp rx=0 tx=0).stop 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 -- 172.21.15.59:0/3777432649 shutdown_connections 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 --2- 172.21.15.59:0/3777432649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880104f50 0x7fc8801053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 --2- 172.21.15.59:0/3777432649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 0x7fc88010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 --2- 172.21.15.59:0/3777432649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880103d60 0x7fc880104180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 -- 172.21.15.59:0/3777432649 >> 172.21.15.59:0/3777432649 conn(0x7fc8800ff2f0 msgr2=0x7fc880101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 -- 172.21.15.59:0/3777432649 shutdown_connections 2024-04-02T17:14:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.729+0000 7fc887200640 1 -- 172.21.15.59:0/3777432649 wait complete. 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 Processor -- start 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 -- start start 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880103d60 0x7fc880199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 0x7fc8801a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc88010f9e0 con 0x7fc880103d60 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc88010f860 con 0x7fc880104f50 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.730+0000 7fc887200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc88010fb60 con 0x7fc880105910 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880103d60 0x7fc880199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc885200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 0x7fc8801a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45446/0 (socket says 172.21.15.59:45446) 2024-04-02T17:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 -- 172.21.15.59:0/1038529598 learned_addr learned my addr 172.21.15.59:0/1038529598 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 -- 172.21.15.59:0/1038529598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 msgr2=0x7fc8801a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 0x7fc8801a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 -- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880103d60 msgr2=0x7fc880199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880103d60 0x7fc880199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 -- 172.21.15.59:0/1038529598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8801a1bc0 con 0x7fc880104f50 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.731+0000 7fc87f400640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ea60 tx=0x7fc86c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.732+0000 7fc87cc00640 1 -- 172.21.15.59:0/1038529598 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00cde0 con 0x7fc880104f50 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.732+0000 7fc87cc00640 1 -- 172.21.15.59:0/1038529598 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c0035f0 con 0x7fc880104f50 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.732+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8801a1eb0 con 0x7fc880104f50 2024-04-02T17:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.732+0000 7fc87cc00640 1 -- 172.21.15.59:0/1038529598 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c010670 con 0x7fc880104f50 2024-04-02T17:14:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.732+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801a23c0 con 0x7fc880104f50 2024-04-02T17:14:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 Processor -- start 2024-04-02T17:14:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 -- start start 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f10487f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104886ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1047e4d70 con 0x55f10487f8c0 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f10478d2a0 con 0x55f10487eed0 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc889f79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1047e0d20 con 0x55f104884a30 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc885200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f10487f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc87cc00640 1 -- 172.21.15.59:0/1038529598 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc86c0037a0 con 0x7fc880104f50 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45208/0 (socket says 172.21.15.59:45208) 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.736+0000 7fc87fe00640 1 -- 172.21.15.59:0/1506516579 learned_addr learned my addr 172.21.15.59:0/1506516579 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 -- 172.21.15.59:0/1506516579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 msgr2=0x55f104886ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87f400640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104886ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104886ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 -- 172.21.15.59:0/1506516579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 msgr2=0x55f10487f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f10487f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 -- 172.21.15.59:0/1506516579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1047a1840 con 0x55f10487f8c0 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc885200640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f10487f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87f400640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104886ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.737+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc870009900 tx=0x7fc870009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b0d3cebf8198d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc875600640 1 -- 172.21.15.59:0/1506516579 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87002cb90 con 0x55f10487f8c0 2024-04-02T17:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc875600640 1 -- 172.21.15.59:0/1506516579 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc870005040 con 0x55f10487f8c0 2024-04-02T17:14:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc87cc00640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 0x7fc8640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc875600640 1 -- 172.21.15.59:0/1506516579 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8700048e0 con 0x55f10487f8c0 2024-04-02T17:14:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc87cc00640 1 -- 172.21.15.59:0/1038529598 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc86c0d0700 con 0x7fc880104f50 2024-04-02T17:14:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc885200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 0x7fc8640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc889f79740 1 -- 172.21.15.59:0/1506516579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 msgr2=0x55f1048843c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.738+0000 7fc889f79740 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc870009900 tx=0x7fc870009560 comp rx=0 tx=0).stop 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 -- 172.21.15.59:0/1506516579 shutdown_connections 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f1048843c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104886ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 --2- 172.21.15.59:0/1506516579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f10487f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 -- 172.21.15.59:0/1506516579 >> 172.21.15.59:0/1506516579 conn(0x55f10487a440 msgr2=0x55f10487a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 -- 172.21.15.59:0/1506516579 shutdown_connections 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc885200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 0x7fc8640a6190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55f104880120 tx=0x7fc878009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 -- 172.21.15.59:0/1506516579 wait complete. 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.739+0000 7fc889f79740 1 -- start start 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 Processor -- start 2024-04-02T17:14:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f104889920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f104890960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f104887980 con 0x55f10487eed0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f1047a0660 con 0x55f104884a30 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f104887b00 con 0x55f10487f8c0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc889f79740 10 client.0 _ll_register_callbacks cb 0x55f1047a0050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45224/0 (socket says 172.21.15.59:45224) 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.740+0000 7fc87fe00640 1 -- 172.21.15.59:0/525242384 learned_addr learned my addr 172.21.15.59:0/525242384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc885200640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f104890960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87f400640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f104889920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 -- 172.21.15.59:0/525242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 msgr2=0x55f104889920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f104889920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 -- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 msgr2=0x55f104890960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f104890960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 -- 172.21.15.59:0/525242384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1047afe80 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc87fe00640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc87002c7c0 tx=0x7fc870005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4857bed65005fca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc86a000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.741+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870004ad0 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc870004c70 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc870005040 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc870034440 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc889f79740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1044f9210 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc870005040 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc870056020 con 0x55f10487eed0 2024-04-02T17:14:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.742+0000 7fc889f79740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc889f79740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc889f79740 20 client.15882 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f104894dc0 0x55f1048971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc889f79740 4 client.15882 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f104794cd0 con 0x55f104894dc0 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc87f400640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f104894dc0 0x55f1048971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc87f400640 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f104894dc0 0x55f1048971b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab1d3a4b6b60053 server_cookie=2c4a561ed78f72b1 in_seq=0 out_seq=0 2024-04-02T17:14:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.743+0000 7fc86a000640 10 client.15882 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.744+0000 7fc86a000640 1 -- 172.21.15.59:0/525242384 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f104794cd0 con 0x55f104894dc0 2024-04-02T17:14:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.744+0000 7fc86a000640 10 client.15882 handle_command_reply: tid=0 2024-04-02T17:14:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 client.15882 shutdown 2024-04-02T17:14:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 msgr2=0x55f1048893e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc87002c7c0 tx=0x7fc870005420 comp rx=0 tx=0).stop 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 shutdown_connections 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f104894dc0 0x55f1048971b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f104890960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f104889920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 --2- 172.21.15.59:0/525242384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f1048893e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc86a000640 0 client.15882 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 >> 172.21.15.59:0/525242384 conn(0x55f10487a440 msgr2=0x55f104879640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 shutdown_connections 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 1 -- 172.21.15.59:0/525242384 wait complete. 2024-04-02T17:14:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.745+0000 7fc889f79740 20 client.15882 trim_cache size 0 max 16384 2024-04-02T17:14:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 Processor -- start 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 -- start start 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104890920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801ab330 con 0x55f10487f8c0 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801acea0 con 0x55f10487eed0 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc889f79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801aba00 con 0x55f104884a30 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc87f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45470/0 (socket says 172.21.15.59:45470) 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc87f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45228/0 (socket says 172.21.15.59:45228) 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc87fe00640 1 -- 172.21.15.59:0/1841190152 learned_addr learned my addr 172.21.15.59:0/1841190152 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.762+0000 7fc885200640 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104890920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 -- 172.21.15.59:0/1841190152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 msgr2=0x55f104890920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104890920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 -- 172.21.15.59:0/1841190152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 msgr2=0x55f1048794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 -- 172.21.15.59:0/1841190152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1045c5620 con 0x55f10487f8c0 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc87f400640 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc86c01f000 tx=0x7fc86c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8ccedf84929634 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc874c00640 1 -- 172.21.15.59:0/1841190152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c09c920 con 0x55f10487f8c0 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc874c00640 1 -- 172.21.15.59:0/1841190152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c0a0cd0 con 0x55f10487f8c0 2024-04-02T17:14:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.763+0000 7fc874c00640 1 -- 172.21.15.59:0/1841190152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c003c50 con 0x55f10487f8c0 2024-04-02T17:14:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- 172.21.15.59:0/1841190152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 msgr2=0x55f104879a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc86c01f000 tx=0x7fc86c087040 comp rx=0 tx=0).stop 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- 172.21.15.59:0/1841190152 shutdown_connections 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487f8c0 0x55f104879a00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f104884a30 0x55f104890920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 --2- 172.21.15.59:0/1841190152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f10487eed0 0x55f1048794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- 172.21.15.59:0/1841190152 >> 172.21.15.59:0/1841190152 conn(0x55f10487a440 msgr2=0x55f104878db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- 172.21.15.59:0/1841190152 shutdown_connections 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- 172.21.15.59:0/1841190152 wait complete. 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.764+0000 7fc889f79740 1 -- start start 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 Processor -- start 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f10488e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f10488f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801ac7a0 con 0x55f10487eed0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801ac0d0 con 0x55f104884a30 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801a9310 con 0x55f10487f8c0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc889f79740 10 client.0 _ll_register_callbacks cb 0x7fc87002cce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45240/0 (socket says 172.21.15.59:45240) 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.765+0000 7fc87fe00640 1 -- 172.21.15.59:0/1803936463 learned_addr learned my addr 172.21.15.59:0/1803936463 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87f400640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f10488e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc885200640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f10488f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 -- 172.21.15.59:0/1803936463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 msgr2=0x55f10488e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f10488e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 -- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 msgr2=0x55f10488f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f10488f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 -- 172.21.15.59:0/1803936463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10488fe00 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc87fe00640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc87002c6f0 tx=0x7fc870009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732d000852081a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.766+0000 7fc86a000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8700056a0 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c0a0cd0 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc870004030 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc870004230 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 10 client.15888 fetch_fsmap learned FSMap version 29 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1048fb570 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c0a0cd0 con 0x55f10487eed0 2024-04-02T17:14:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc870056040 con 0x55f10487eed0 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 20 client.15888 populate_metadata read hostname 'smithi059' 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f1048fbf90 0x55f1048fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 4 client.15888 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.767+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1047f22b0 con 0x55f1048fbf90 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.768+0000 7fc87f400640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f1048fbf90 0x55f1048fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.768+0000 7fc87f400640 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f1048fbf90 0x55f1048fe3b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd90110d04616464 server_cookie=6f37cfcb3578ac01 in_seq=0 out_seq=0 2024-04-02T17:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.768+0000 7fc86a000640 10 client.15888 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:49 smithi059 ceph-mon[31664]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc86a000640 1 -- 172.21.15.59:0/1803936463 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1047f22b0 con 0x55f1048fbf90 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc86a000640 10 client.15888 handle_command_reply: tid=0 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 client.15888 shutdown 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 msgr2=0x55f10488e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc87002c6f0 tx=0x7fc870009650 comp rx=0 tx=0).stop 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 shutdown_connections 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f1048fbf90 0x55f1048fe3b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f104884a30 0x55f10488f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f10487f8c0 0x55f10488e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc86a000640 0 client.15888 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 --2- 172.21.15.59:0/1803936463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f10487eed0 0x55f10488e2c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.851+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 >> 172.21.15.59:0/1803936463 conn(0x55f10487a440 msgr2=0x55f104879640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 shutdown_connections 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc889f79740 1 -- 172.21.15.59:0/1803936463 wait complete. 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc889f79740 20 client.15888 trim_cache size 0 max 16384 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:14:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 msgr2=0x7fc8640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 0x7fc8640a6190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55f104880120 tx=0x7fc878009500 comp rx=0 tx=0).stop 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 msgr2=0x7fc88019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ea60 tx=0x7fc86c00ef20 comp rx=0 tx=0).stop 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.852+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 shutdown_connections 2024-04-02T17:14:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc880105910 0x7fc8801a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc880104f50 0x7fc88019a4e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc8640a3cd0 0x7fc8640a6190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 --2- 172.21.15.59:0/1038529598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc880103d60 0x7fc880199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 >> 172.21.15.59:0/1038529598 conn(0x7fc8800ff2f0 msgr2=0x7fc8801081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 shutdown_connections 2024-04-02T17:14:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:14:49.853+0000 7fc887200640 1 -- 172.21.15.59:0/1038529598 wait complete. 2024-04-02T17:14:50.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25199 9.09 MB/sec warmup 17 sec latency 1603.477 ms 2024-04-02T17:14:51.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25199 8.58 MB/sec warmup 18 sec latency 2603.639 ms 2024-04-02T17:14:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:51 smithi164 ceph-mon[36499]: pgmap v441: 97 pgs: 97 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-02T17:14:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:51 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:51 smithi154 ceph-mon[36479]: pgmap v441: 97 pgs: 97 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-02T17:14:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:51 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:51.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:51 smithi059 ceph-mon[31664]: pgmap v441: 97 pgs: 97 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-02T17:14:51.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:51 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:51.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:14:51.326+0000 7fe5e1c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T17:14:52.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25199 8.13 MB/sec warmup 19 sec latency 3603.734 ms 2024-04-02T17:14:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:52 smithi164 ceph-mon[36499]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T17:14:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:52 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:14:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:52 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:52.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:52 smithi154 ceph-mon[36479]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T17:14:52.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:52 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:14:52.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:52 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:52.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:52 smithi059 ceph-mon[31664]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-02T17:14:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:52 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] up:active 2024-04-02T17:14:52.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:52 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:14:53.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25199 7.73 MB/sec warmup 20 sec latency 4603.889 ms 2024-04-02T17:14:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:53 smithi154 ceph-mon[36479]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 807 KiB/s wr, 157 op/s 2024-04-02T17:14:53.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:53 smithi059 ceph-mon[31664]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 807 KiB/s wr, 157 op/s 2024-04-02T17:14:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:53 smithi164 ceph-mon[36499]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 807 KiB/s wr, 157 op/s 2024-04-02T17:14:54.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25247 7.37 MB/sec warmup 21 sec latency 4718.788 ms 2024-04-02T17:14:55.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25247 7.04 MB/sec warmup 22 sec latency 1769.666 ms 2024-04-02T17:14:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:55 smithi154 ceph-mon[36479]: pgmap v443: 97 pgs: 97 active+clean; 8.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 742 KiB/s wr, 157 op/s 2024-04-02T17:14:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:55 smithi154 ceph-mon[36479]: scrub summary: idle 2024-04-02T17:14:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:55 smithi154 ceph-mon[36479]: scrub summary: idle 2024-04-02T17:14:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:55 smithi059 ceph-mon[31664]: pgmap v443: 97 pgs: 97 active+clean; 8.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 742 KiB/s wr, 157 op/s 2024-04-02T17:14:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:55 smithi059 ceph-mon[31664]: scrub summary: idle 2024-04-02T17:14:55.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:55 smithi059 ceph-mon[31664]: scrub summary: idle 2024-04-02T17:14:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:55 smithi164 ceph-mon[36499]: pgmap v443: 97 pgs: 97 active+clean; 8.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 742 KiB/s wr, 157 op/s 2024-04-02T17:14:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:55 smithi164 ceph-mon[36499]: scrub summary: idle 2024-04-02T17:14:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:55 smithi164 ceph-mon[36499]: scrub summary: idle 2024-04-02T17:14:56.031 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25247 6.73 MB/sec warmup 23 sec latency 2769.808 ms 2024-04-02T17:14:57.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25247 6.45 MB/sec warmup 24 sec latency 3769.944 ms 2024-04-02T17:14:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:57 smithi154 ceph-mon[36479]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:14:57.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:57 smithi059 ceph-mon[31664]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:14:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:57 smithi164 ceph-mon[36499]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:14:58.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25247 6.19 MB/sec warmup 25 sec latency 4770.114 ms 2024-04-02T17:14:59.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25596 5.96 MB/sec warmup 26 sec latency 4894.275 ms 2024-04-02T17:14:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:14:59 smithi154 ceph-mon[36479]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-02T17:14:59.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:14:59 smithi059 ceph-mon[31664]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-02T17:14:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:14:59 smithi164 ceph-mon[36499]: pgmap v445: 97 pgs: 97 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-02T17:15:00.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25596 5.74 MB/sec warmup 27 sec latency 1163.175 ms 2024-04-02T17:15:01.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25596 5.54 MB/sec warmup 28 sec latency 2163.276 ms 2024-04-02T17:15:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:01 smithi154 ceph-mon[36479]: pgmap v446: 97 pgs: 97 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-04-02T17:15:01.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:01 smithi059 ceph-mon[31664]: pgmap v446: 97 pgs: 97 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-04-02T17:15:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:01 smithi164 ceph-mon[36499]: pgmap v446: 97 pgs: 97 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.0 MiB/s wr, 110 op/s 2024-04-02T17:15:02.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25596 5.35 MB/sec warmup 29 sec latency 3163.433 ms 2024-04-02T17:15:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 -- 172.21.15.59:0/4026407466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30105910 msgr2=0x7f1c3010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 --2- 172.21.15.59:0/4026407466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30105910 0x7f1c3010c1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009900 tx=0x7f1c2802bc30 comp rx=0 tx=0).stop 2024-04-02T17:15:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 -- 172.21.15.59:0/4026407466 shutdown_connections 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 --2- 172.21.15.59:0/4026407466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30105910 0x7f1c3010c1a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 --2- 172.21.15.59:0/4026407466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30104f50 0x7f1c301053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 --2- 172.21.15.59:0/4026407466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 -- 172.21.15.59:0/4026407466 >> 172.21.15.59:0/4026407466 conn(0x7f1c300ff2f0 msgr2=0x7f1c30101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 -- 172.21.15.59:0/4026407466 shutdown_connections 2024-04-02T17:15:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.998+0000 7f1c35a00640 1 -- 172.21.15.59:0/4026407466 wait complete. 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.999+0000 7f1c35a00640 1 Processor -- start 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:02.999+0000 7f1c35a00640 1 -- start start 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30104f50 0x7f1c3019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30105910 0x7f1c301a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010f9c0 con 0x7f1c30104f50 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010f840 con 0x7f1c30105910 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c35a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010fb40 con 0x7f1c30103d60 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c26000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30104f50 0x7f1c3019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60930/0 (socket says 172.21.15.59:60930) 2024-04-02T17:15:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 -- 172.21.15.59:0/2939048974 learned_addr learned my addr 172.21.15.59:0/2939048974 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2f400640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30105910 0x7f1c301a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 -- 172.21.15.59:0/2939048974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30105910 msgr2=0x7f1c301a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30105910 0x7f1c301a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 -- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30104f50 msgr2=0x7f1c3019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30104f50 0x7f1c3019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.000+0000 7f1c2ea00640 1 -- 172.21.15.59:0/2939048974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c301a1c10 con 0x7f1c30103d60 2024-04-02T17:15:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c2ea00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000e9c0 tx=0x7f1c2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c27e00640 1 -- 172.21.15.59:0/2939048974 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2000cd80 con 0x7f1c30103d60 2024-04-02T17:15:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c27e00640 1 -- 172.21.15.59:0/2939048974 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c200035f0 con 0x7f1c30103d60 2024-04-02T17:15:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c301a1f00 con 0x7f1c30103d60 2024-04-02T17:15:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c27e00640 1 -- 172.21.15.59:0/2939048974 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20010690 con 0x7f1c30103d60 2024-04-02T17:15:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.001+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c301a2360 con 0x7f1c30103d60 2024-04-02T17:15:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.004+0000 7f1c27e00640 1 -- 172.21.15.59:0/2939048974 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c200037a0 con 0x7f1c30103d60 2024-04-02T17:15:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.004+0000 7f1c38676740 1 Processor -- start 2024-04-02T17:15:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 -- start start 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e02602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 0x55d7e02653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e0265a30 0x55d7e0267ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d7e01c5d70 con 0x55d7e02608c0 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d7e016e2a0 con 0x55d7e0265a30 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c38676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d7e01c1d20 con 0x55d7e025fed0 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c2ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 0x55d7e02653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c2ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 0x55d7e02653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34284/0 (socket says 172.21.15.59:34284) 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c2ea00640 1 -- 172.21.15.59:0/1604090024 learned_addr learned my addr 172.21.15.59:0/1604090024 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c2f400640 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e02602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.005+0000 7f1c26000640 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e0265a30 0x55d7e0267ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 -- 172.21.15.59:0/1604090024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e0265a30 msgr2=0x55d7e0267ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e0265a30 0x55d7e0267ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 -- 172.21.15.59:0/1604090024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 msgr2=0x55d7e02653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 0x55d7e02653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 -- 172.21.15.59:0/1604090024 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e0182840 con 0x55d7e025fed0 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c2f400640 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e02602f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c2802c170 tx=0x7f1c2802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be30dda5fb8ed443 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c27e00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 0x7f1c140a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.006+0000 7f1c1fe00640 1 -- 172.21.15.59:0/1604090024 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005d50 con 0x55d7e025fed0 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c27e00640 1 -- 172.21.15.59:0/2939048974 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1c200d0970 con 0x7f1c30103d60 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c2ea00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 0x7f1c140a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c1fe00640 1 -- 172.21.15.59:0/1604090024 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2802cb80 con 0x55d7e025fed0 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c1fe00640 1 -- 172.21.15.59:0/1604090024 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280046c0 con 0x55d7e025fed0 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c38676740 1 -- 172.21.15.59:0/1604090024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 msgr2=0x55d7e02602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c2ea00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 0x7f1c140a6110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c200d0c10 tx=0x7f1c200d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c38676740 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e02602f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c2802c170 tx=0x7f1c2802c650 comp rx=0 tx=0).stop 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c38676740 1 -- 172.21.15.59:0/1604090024 shutdown_connections 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.007+0000 7f1c38676740 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e02608c0 0x55d7e02653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e0265a30 0x55d7e0267ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 --2- 172.21.15.59:0/1604090024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e02602f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 -- 172.21.15.59:0/1604090024 >> 172.21.15.59:0/1604090024 conn(0x55d7e025b440 msgr2=0x55d7e025b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 -- 172.21.15.59:0/1604090024 shutdown_connections 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 -- 172.21.15.59:0/1604090024 wait complete. 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 -- start start 2024-04-02T17:15:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.008+0000 7f1c38676740 1 Processor -- start 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e026a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e0265a30 0x55d7e0271910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d7e0268980 con 0x55d7e0265a30 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d7e0181660 con 0x55d7e02608c0 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d7e0268b00 con 0x55d7e025fed0 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c38676740 10 client.0 _ll_register_callbacks cb 0x55d7e0181050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56250/0 (socket says 172.21.15.59:56250) 2024-04-02T17:15:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 -- 172.21.15.59:0/3942531033 learned_addr learned my addr 172.21.15.59:0/3942531033 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e0265a30 0x55d7e0271910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e026a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 -- 172.21.15.59:0/3942531033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 msgr2=0x55d7e026a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e026a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 -- 172.21.15.59:0/3942531033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e0265a30 msgr2=0x55d7e0271910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.009+0000 7f1c26000640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e0265a30 0x55d7e0271910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c26000640 1 -- 172.21.15.59:0/3942531033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e0190e80 con 0x55d7e02608c0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e026a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c26000640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011d10 tx=0x7f1c1800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b58770dac7ef1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18003970 con 0x55d7e02608c0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2802cb80 con 0x55d7e02608c0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1c18017070 con 0x55d7e02608c0 2024-04-02T17:15:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c18003b10 con 0x55d7e02608c0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c38676740 10 client.25982 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.010+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7dfeda210 con 0x55d7e02608c0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2802cb80 con 0x55d7e02608c0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f1c18023020 con 0x55d7e02608c0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 10 client.25982 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 20 client.25982 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e0276370 0x55d7e02787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 4 client.25982 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.011+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e0175cd0 con 0x55d7e0276370 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.012+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e0276370 0x55d7e02787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.012+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e0276370 0x55d7e02787b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9095ae06e35442e0 server_cookie=3779c60f598deb2f in_seq=0 out_seq=0 2024-04-02T17:15:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.012+0000 7f1c1cc00640 10 client.25982 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.013+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3942531033 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7e0175cd0 con 0x55d7e0276370 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.013+0000 7f1c1cc00640 10 client.25982 handle_command_reply: tid=0 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.013+0000 7f1c38676740 1 client.25982 shutdown 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 msgr2=0x55d7e026a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011d10 tx=0x7f1c1800f790 comp rx=0 tx=0).stop 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 shutdown_connections 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e0276370 0x55d7e02787b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e0265a30 0x55d7e0271910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026a8d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 --2- 172.21.15.59:0/3942531033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e025fed0 0x55d7e026a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 >> 172.21.15.59:0/3942531033 conn(0x55d7e025b440 msgr2=0x55d7e025a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c1cc00640 0 client.25982 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 shutdown_connections 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 1 -- 172.21.15.59:0/3942531033 wait complete. 2024-04-02T17:15:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.014+0000 7f1c38676740 20 client.25982 trim_cache size 0 max 16384 2024-04-02T17:15:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 Processor -- start 2024-04-02T17:15:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 -- start start 2024-04-02T17:15:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e0268ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e02693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e02718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301ab3a0 con 0x55d7e025fed0 2024-04-02T17:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301acfa0 con 0x55d7e02608c0 2024-04-02T17:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.031+0000 7f1c38676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301aba70 con 0x55d7e0265a30 2024-04-02T17:15:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e0268ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e02693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e0268ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34296/0 (socket says 172.21.15.59:34296) 2024-04-02T17:15:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c2f400640 1 -- 172.21.15.59:0/2977740153 learned_addr learned my addr 172.21.15.59:0/2977740153 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c2f400640 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e02718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 -- 172.21.15.59:0/2977740153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 msgr2=0x55d7e02718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e02718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 -- 172.21.15.59:0/2977740153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 msgr2=0x55d7e0268ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e0268ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 -- 172.21.15.59:0/2977740153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dffa6620 con 0x55d7e02608c0 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.032+0000 7f1c26000640 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e02693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c18025770 tx=0x7f1c1800fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7e8144f6b9678e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.033+0000 7f1c1f400640 1 -- 172.21.15.59:0/2977740153 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18025960 con 0x55d7e02608c0 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.033+0000 7f1c1f400640 1 -- 172.21.15.59:0/2977740153 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18025b00 con 0x55d7e02608c0 2024-04-02T17:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.033+0000 7f1c1f400640 1 -- 172.21.15.59:0/2977740153 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801b660 con 0x55d7e02608c0 2024-04-02T17:15:03.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25596 5.17 MB/sec warmup 30 sec latency 4163.502 ms 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.033+0000 7f1c38676740 1 -- 172.21.15.59:0/2977740153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 msgr2=0x55d7e02693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.033+0000 7f1c38676740 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e02693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c18025770 tx=0x7f1c1800fd10 comp rx=0 tx=0).stop 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 -- 172.21.15.59:0/2977740153 shutdown_connections 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e02718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e02693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 --2- 172.21.15.59:0/2977740153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e0268ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 -- 172.21.15.59:0/2977740153 >> 172.21.15.59:0/2977740153 conn(0x55d7e025b440 msgr2=0x55d7e025b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 -- 172.21.15.59:0/2977740153 shutdown_connections 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 -- 172.21.15.59:0/2977740153 wait complete. 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 -- start start 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.034+0000 7f1c38676740 1 Processor -- start 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e026ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e026ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301ac810 con 0x55d7e025fed0 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301ac140 con 0x55d7e02608c0 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301a9380 con 0x55d7e0265a30 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c38676740 10 client.0 _ll_register_callbacks cb 0x7f1c18011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e026ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c26000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c26000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56260/0 (socket says 172.21.15.59:56260) 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c26000640 1 -- 172.21.15.59:0/3073940922 learned_addr learned my addr 172.21.15.59:0/3073940922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.035+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e026ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 -- 172.21.15.59:0/3073940922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 msgr2=0x55d7e026ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e026ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 -- 172.21.15.59:0/3073940922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 msgr2=0x55d7e026ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e026ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 -- 172.21.15.59:0/3073940922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e0270540 con 0x55d7e02608c0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e026ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c26000640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c18005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9967a7fa632bb27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1803b060 con 0x55d7e02608c0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.036+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18025b00 con 0x55d7e02608c0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1c1801b7b0 con 0x55d7e02608c0 2024-04-02T17:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c18025910 con 0x55d7e02608c0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 10 client.25988 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7e0271850 con 0x55d7e02608c0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18025b00 con 0x55d7e02608c0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f1c1803f030 con 0x55d7e02608c0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 10 client.25988 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 20 client.25988 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e02dccf0 0x55d7e02df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.037+0000 7f1c38676740 4 client.25988 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.038+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7e01d32b0 con 0x55d7e02dccf0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.038+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e02dccf0 0x55d7e02df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.038+0000 7f1c2f400640 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e02dccf0 0x55d7e02df110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02cd4e71deb25cc server_cookie=c7026985c2b52db6 in_seq=0 out_seq=0 2024-04-02T17:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.038+0000 7f1c1cc00640 10 client.25988 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.062+0000 7f1c1cc00640 1 -- 172.21.15.59:0/3073940922 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7e01d32b0 con 0x55d7e02dccf0 2024-04-02T17:15:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.062+0000 7f1c1cc00640 10 client.25988 handle_command_reply: tid=0 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.062+0000 7f1c38676740 1 client.25988 shutdown 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 msgr2=0x55d7e026ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c18005780 comp rx=0 tx=0).stop 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 shutdown_connections 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55d7e02dccf0 0x55d7e02df110 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d7e0265a30 0x55d7e026ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d7e02608c0 0x55d7e026ef40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 --2- 172.21.15.59:0/3073940922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d7e025fed0 0x55d7e026ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c1cc00640 0 client.25988 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 >> 172.21.15.59:0/3073940922 conn(0x55d7e025b440 msgr2=0x55d7e026a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 shutdown_connections 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 1 -- 172.21.15.59:0/3073940922 wait complete. 2024-04-02T17:15:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.063+0000 7f1c38676740 20 client.25988 trim_cache size 0 max 16384 2024-04-02T17:15:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:15:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:15:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:15:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:15:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 msgr2=0x7f1c140a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 0x7f1c140a6110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c200d0c10 tx=0x7f1c200d6530 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 msgr2=0x7f1c30199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000e9c0 tx=0x7f1c2000ee80 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 shutdown_connections 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1c140a3c50 0x7f1c140a6110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c30104f50 0x7f1c3019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c30105910 0x7f1c301a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 --2- 172.21.15.59:0/2939048974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c30103d60 0x7f1c30199ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.064+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 >> 172.21.15.59:0/2939048974 conn(0x7f1c300ff2f0 msgr2=0x7f1c30108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.065+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 shutdown_connections 2024-04-02T17:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:03.065+0000 7f1c35a00640 1 -- 172.21.15.59:0/2939048974 wait complete. 2024-04-02T17:15:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:03 smithi154 ceph-mon[36479]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-04-02T17:15:03.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:03 smithi059 ceph-mon[31664]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-04-02T17:15:03.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:03 smithi164 ceph-mon[36499]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-04-02T17:15:04.032 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25664 5.01 MB/sec warmup 31 sec latency 4289.521 ms 2024-04-02T17:15:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:15:04.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:15:04.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:15:05.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25664 4.85 MB/sec warmup 32 sec latency 1783.420 ms 2024-04-02T17:15:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:05 smithi154 ceph-mon[36479]: pgmap v448: 97 pgs: 97 active+clean; 31 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-04-02T17:15:05.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:05 smithi059 ceph-mon[31664]: pgmap v448: 97 pgs: 97 active+clean; 31 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-04-02T17:15:05.723 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:05 smithi164 ceph-mon[36499]: pgmap v448: 97 pgs: 97 active+clean; 31 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-04-02T17:15:06.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25664 4.70 MB/sec warmup 33 sec latency 2783.601 ms 2024-04-02T17:15:06.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:15:06.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:15:06.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:15:06.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:15:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:15:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:15:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:15:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:15:06.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:15:07.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25664 4.57 MB/sec warmup 34 sec latency 3783.775 ms 2024-04-02T17:15:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:07 smithi154 ceph-mon[36479]: pgmap v449: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.4 MiB/s wr, 115 op/s 2024-04-02T17:15:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:07 smithi059 ceph-mon[31664]: pgmap v449: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.4 MiB/s wr, 115 op/s 2024-04-02T17:15:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:07 smithi164 ceph-mon[36499]: pgmap v449: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.4 MiB/s wr, 115 op/s 2024-04-02T17:15:08.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25664 4.44 MB/sec warmup 35 sec latency 4783.871 ms 2024-04-02T17:15:08.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:08 smithi154 ceph-mon[36479]: pgmap v450: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-04-02T17:15:08.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:08 smithi059 ceph-mon[31664]: pgmap v450: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-04-02T17:15:08.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:08 smithi164 ceph-mon[36499]: pgmap v450: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-04-02T17:15:09.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25787 4.32 MB/sec warmup 36 sec latency 4905.822 ms 2024-04-02T17:15:10.033 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25787 4.20 MB/sec warmup 37 sec latency 1732.446 ms 2024-04-02T17:15:11.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25787 4.09 MB/sec warmup 38 sec latency 2732.591 ms 2024-04-02T17:15:11.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:10 smithi059 ceph-mon[31664]: pgmap v451: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-04-02T17:15:11.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:10 smithi164 ceph-mon[36499]: pgmap v451: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-04-02T17:15:11.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:10 smithi154 ceph-mon[36479]: pgmap v451: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-04-02T17:15:12.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25787 3.99 MB/sec warmup 39 sec latency 3732.757 ms 2024-04-02T17:15:13.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 25787 3.89 MB/sec warmup 40 sec latency 4732.896 ms 2024-04-02T17:15:13.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:12 smithi059 ceph-mon[31664]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-04-02T17:15:13.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:12 smithi164 ceph-mon[36499]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-04-02T17:15:13.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:12 smithi154 ceph-mon[36479]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-04-02T17:15:14.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 27283 3.95 MB/sec warmup 41 sec latency 4851.700 ms 2024-04-02T17:15:15.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 28638 4.13 MB/sec warmup 42 sec latency 20.855 ms 2024-04-02T17:15:15.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:14 smithi059 ceph-mon[31664]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 628 KiB/s wr, 60 op/s 2024-04-02T17:15:15.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:14 smithi164 ceph-mon[36499]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 628 KiB/s wr, 60 op/s 2024-04-02T17:15:15.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:14 smithi154 ceph-mon[36479]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 628 KiB/s wr, 60 op/s 2024-04-02T17:15:15.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T17:15:15.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T17:15:16.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 28947 4.05 MB/sec warmup 43 sec latency 18.717 ms 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 -- 172.21.15.59:0/1261126699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 msgr2=0x7f9f0c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 --2- 172.21.15.59:0/1261126699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1103c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009be0 tx=0x7f9f0402bef0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 -- 172.21.15.59:0/1261126699 shutdown_connections 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 --2- 172.21.15.59:0/1261126699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1103c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 --2- 172.21.15.59:0/1261126699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 --2- 172.21.15.59:0/1261126699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.205+0000 7f9f13000640 1 -- 172.21.15.59:0/1261126699 >> 172.21.15.59:0/1261126699 conn(0x7f9f0c076f40 msgr2=0x7f9f0c079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.206+0000 7f9f13000640 1 -- 172.21.15.59:0/1261126699 shutdown_connections 2024-04-02T17:15:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.206+0000 7f9f13000640 1 -- 172.21.15.59:0/1261126699 wait complete. 2024-04-02T17:15:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 Processor -- start 2024-04-02T17:15:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 -- start start 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c113c10 con 0x7f9f0c107f80 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c113a90 con 0x7f9f0c109b30 2024-04-02T17:15:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.207+0000 7f9f13000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c113d90 con 0x7f9f0c109170 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46876/0 (socket says 172.21.15.59:46876) 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 -- 172.21.15.59:0/4136495437 learned_addr learned my addr 172.21.15.59:0/4136495437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11c00640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f12600640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 -- 172.21.15.59:0/4136495437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 msgr2=0x7f9f0c1a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 -- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 msgr2=0x7f9f0c19e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c19e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11200640 1 -- 172.21.15.59:0/4136495437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c1a5e60 con 0x7f9f0c109170 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f11c00640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.208+0000 7f9f12600640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f11200640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9f0 tx=0x7f9ef800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f0a000640 1 -- 172.21.15.59:0/4136495437 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800cd80 con 0x7f9f0c109170 2024-04-02T17:15:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f0a000640 1 -- 172.21.15.59:0/4136495437 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef8005500 con 0x7f9f0c109170 2024-04-02T17:15:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f0a000640 1 -- 172.21.15.59:0/4136495437 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8010690 con 0x7f9f0c109170 2024-04-02T17:15:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c1a6150 con 0x7f9f0c109170 2024-04-02T17:15:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.209+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c1a6650 con 0x7f9f0c109170 2024-04-02T17:15:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.213+0000 7f9f0a000640 1 -- 172.21.15.59:0/4136495437 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ef80046c0 con 0x7f9f0c109170 2024-04-02T17:15:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 Processor -- start 2024-04-02T17:15:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 -- start start 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 0x55ad33eac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eaf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad33e0cd70 con 0x55ad33eacc60 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad33db52a0 con 0x55ad33ea7100 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.214+0000 7f9f1738d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad33e08d20 con 0x55ad33ea7af0 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f11c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 0x55ad33eac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53990/0 (socket says 172.21.15.59:53990) 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 -- 172.21.15.59:0/3455371983 learned_addr learned my addr 172.21.15.59:0/3455371983 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f11c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 0x55ad33eac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46892/0 (socket says 172.21.15.59:46892) 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f11200640 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eaf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 -- 172.21.15.59:0/3455371983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 msgr2=0x55ad33eac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 0x55ad33eac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 -- 172.21.15.59:0/3455371983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 msgr2=0x55ad33eaf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eaf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f9f12600640 1 -- 172.21.15.59:0/3455371983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad33dc9840 con 0x55ad33ea7100 2024-04-02T17:15:16.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 -- 172.21.15.59:0/4004213796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 msgr2=0x7f4c94101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 --2- 172.21.15.59:0/4004213796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94101e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8402bc30 comp rx=0 tx=0).stop 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 -- 172.21.15.59:0/4004213796 shutdown_connections 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 --2- 172.21.15.59:0/4004213796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c94102bf0 0x7f4c94103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 --2- 172.21.15.59:0/4004213796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c941035b0 0x7f4c94109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 --2- 172.21.15.59:0/4004213796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94101e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.215+0000 7f4c93200640 1 -- 172.21.15.59:0/4004213796 >> 172.21.15.59:0/4004213796 conn(0x7f4c940fcf70 msgr2=0x7f4c940ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9f0a000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 0x7f9ef40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9f12600640 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f04034fd0 tx=0x7f9f04009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880654d40f3f585a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9f0a000640 1 -- 172.21.15.59:0/4136495437 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9ef80d0990 con 0x7f9f0c109170 2024-04-02T17:15:16.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f4c93200640 1 -- 172.21.15.59:0/4004213796 shutdown_connections 2024-04-02T17:15:16.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f4c93200640 1 -- 172.21.15.59:0/4004213796 wait complete. 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9f11c00640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 0x7f9ef40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9efea00640 1 -- 172.21.15.59:0/3455371983 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04043070 con 0x55ad33ea7100 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9efea00640 1 -- 172.21.15.59:0/3455371983 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f04033cd0 con 0x55ad33ea7100 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.216+0000 7f9efea00640 1 -- 172.21.15.59:0/3455371983 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0402ccb0 con 0x55ad33ea7100 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 -- 172.21.15.59:0/3455371983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 msgr2=0x55ad33ea7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f04034fd0 tx=0x7f9f04009300 comp rx=0 tx=0).stop 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f11c00640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 0x7f9ef40a60e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55ad33ea7ef0 tx=0x7f9f00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 -- 172.21.15.59:0/3455371983 shutdown_connections 2024-04-02T17:15:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eaf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7af0 0x55ad33eac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f4c93200640 1 Processor -- start 2024-04-02T17:15:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 --2- 172.21.15.59:0/3455371983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7100 0x55ad33ea7520 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.217+0000 7f9f1738d740 1 -- 172.21.15.59:0/3455371983 >> 172.21.15.59:0/3455371983 conn(0x55ad33ea2670 msgr2=0x55ad33ea2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 -- start start 2024-04-02T17:15:16.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 0x7f4c9419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f9f1738d740 1 -- 172.21.15.59:0/3455371983 shutdown_connections 2024-04-02T17:15:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f9f1738d740 1 -- 172.21.15.59:0/3455371983 wait complete. 2024-04-02T17:15:16.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c91400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 0x7f4c9419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46908/0 (socket says 172.21.15.59:46908) 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c91400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 0x7f4c9419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54004/0 (socket says 172.21.15.59:54004) 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c92800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c92800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50734/0 (socket says 172.21.15.59:50734) 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410d6a0 con 0x7f4c941035b0 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c91e00640 1 -- 172.21.15.59:0/1083005265 learned_addr learned my addr 172.21.15.59:0/1083005265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410d520 con 0x7f4c94102bf0 2024-04-02T17:15:16.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.218+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9410d820 con 0x7f4c94101a00 2024-04-02T17:15:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f9f1738d740 1 -- start start 2024-04-02T17:15:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f9f1738d740 1 Processor -- start 2024-04-02T17:15:16.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c91e00640 1 -- 172.21.15.59:0/1083005265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 msgr2=0x7f4c9419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c91e00640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 0x7f4c9419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c91e00640 1 -- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 msgr2=0x7f4c941a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c91e00640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c91e00640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c941a1b40 con 0x7f4c94101a00 2024-04-02T17:15:16.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.219+0000 7f4c92800640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:16.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f4c91e00640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c84000c00 tx=0x7f4c8402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7100 0x55ad33eb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad33eafbb0 con 0x55ad33eacc60 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad33dc8660 con 0x55ad33ea7af0 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad33eafd30 con 0x55ad33ea7100 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f1738d740 10 client.0 _ll_register_callbacks cb 0x55ad33dc8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f11200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f11200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54012/0 (socket says 172.21.15.59:54012) 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f9f11200640 1 -- 172.21.15.59:0/2687768867 learned_addr learned my addr 172.21.15.59:0/2687768867 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11c00640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7100 0x55ad33eb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f4c8aa00640 1 -- 172.21.15.59:0/1083005265 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84043020 con 0x7f4c94101a00 2024-04-02T17:15:16.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f4c8aa00640 1 -- 172.21.15.59:0/1083005265 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840054e0 con 0x7f4c94101a00 2024-04-02T17:15:16.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.220+0000 7f4c8aa00640 1 -- 172.21.15.59:0/1083005265 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84003ce0 con 0x7f4c94101a00 2024-04-02T17:15:16.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c941a1dd0 con 0x7f4c94101a00 2024-04-02T17:15:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f12600640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c94104140 con 0x7f4c94101a00 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11200640 1 -- 172.21.15.59:0/2687768867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7100 msgr2=0x55ad33eb1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11200640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7100 0x55ad33eb1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11200640 1 -- 172.21.15.59:0/2687768867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 msgr2=0x55ad33eb8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11200640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f11200640 1 -- 172.21.15.59:0/2687768867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad33dd7e80 con 0x55ad33ea7af0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.221+0000 7f9f12600640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.222+0000 7f9f11200640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8099010 tx=0x7f9ef809c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79e78e84bb1084f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.222+0000 7f9ef3400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.222+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80a0f60 con 0x55ad33ea7af0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.222+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9ef80bd0c0 con 0x55ad33ea7af0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.222+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04033cd0 con 0x55ad33ea7af0 2024-04-02T17:15:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9ef80a5820 con 0x55ad33ea7af0 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9f1738d740 10 client.26006 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ad33b21210 con 0x55ad33ea7af0 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f04033cd0 con 0x55ad33ea7af0 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f9ef809c120 con 0x55ad33ea7af0 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9f1738d740 10 client.26006 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.223+0000 7f9f1738d740 10 client.26006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f9f1738d740 20 client.26006 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33ebd480 0x55ad33ebf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f9f1738d740 4 client.26006 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad33dbccd0 con 0x55ad33ebd480 2024-04-02T17:15:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f9f11c00640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33ebd480 0x55ad33ebf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.224+0000 7f4c8aa00640 1 -- 172.21.15.59:0/1083005265 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c84046020 con 0x7f4c94101a00 2024-04-02T17:15:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f9f11c00640 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33ebd480 0x55ad33ebf8a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05d0933f69dbba9 server_cookie=885565ba9bc02ecc in_seq=0 out_seq=0 2024-04-02T17:15:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f9ef3400640 10 client.26006 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f4c9bd7e740 1 Processor -- start 2024-04-02T17:15:16.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f4c9bd7e740 1 -- start start 2024-04-02T17:15:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f9ef3400640 1 -- 172.21.15.59:0/2687768867 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad33dbccd0 con 0x55ad33ebd480 2024-04-02T17:15:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.225+0000 7f9ef3400640 10 client.26006 handle_command_reply: tid=0 2024-04-02T17:15:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 client.26006 shutdown 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccde970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5557fcc3b770 con 0x5557fccdc4e0 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557fcbe3700 con 0x5557fccd6980 2024-04-02T17:15:16.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fcc37720 con 0x5557fccd7370 2024-04-02T17:15:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 msgr2=0x55ad33eb1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8099010 tx=0x7f9ef809c5d0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c8aa00640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 0x7f4c780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 shutdown_connections 2024-04-02T17:15:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33ebd480 0x55ad33ebf8a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33ea7af0 0x55ad33eb1b50 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8099010 tx=0x7f9ef809c5d0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 --2- 172.21.15.59:0/2687768867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33ea7100 0x55ad33eb1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 >> 172.21.15.59:0/2687768867 conn(0x55ad33ea2670 msgr2=0x55ad33ea1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f9ef3400640 0 client.26006 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46926/0 (socket says 172.21.15.59:46926) 2024-04-02T17:15:16.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c91e00640 1 -- 172.21.15.59:0/3674059151 learned_addr learned my addr 172.21.15.59:0/3674059151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.226+0000 7f4c92800640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91400640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccde970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c92800640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 0x7f4c780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c8aa00640 1 -- 172.21.15.59:0/1083005265 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4c840f2580 con 0x7f4c94101a00 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91e00640 1 -- 172.21.15.59:0/3674059151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 msgr2=0x5557fccd6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91e00640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91e00640 1 -- 172.21.15.59:0/3674059151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 msgr2=0x5557fccde970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91e00640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccde970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91e00640 1 -- 172.21.15.59:0/3674059151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fca197d0 con 0x5557fccd7370 2024-04-02T17:15:16.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f4c91400640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccde970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 shutdown_connections 2024-04-02T17:15:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f9f1738d740 1 -- 172.21.15.59:0/2687768867 wait complete. 2024-04-02T17:15:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.227+0000 7f9f1738d740 20 client.26006 trim_cache size 0 max 16384 2024-04-02T17:15:16.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.228+0000 7f4c91e00640 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c8404e000 tx=0x7f4c8402c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fbf47c32f1d1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.228+0000 7f4c92800640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 0x7f4c780a60f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5557fccd7bd0 tx=0x7f4c8c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.228+0000 7f4c7f400640 1 -- 172.21.15.59:0/3674059151 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840be100 con 0x5557fccd7370 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.228+0000 7f4c7f400640 1 -- 172.21.15.59:0/3674059151 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84004c80 con 0x5557fccd7370 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.228+0000 7f4c7f400640 1 -- 172.21.15.59:0/3674059151 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840c7410 con 0x5557fccd7370 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3674059151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 msgr2=0x5557fccdbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c8404e000 tx=0x7f4c8402c8a0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3674059151 shutdown_connections 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccde970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fccdbe70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3674059151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3674059151 >> 172.21.15.59:0/3674059151 conn(0x5557fccd1ef0 msgr2=0x5557fccd22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3674059151 shutdown_connections 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.229+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3674059151 wait complete. 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.230+0000 7f4c9bd7e740 1 -- start start 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.230+0000 7f4c9bd7e740 1 Processor -- start 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 0x5557fcce0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccdc4e0 0x5557fcce8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c91400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5557fccdf250 con 0x5557fccd6980 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c91400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54036/0 (socket says 172.21.15.59:54036) 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c91400640 1 -- 172.21.15.59:0/2718130141 learned_addr learned my addr 172.21.15.59:0/2718130141 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557fcbf6ac0 con 0x5557fccd7370 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557fccdf3d0 con 0x5557fccdc4e0 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 0x5557fcce0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c9bd7e740 10 client.0 _ll_register_callbacks cb 0x5557fcbf9180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:16.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.231+0000 7f4c92800640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccdc4e0 0x5557fcce8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 -- 172.21.15.59:0/2718130141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccdc4e0 msgr2=0x5557fcce8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccdc4e0 0x5557fcce8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 -- 172.21.15.59:0/2718130141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 msgr2=0x5557fcce0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 0x5557fcce0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fc94f3f0 con 0x5557fccd7370 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 0x5557fcce0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c91400640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c800136e0 tx=0x7f4c80013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e139d18bb39e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c73e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.232+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800119c0 con 0x5557fccd7370 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.233+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4c80011b60 con 0x5557fccd7370 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.233+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84004c80 con 0x5557fccd7370 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.233+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4c80011d60 con 0x5557fccd7370 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.233+0000 7f4c9bd7e740 10 client.26012 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:16.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.233+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557fc90f0f0 con 0x5557fccd7370 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84004c80 con 0x5557fccd7370 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f4c8001a070 con 0x5557fccd7370 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 10 client.26012 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 20 client.26012 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fccec6e0 0x5557fcceeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 4 client.26012 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.234+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557fcbdc630 con 0x5557fccec6e0 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.235+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fccec6e0 0x5557fcceeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.236+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fccec6e0 0x5557fcceeba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8e5d2426d1075b server_cookie=d2949487e6f7efce in_seq=0 out_seq=0 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.236+0000 7f4c73e00640 10 client.26012 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.238+0000 7f4c73e00640 1 -- 172.21.15.59:0/2718130141 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557fcbdc630 con 0x5557fccec6e0 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.238+0000 7f4c73e00640 10 client.26012 handle_command_reply: tid=0 2024-04-02T17:15:16.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.238+0000 7f4c9bd7e740 1 client.26012 shutdown 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 msgr2=0x5557fcce1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c800136e0 tx=0x7f4c80013ba0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 shutdown_connections 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fccec6e0 0x5557fcceeba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccdc4e0 0x5557fcce8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd7370 0x5557fcce1100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2718130141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd6980 0x5557fcce0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 >> 172.21.15.59:0/2718130141 conn(0x5557fccd1ef0 msgr2=0x5557fccd3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c73e00640 0 client.26012 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 shutdown_connections 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.239+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2718130141 wait complete. 2024-04-02T17:15:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.240+0000 7f4c9bd7e740 20 client.26012 trim_cache size 0 max 16384 2024-04-02T17:15:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 Processor -- start 2024-04-02T17:15:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 -- start start 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33ea9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1af5f0 con 0x55ad33ea7100 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1b11f0 con 0x55ad33eb07e0 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.252+0000 7f9f1738d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1afcc0 con 0x55ad33eacc60 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f11c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33ea9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54044/0 (socket says 172.21.15.59:54044) 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 -- 172.21.15.59:0/433109830 learned_addr learned my addr 172.21.15.59:0/433109830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f11200640 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 -- 172.21.15.59:0/433109830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 msgr2=0x55ad33eb8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 -- 172.21.15.59:0/433109830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 msgr2=0x55ad33ea9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33ea9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.253+0000 7f9f12600640 1 -- 172.21.15.59:0/433109830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad33bed620 con 0x55ad33eb07e0 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.254+0000 7f9f12600640 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009370 tx=0x7f9f04009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1560e843c4de473d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.254+0000 7f9efe000640 1 -- 172.21.15.59:0/433109830 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04043040 con 0x55ad33eb07e0 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.254+0000 7f9efe000640 1 -- 172.21.15.59:0/433109830 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f04003e50 con 0x55ad33eb07e0 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.254+0000 7f9efe000640 1 -- 172.21.15.59:0/433109830 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04004040 con 0x55ad33eb07e0 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 -- 172.21.15.59:0/433109830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 msgr2=0x55ad33ea9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009370 tx=0x7f9f04009500 comp rx=0 tx=0).stop 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 -- 172.21.15.59:0/433109830 shutdown_connections 2024-04-02T17:15:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33ea9570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 --2- 172.21.15.59:0/433109830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33ea9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 -- 172.21.15.59:0/433109830 >> 172.21.15.59:0/433109830 conn(0x55ad33ea2670 msgr2=0x55ad33eb00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 -- 172.21.15.59:0/433109830 shutdown_connections 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.255+0000 7f9f1738d740 1 -- 172.21.15.59:0/433109830 wait complete. 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 -- start start 2024-04-02T17:15:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 Processor -- start 2024-04-02T17:15:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33eb5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33eb74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1b0a60 con 0x55ad33ea7100 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1b0390 con 0x55ad33eb07e0 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ad5d0 con 0x55ad33eacc60 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.256+0000 7f9f1738d740 10 client.0 _ll_register_callbacks cb 0x7f9ef809c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33eb5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46954/0 (socket says 172.21.15.59:46954) 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33eb74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 -- 172.21.15.59:0/1809336949 learned_addr learned my addr 172.21.15.59:0/1809336949 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 -- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 msgr2=0x55ad33eb74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33eb74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 -- 172.21.15.59:0/1809336949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 msgr2=0x55ad33eb5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33eb5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 -- 172.21.15.59:0/1809336949 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad33eb7a90 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9f11200640 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ef809c8b0 tx=0x7f9ef809c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53de2cf2ef6d7a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.257+0000 7f9ef3400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef809c6f0 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9ef809c000 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04003e50 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9ef80a0f60 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 10 client.26044 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ad33eb8e10 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f04003e50 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f9ef8004f40 con 0x55ad33eacc60 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 10 client.26044 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 20 client.26044 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33f24060 0x55ad33f26480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 4 client.26044 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.258+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad33e1a2b0 con 0x55ad33f24060 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.259+0000 7f9f11c00640 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33f24060 0x55ad33f26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.259+0000 7f9f11c00640 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33f24060 0x55ad33f26480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8ea691207ab4a2 server_cookie=3a943b8ac95423d in_seq=0 out_seq=0 2024-04-02T17:15:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.259+0000 7f9ef3400640 10 client.26044 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 Processor -- start 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 -- start start 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccd1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941af560 con 0x5557fccdc4e0 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941b1160 con 0x5557fccd6980 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941afc30 con 0x5557fccd7370 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.266+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46956/0 (socket says 172.21.15.59:46956) 2024-04-02T17:15:16.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 -- 172.21.15.59:0/3632511830 learned_addr learned my addr 172.21.15.59:0/3632511830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c92800640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccd1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 -- 172.21.15.59:0/3632511830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 msgr2=0x5557fccd22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 -- 172.21.15.59:0/3632511830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 msgr2=0x5557fccd1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccd1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91400640 1 -- 172.21.15.59:0/3632511830 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fcce1bb0 con 0x5557fccd7370 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c91e00640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.267+0000 7f4c92800640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccd1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.268+0000 7f4c91400640 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c80009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba7edaf6d2ce7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.268+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3632511830 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80026530 con 0x5557fccd7370 2024-04-02T17:15:16.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.269+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3632511830 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c800119c0 con 0x5557fccd7370 2024-04-02T17:15:16.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.269+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3632511830 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004680 con 0x5557fccd7370 2024-04-02T17:15:16.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3632511830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 msgr2=0x5557fcce8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c80009640 comp rx=0 tx=0).stop 2024-04-02T17:15:16.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3632511830 shutdown_connections 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccdc4e0 0x5557fccd1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd7370 0x5557fcce8100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/3632511830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccd6980 0x5557fccd22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3632511830 >> 172.21.15.59:0/3632511830 conn(0x5557fccd1ef0 msgr2=0x5557fccd0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.270+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3632511830 shutdown_connections 2024-04-02T17:15:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.271+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/3632511830 wait complete. 2024-04-02T17:15:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.271+0000 7f4c9bd7e740 1 -- start start 2024-04-02T17:15:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.271+0000 7f4c9bd7e740 1 Processor -- start 2024-04-02T17:15:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.273+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 0x5557fcce5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.273+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 0x5557fcce5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.273+0000 7f4c9bd7e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.273+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 0x5557fcce5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c91e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 0x5557fcce5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46970/0 (socket says 172.21.15.59:46970) 2024-04-02T17:15:16.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941b09d0 con 0x5557fccd7370 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941b0300 con 0x5557fccdc4e0 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c9bd7e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941ad540 con 0x5557fccd6980 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c9bd7e740 10 client.0 _ll_register_callbacks cb 0x7f4c800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c92800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c92800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54072/0 (socket says 172.21.15.59:54072) 2024-04-02T17:15:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.274+0000 7f4c92800640 1 -- 172.21.15.59:0/2899832937 learned_addr learned my addr 172.21.15.59:0/2899832937 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 -- 172.21.15.59:0/2899832937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 msgr2=0x5557fcce5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 0x5557fcce5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 -- 172.21.15.59:0/2899832937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 msgr2=0x5557fcce5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c91400640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 0x5557fcce5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 0x5557fcce5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 -- 172.21.15.59:0/2899832937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fcce6e70 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c91400640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 0x5557fcce5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c92800640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00de60 tx=0x7f4c8c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157303680dfd8bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.275+0000 7f4c73e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c004b20 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4c8c01b070 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c800119c0 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4c8c004cc0 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557fcce8170 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c800119c0 con 0x5557fccdc4e0 2024-04-02T17:15:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f4c8c005320 con 0x5557fccdc4e0 2024-04-02T17:15:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 20 client.26021 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fcd53380 0x5557fcd557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.276+0000 7f4c9bd7e740 4 client.26021 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T17:15:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.277+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557fcc48cb0 con 0x5557fcd53380 2024-04-02T17:15:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.277+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fcd53380 0x5557fcd557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.277+0000 7f4c91e00640 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fcd53380 0x5557fcd557a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb9b2aba81e7a8c server_cookie=e0733470552630bd in_seq=0 out_seq=0 2024-04-02T17:15:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.277+0000 7f4c73e00640 10 client.26021 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9ef3400640 1 -- 172.21.15.59:0/1809336949 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad33e1a2b0 con 0x55ad33f24060 2024-04-02T17:15:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9ef3400640 10 client.26044 handle_command_reply: tid=0 2024-04-02T17:15:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 client.26044 shutdown 2024-04-02T17:15:16.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c73e00640 1 -- 172.21.15.59:0/2899832937 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5557fcc48cb0 con 0x5557fcd53380 2024-04-02T17:15:16.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c73e00640 10 client.26021 handle_command_reply: tid=0 2024-04-02T17:15:16.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 client.26021 shutdown 2024-04-02T17:15:16.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 msgr2=0x5557fcce6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00de60 tx=0x7f4c8c004ab0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 msgr2=0x55ad33eb6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ef809c8b0 tx=0x7f9ef809c640 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 shutdown_connections 2024-04-02T17:15:16.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557fcd53380 0x5557fcd557a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557fccd7370 0x5557fcce5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557fccdc4e0 0x5557fcce6880 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 --2- 172.21.15.59:0/2899832937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557fccd6980 0x5557fcce5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 shutdown_connections 2024-04-02T17:15:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad33f24060 0x55ad33f26480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad33eb07e0 0x55ad33eb74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad33eacc60 0x55ad33eb6490 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 --2- 172.21.15.59:0/1809336949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad33ea7100 0x55ad33eb5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 >> 172.21.15.59:0/1809336949 conn(0x55ad33ea2670 msgr2=0x55ad33eb15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c73e00640 0 client.26021 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 >> 172.21.15.59:0/2899832937 conn(0x5557fccd1ef0 msgr2=0x5557fccd5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.283+0000 7f9ef3400640 0 client.26044 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 shutdown_connections 2024-04-02T17:15:16.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 shutdown_connections 2024-04-02T17:15:16.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c9bd7e740 1 -- 172.21.15.59:0/2899832937 wait complete. 2024-04-02T17:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f9f1738d740 1 -- 172.21.15.59:0/1809336949 wait complete. 2024-04-02T17:15:16.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c9bd7e740 20 client.26021 trim_cache size 0 max 16384 2024-04-02T17:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f9f1738d740 20 client.26044 trim_cache size 0 max 16384 2024-04-02T17:15:16.288 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:15:16.288 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-02T17:15:16.288 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-02T17:15:16.288 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:15:16.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 msgr2=0x7f4c780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 0x7f4c780a60f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5557fccd7bd0 tx=0x7f4c8c005bd0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 msgr2=0x7f4c94199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.284+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c84000c00 tx=0x7f4c8402c6c0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 msgr2=0x7f9ef40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 0x7f9ef40a60e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55ad33ea7ef0 tx=0x7f9f00009260 comp rx=0 tx=0).stop 2024-04-02T17:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 msgr2=0x7f9f0c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9f0 tx=0x7f9ef800edf0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 shutdown_connections 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4c780a3c30 0x7f4c780a60f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c941035b0 0x7f4c941a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c94102bf0 0x7f4c9419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 --2- 172.21.15.59:0/1083005265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c94101a00 0x7f4c94199f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 >> 172.21.15.59:0/1083005265 conn(0x7f4c940fcf70 msgr2=0x7f4c94073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 shutdown_connections 2024-04-02T17:15:16.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f4c93200640 1 -- 172.21.15.59:0/1083005265 wait complete. 2024-04-02T17:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 shutdown_connections 2024-04-02T17:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f0c109b30 0x7f9f0c1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f0c109170 0x7f9f0c19e780 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9ef40a3c20 0x7f9ef40a60e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 --2- 172.21.15.59:0/4136495437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c107f80 0x7f9f0c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 >> 172.21.15.59:0/4136495437 conn(0x7f9f0c076f40 msgr2=0x7f9f0c10c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 shutdown_connections 2024-04-02T17:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:16.285+0000 7f9f13000640 1 -- 172.21.15.59:0/4136495437 wait complete. 2024-04-02T17:15:16.302 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T17:15:16.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:15:16.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 -- 172.21.15.59:0/4099911759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0101b40 msgr2=0x7f6ae0101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 --2- 172.21.15.59:0/4099911759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0101b40 0x7f6ae0101f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009900 tx=0x7f6ad002bc50 comp rx=0 tx=0).stop 2024-04-02T17:15:16.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 -- 172.21.15.59:0/4099911759 shutdown_connections 2024-04-02T17:15:16.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 --2- 172.21.15.59:0/4099911759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 0x7f6ae0109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 --2- 172.21.15.59:0/4099911759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0102d30 0x7f6ae01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 --2- 172.21.15.59:0/4099911759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0101b40 0x7f6ae0101f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 -- 172.21.15.59:0/4099911759 >> 172.21.15.59:0/4099911759 conn(0x7f6ae00fd0b0 msgr2=0x7f6ae00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 -- 172.21.15.59:0/4099911759 shutdown_connections 2024-04-02T17:15:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.558+0000 7f6ae5600640 1 -- 172.21.15.59:0/4099911759 wait complete. 2024-04-02T17:15:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.559+0000 7f6ae5600640 1 Processor -- start 2024-04-02T17:15:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.559+0000 7f6ae5600640 1 -- start start 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0102d30 0x7f6ae019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 0x7f6ae01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae010d790 con 0x7f6ae0102d30 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae010d610 con 0x7f6ae01036f0 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ae5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae010d910 con 0x7f6ae0101b40 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6ad6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0102d30 0x7f6ae019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46986/0 (socket says 172.21.15.59:46986) 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 -- 172.21.15.59:0/2909188224 learned_addr learned my addr 172.21.15.59:0/2909188224 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adf400640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 0x7f6ae01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 -- 172.21.15.59:0/2909188224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 msgr2=0x7f6ae01a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 0x7f6ae01a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 -- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0102d30 msgr2=0x7f6ae019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0102d30 0x7f6ae019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.560+0000 7f6adea00640 1 -- 172.21.15.59:0/2909188224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae01a1d80 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6adea00640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0042020 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad0004450 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae01a2010 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0003df0 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.561+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae01a2560 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.562+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae01a2760 con 0x7f6ae0101b40 2024-04-02T17:15:16.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.564+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ad0045020 con 0x7f6ae0101b40 2024-04-02T17:15:16.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.566+0000 7f6ad7e00640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 0x7f6ac40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:16.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.566+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6ad00f2400 con 0x7f6ae0101b40 2024-04-02T17:15:16.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.566+0000 7f6ad6000640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 0x7f6ac40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:16.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.567+0000 7f6ad6000640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 0x7f6ac40a6070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80095f0 tx=0x7f6ac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:16.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.569+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6ad004c020 con 0x7f6ae0101b40 2024-04-02T17:15:16.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.823+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ae01a2a10 con 0x7f6ae0101b40 2024-04-02T17:15:16.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.824+0000 7f6ad7e00640 1 -- 172.21.15.59:0/2909188224 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11938 (secure 0 0 0) 0x7f6ad00bedb0 con 0x7f6ae0101b40 2024-04-02T17:15:16.825 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:15:16.825 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:14:51.063784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2083325907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2083325907},{"type":"v1","addr":"172.21.15.164:6835","nonce":2083325907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":68,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24560,24584,24547,24530,24529]},"id":1}]} 2024-04-02T17:15:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 31 2024-04-02T17:15:16.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 msgr2=0x7f6ac40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 0x7f6ac40a6070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80095f0 tx=0x7f6ac8005f90 comp rx=0 tx=0).stop 2024-04-02T17:15:16.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 msgr2=0x7f6ae019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:16.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad002c660 comp rx=0 tx=0).stop 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 shutdown_connections 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6ac40a3bb0 0x7f6ac40a6070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ae0102d30 0x7f6ae019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae01036f0 0x7f6ae01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 --2- 172.21.15.59:0/2909188224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ae0101b40 0x7f6ae019a1b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 >> 172.21.15.59:0/2909188224 conn(0x7f6ae00fd0b0 msgr2=0x7f6ae0073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 shutdown_connections 2024-04-02T17:15:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:16.828+0000 7f6ae5600640 1 -- 172.21.15.59:0/2909188224 wait complete. 2024-04-02T17:15:16.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T17:15:16.928 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:16 smithi059 ceph-mon[31664]: pgmap v454: 97 pgs: 97 active+clean; 26 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 229 KiB/s wr, 49 op/s 2024-04-02T17:15:17.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29139 3.96 MB/sec warmup 44 sec latency 751.561 ms 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 -- 172.21.15.59:0/853694980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 msgr2=0x7ff8b0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 --2- 172.21.15.59:0/853694980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b0102070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802bc30 comp rx=0 tx=0).stop 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 -- 172.21.15.59:0/853694980 shutdown_connections 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 --2- 172.21.15.59:0/853694980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 0x7ff8b0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 --2- 172.21.15.59:0/853694980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b0102070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 --2- 172.21.15.59:0/853694980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 -- 172.21.15.59:0/853694980 >> 172.21.15.59:0/853694980 conn(0x7ff8b00fbf90 msgr2=0x7ff8b00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 -- 172.21.15.59:0/853694980 shutdown_connections 2024-04-02T17:15:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.071+0000 7ff8b5800640 1 -- 172.21.15.59:0/853694980 wait complete. 2024-04-02T17:15:17.073 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:16 smithi164 ceph-mon[36499]: pgmap v454: 97 pgs: 97 active+clean; 26 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 229 KiB/s wr, 49 op/s 2024-04-02T17:15:17.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 Processor -- start 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 -- start start 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 0x7ff8b019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b010c6d0 con 0x7ff8b0100a00 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b010c550 con 0x7ff8b01025b0 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.073+0000 7ff8b5800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b010c850 con 0x7ff8b0101bf0 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46998/0 (socket says 172.21.15.59:46998) 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50830/0 (socket says 172.21.15.59:50830) 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 -- 172.21.15.59:0/3338198526 learned_addr learned my addr 172.21.15.59:0/3338198526 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8af400640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 0x7ff8b019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 -- 172.21.15.59:0/3338198526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 msgr2=0x7ff8b019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 0x7ff8b019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 -- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 msgr2=0x7ff8b0197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 -- 172.21.15.59:0/3338198526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019fa80 con 0x7ff8b0101bf0 2024-04-02T17:15:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.074+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff89802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.075+0000 7ff8a7400640 1 -- 172.21.15.59:0/3338198526 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898043020 con 0x7ff8b0101bf0 2024-04-02T17:15:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.075+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019fd10 con 0x7ff8b0101bf0 2024-04-02T17:15:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.075+0000 7ff8a7400640 1 -- 172.21.15.59:0/3338198526 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8980054e0 con 0x7ff8b0101bf0 2024-04-02T17:15:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.075+0000 7ff8a7400640 1 -- 172.21.15.59:0/3338198526 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898003c40 con 0x7ff8b0101bf0 2024-04-02T17:15:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.075+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01a0200 con 0x7ff8b0101bf0 2024-04-02T17:15:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.078+0000 7ff8b85d7740 1 Processor -- start 2024-04-02T17:15:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.078+0000 7ff8b85d7740 1 -- start start 2024-04-02T17:15:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8a7400640 1 -- 172.21.15.59:0/3338198526 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff898046020 con 0x7ff8b0101bf0 2024-04-02T17:15:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e21c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a01d5b400 con 0x564a01e1a660 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a01d3ff10 con 0x564a01e1f7d0 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564a01d7f6f0 con 0x564a01e19c70 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e21c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8af400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50832/0 (socket says 172.21.15.59:50832) 2024-04-02T17:15:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.079+0000 7ff8aea00640 1 -- 172.21.15.59:0/2476010693 learned_addr learned my addr 172.21.15.59:0/2476010693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 -- 172.21.15.59:0/2476010693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 msgr2=0x564a01e1a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e1a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 -- 172.21.15.59:0/2476010693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 msgr2=0x564a01e21c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e21c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 -- 172.21.15.59:0/2476010693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a01a933f0 con 0x564a01e1a660 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e21c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009c80 tx=0x7ff8a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c5c2234b8404af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff89fe00640 1 -- 172.21.15.59:0/2476010693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001e070 con 0x564a01e1a660 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff89fe00640 1 -- 172.21.15.59:0/2476010693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0009e30 con 0x564a01e1a660 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.080+0000 7ff89fe00640 1 -- 172.21.15.59:0/2476010693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000ea20 con 0x564a01e1a660 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8a7400640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8a7400640 1 -- 172.21.15.59:0/3338198526 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8980f2490 con 0x7ff8b0101bf0 2024-04-02T17:15:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8af400640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2476010693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 msgr2=0x564a01e1f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009c80 tx=0x7ff8a0011810 comp rx=0 tx=0).stop 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2476010693 shutdown_connections 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e1f160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e21c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2476010693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2476010693 >> 172.21.15.59:0/2476010693 conn(0x564a01de1a50 msgr2=0x564a01e155e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2476010693 shutdown_connections 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.081+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2476010693 wait complete. 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.082+0000 7ff8af400640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x564a01e1aec0 tx=0x7ff8a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.082+0000 7ff8b85d7740 1 -- start start 2024-04-02T17:15:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.082+0000 7ff8b85d7740 1 Processor -- start 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 0x564a01e28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a01d7b6a0 con 0x564a01e19c70 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a01d27980 con 0x564a01e1f7d0 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564a01d3acb0 con 0x564a01e1a660 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8b85d7740 10 client.0 _ll_register_callbacks cb 0x564a01d3a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50834/0 (socket says 172.21.15.59:50834) 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 -- 172.21.15.59:0/854204414 learned_addr learned my addr 172.21.15.59:0/854204414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 0x564a01e28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8af400640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 -- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 msgr2=0x564a01e28870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 0x564a01e28870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 -- 172.21.15.59:0/854204414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 msgr2=0x564a01e2f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8aea00640 1 -- 172.21.15.59:0/854204414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a01b5d7d0 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 0x564a01e28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.083+0000 7ff8af400640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff8aea00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00058c0 tx=0x7ff8a0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f695629db8cc4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff89cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001e040 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8a0004230 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a0009e30 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8a0004470 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff8b85d7740 10 client.15954 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.084+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564a01e32bb0 con 0x564a01e19c70 2024-04-02T17:15:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0009e30 con 0x564a01e19c70 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff8a0004b30 con 0x564a01e19c70 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 10 client.15954 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 10 client.15954 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 20 client.15954 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e34010 0x564a01e36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 4 client.15954 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a01d3f040 con 0x564a01e34010 2024-04-02T17:15:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.085+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e34010 0x564a01e36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.086+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e34010 0x564a01e36420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f871aee3fd73d07 server_cookie=15bf435fa8c3b010 in_seq=0 out_seq=0 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.086+0000 7ff89cc00640 10 client.15954 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff89cc00640 1 -- 172.21.15.59:0/854204414 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a01d3f040 con 0x564a01e34010 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff89cc00640 10 client.15954 handle_command_reply: tid=0 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 client.15954 shutdown 2024-04-02T17:15:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 msgr2=0x564a01e28330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00058c0 tx=0x7ff8a0005720 comp rx=0 tx=0).stop 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 shutdown_connections 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e34010 0x564a01e36420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1a660 0x564a01e28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/854204414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e19c70 0x564a01e28330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 >> 172.21.15.59:0/854204414 conn(0x564a01de1a50 msgr2=0x564a01e148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff89cc00640 0 client.15954 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 shutdown_connections 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 1 -- 172.21.15.59:0/854204414 wait complete. 2024-04-02T17:15:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.087+0000 7ff8b85d7740 20 client.15954 trim_cache size 0 max 16384 2024-04-02T17:15:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 Processor -- start 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 -- start start 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e19c70 0x564a01e3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1f7d0 0x564a01e3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a6fa0 con 0x564a01e1a660 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a8ba0 con 0x564a01e19c70 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a7670 con 0x564a01e1f7d0 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50836/0 (socket says 172.21.15.59:50836) 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8a7e00640 1 -- 172.21.15.59:0/4138754223 learned_addr learned my addr 172.21.15.59:0/4138754223 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8aea00640 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e19c70 0x564a01e3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.103+0000 7ff8af400640 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1f7d0 0x564a01e3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 -- 172.21.15.59:0/4138754223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1f7d0 msgr2=0x564a01e3b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1f7d0 0x564a01e3b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 -- 172.21.15.59:0/4138754223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e19c70 msgr2=0x564a01e3ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e19c70 0x564a01e3ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 -- 172.21.15.59:0/4138754223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a01e26610 con 0x564a01e1a660 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8980bb030 tx=0x7ff8980efda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730b846226142894 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff89f400640 1 -- 172.21.15.59:0/4138754223 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980be480 con 0x564a01e1a660 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff89f400640 1 -- 172.21.15.59:0/4138754223 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898004250 con 0x564a01e1a660 2024-04-02T17:15:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.104+0000 7ff89f400640 1 -- 172.21.15.59:0/4138754223 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980be010 con 0x564a01e1a660 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 -- 172.21.15.59:0/4138754223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 msgr2=0x564a01e2f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8980bb030 tx=0x7ff8980efda0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 -- 172.21.15.59:0/4138754223 shutdown_connections 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2f800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e1f7d0 0x564a01e3b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/4138754223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e19c70 0x564a01e3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 -- 172.21.15.59:0/4138754223 >> 172.21.15.59:0/4138754223 conn(0x564a01de1a50 msgr2=0x564a01e148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 -- 172.21.15.59:0/4138754223 shutdown_connections 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.105+0000 7ff8b85d7740 1 -- 172.21.15.59:0/4138754223 wait complete. 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 -- start start 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 Processor -- start 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a8410 con 0x564a01e1a660 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a7d40 con 0x564a01e1f7d0 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a4f80 con 0x564a01e19c70 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8b85d7740 10 client.0 _ll_register_callbacks cb 0x7ff8a8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.106+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47034/0 (socket says 172.21.15.59:47034) 2024-04-02T17:15:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8aea00640 1 -- 172.21.15.59:0/2434276036 learned_addr learned my addr 172.21.15.59:0/2434276036 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8af400640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 -- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 msgr2=0x564a01e2df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 -- 172.21.15.59:0/2434276036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 msgr2=0x564a01e2e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 -- 172.21.15.59:0/2434276036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a01e98550 con 0x564a01e1a660 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff8a7e00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2e440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x564a01e1adc0 tx=0x7ff8980f5d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e148b1a0d1ec331d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff89cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.107+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980be1f0 con 0x564a01e1a660 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff8980bc9b0 con 0x564a01e1a660 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a01d8cc30 con 0x564a01e1a660 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8980be480 con 0x564a01e1a660 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 10 client.15960 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564a01e99720 con 0x564a01e1a660 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564a01d8cc30 con 0x564a01e1a660 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff8980c6e90 con 0x564a01e1a660 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 10 client.15960 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 10 client.15960 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 20 client.15960 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e99eb0 0x564a01e9c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 4 client.15960 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.108+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 --> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a01d86f80 con 0x564a01e99eb0 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e99eb0 0x564a01e9c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff8aea00640 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e99eb0 0x564a01e9c2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c61ed30d8aec461d server_cookie=aa7343b340f8e6c5 in_seq=0 out_seq=0 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff89cc00640 10 client.15960 ms_handle_connect on v2:172.21.15.164:6836/316018252 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff89cc00640 1 -- 172.21.15.59:0/2434276036 <== mds.4 v2:172.21.15.164:6836/316018252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a01d86f80 con 0x564a01e99eb0 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff89cc00640 10 client.15960 handle_command_reply: tid=0 2024-04-02T17:15:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.109+0000 7ff8b85d7740 1 client.15960 shutdown 2024-04-02T17:15:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 msgr2=0x564a01e2e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2e440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x564a01e1adc0 tx=0x7ff8980f5d30 comp rx=0 tx=0).stop 2024-04-02T17:15:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 shutdown_connections 2024-04-02T17:15:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:6836/316018252,v1:172.21.15.164:6837/316018252] conn(0x564a01e99eb0 0x564a01e9c2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a01e1a660 0x564a01e2e440 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a01e1f7d0 0x564a01e2e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 --2- 172.21.15.59:0/2434276036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01e19c70 0x564a01e2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff89cc00640 0 client.15960 ms_handle_reset on v2:172.21.15.164:6836/316018252 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 >> 172.21.15.59:0/2434276036 conn(0x564a01de1a50 msgr2=0x564a01e18820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 shutdown_connections 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 1 -- 172.21.15.59:0/2434276036 wait complete. 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.110+0000 7ff8b85d7740 20 client.15960 trim_cache size 0 max 16384 2024-04-02T17:15:17.116 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:15:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 msgr2=0x7ff8940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x564a01e1aec0 tx=0x7ff8a8009500 comp rx=0 tx=0).stop 2024-04-02T17:15:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 msgr2=0x7ff8b01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff89802c650 comp rx=0 tx=0).stop 2024-04-02T17:15:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 shutdown_connections 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8b01025b0 0x7ff8b019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101bf0 0x7ff8b01983a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 --2- 172.21.15.59:0/3338198526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b0100a00 0x7ff8b0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 >> 172.21.15.59:0/3338198526 conn(0x7ff8b00fbf90 msgr2=0x7ff8b00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 shutdown_connections 2024-04-02T17:15:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.111+0000 7ff8b5800640 1 -- 172.21.15.59:0/3338198526 wait complete. 2024-04-02T17:15:17.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T17:15:17.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:16 smithi154 ceph-mon[36479]: pgmap v454: 97 pgs: 97 active+clean; 26 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 229 KiB/s wr, 49 op/s 2024-04-02T17:15:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 -- 172.21.15.59:0/215505481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f4103590 msgr2=0x7f12f4109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 --2- 172.21.15.59:0/215505481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f4103590 0x7f12f4109e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009be0 tx=0x7f12ec02bef0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 -- 172.21.15.59:0/215505481 shutdown_connections 2024-04-02T17:15:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 --2- 172.21.15.59:0/215505481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f4103590 0x7f12f4109e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 --2- 172.21.15.59:0/215505481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 0x7f12f4103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 --2- 172.21.15.59:0/215505481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f41019e0 0x7f12f4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.344+0000 7f12f3e00640 1 -- 172.21.15.59:0/215505481 >> 172.21.15.59:0/215505481 conn(0x7f12f40fcf70 msgr2=0x7f12f40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.345+0000 7f12f3e00640 1 -- 172.21.15.59:0/215505481 shutdown_connections 2024-04-02T17:15:17.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.345+0000 7f12f3e00640 1 -- 172.21.15.59:0/215505481 wait complete. 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 Processor -- start 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 -- start start 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f41019e0 0x7f12f4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 0x7f12f419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410d660 con 0x7f12f4103590 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410d4e0 con 0x7f12f41019e0 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12f410d7e0 con 0x7f12f4102bd0 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f41019e0 0x7f12f4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50852/0 (socket says 172.21.15.59:50852) 2024-04-02T17:15:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f3400640 1 -- 172.21.15.59:0/927451703 learned_addr learned my addr 172.21.15.59:0/927451703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.346+0000 7f12f2000640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 0x7f12f419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 -- 172.21.15.59:0/927451703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 msgr2=0x7f12f419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 0x7f12f419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 -- 172.21.15.59:0/927451703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f41019e0 msgr2=0x7f12f4199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f41019e0 0x7f12f4199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 -- 172.21.15.59:0/927451703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f41a1a90 con 0x7f12f4103590 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.347+0000 7f12f3400640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12ec036040 tx=0x7f12ec0337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.348+0000 7f12eb400640 1 -- 172.21.15.59:0/927451703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec043020 con 0x7f12f4103590 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.348+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41a1d20 con 0x7f12f4103590 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.348+0000 7f12eb400640 1 -- 172.21.15.59:0/927451703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec02ca60 con 0x7f12f4103590 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.348+0000 7f12eb400640 1 -- 172.21.15.59:0/927451703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec034690 con 0x7f12f4103590 2024-04-02T17:15:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.348+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f41a2260 con 0x7f12f4103590 2024-04-02T17:15:17.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.351+0000 7f12eb400640 1 -- 172.21.15.59:0/927451703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f12ec046020 con 0x7f12f4103590 2024-04-02T17:15:17.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.351+0000 7f12fc363740 1 Processor -- start 2024-04-02T17:15:17.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.351+0000 7f12fc363740 1 -- start start 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.351+0000 7f12fc363740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12fc363740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12fc363740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e29e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12fc363740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a76d63400 con 0x564a76e21e20 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12fc363740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a76d47f10 con 0x564a76e22810 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12fc363740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564a76d876f0 con 0x564a76e27980 2024-04-02T17:15:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50856/0 (socket says 172.21.15.59:50856) 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2000640 1 -- 172.21.15.59:0/938460677 learned_addr learned my addr 172.21.15.59:0/938460677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2000640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e29e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 -- 172.21.15.59:0/938460677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 msgr2=0x564a76e29e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e29e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 -- 172.21.15.59:0/938460677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 msgr2=0x564a76e22240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2a00640 1 -- 172.21.15.59:0/938460677 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a76a9b3f0 con 0x564a76e22810 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12f2000640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.352+0000 7f12eb400640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 0x7f12d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12eb400640 1 -- 172.21.15.59:0/927451703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f12ec0f29f0 con 0x7f12f4103590 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12f2a00640 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e27310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005fd0 tx=0x7f12e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b129bdd794c173 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12f2000640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 0x7f12d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12df400640 1 -- 172.21.15.59:0/938460677 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e401e070 con 0x564a76e22810 2024-04-02T17:15:17.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12df400640 1 -- 172.21.15.59:0/938460677 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e4009db0 con 0x564a76e22810 2024-04-02T17:15:17.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.353+0000 7f12df400640 1 -- 172.21.15.59:0/938460677 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4019450 con 0x564a76e22810 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 -- 172.21.15.59:0/938460677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 msgr2=0x564a76e27310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12f2000640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 0x7f12d80a60f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x564a76e23070 tx=0x7f12e0009830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e27310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005fd0 tx=0x7f12e4011810 comp rx=0 tx=0).stop 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 -- 172.21.15.59:0/938460677 shutdown_connections 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e29e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e27310 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 --2- 172.21.15.59:0/938460677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e22240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 -- 172.21.15.59:0/938460677 >> 172.21.15.59:0/938460677 conn(0x564a76de9a50 msgr2=0x564a76e1d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 -- 172.21.15.59:0/938460677 shutdown_connections 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.354+0000 7f12fc363740 1 -- 172.21.15.59:0/938460677 wait complete. 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 -- start start 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 Processor -- start 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a76d836a0 con 0x564a76e21e20 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a76d2f980 con 0x564a76e22810 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564a76d42cb0 con 0x564a76e27980 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.355+0000 7f12fc363740 10 client.0 _ll_register_callbacks cb 0x564a76d426a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50858/0 (socket says 172.21.15.59:50858) 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 -- 172.21.15.59:0/1095944495 learned_addr learned my addr 172.21.15.59:0/1095944495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2000640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f3400640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 -- 172.21.15.59:0/1095944495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 msgr2=0x564a76e33610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e33610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 -- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 msgr2=0x564a76e2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e2c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 -- 172.21.15.59:0/1095944495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a76b657d0 con 0x564a76e21e20 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2000640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f3400640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f12e4011d10 tx=0x7f12e40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210014859486e43d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.356+0000 7f12d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4027020 con 0x564a76e21e20 2024-04-02T17:15:17.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e4009db0 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f12e4003d60 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12e4032050 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12fc363740 10 client.15972 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564a76e369d0 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.357+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e4009db0 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f12e4004c50 con 0x564a76e21e20 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 10 client.15972 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 10 client.15972 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 20 client.15972 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e37de0 0x564a76e3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 4 client.15972 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a76d47040 con 0x564a76e37de0 2024-04-02T17:15:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.358+0000 7f12f2000640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e37de0 0x564a76e3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.359+0000 7f12f2000640 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e37de0 0x564a76e3a210 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7fff337a974c01e server_cookie=4476d1de17be90ff in_seq=0 out_seq=0 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.359+0000 7f12d3e00640 10 client.15972 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12d3e00640 1 -- 172.21.15.59:0/1095944495 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a76d47040 con 0x564a76e37de0 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12d3e00640 10 client.15972 handle_command_reply: tid=0 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 client.15972 shutdown 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 msgr2=0x564a76e2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f12e4011d10 tx=0x7f12e40056a0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 shutdown_connections 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e37de0 0x564a76e3a210 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e27980 0x564a76e33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e22810 0x564a76e2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 --2- 172.21.15.59:0/1095944495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76e21e20 0x564a76e2c090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12d3e00640 0 client.15972 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:15:17.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 >> 172.21.15.59:0/1095944495 conn(0x564a76de9a50 msgr2=0x564a76e1caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 shutdown_connections 2024-04-02T17:15:17.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 1 -- 172.21.15.59:0/1095944495 wait complete. 2024-04-02T17:15:17.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.360+0000 7f12fc363740 20 client.15972 trim_cache size 0 max 16384 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.375+0000 7f12fc363740 1 Processor -- start 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.375+0000 7f12fc363740 1 -- start start 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e21620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41ab190 con 0x564a76de9a50 2024-04-02T17:15:17.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41acd90 con 0x564a76e21e20 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12fc363740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41ab860 con 0x564a76e22810 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47100/0 (socket says 172.21.15.59:47100) 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 -- 172.21.15.59:0/3594615918 learned_addr learned my addr 172.21.15.59:0/3594615918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f2a00640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f2000640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e21620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 -- 172.21.15.59:0/3594615918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 msgr2=0x564a76e21620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e21620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 -- 172.21.15.59:0/3594615918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 msgr2=0x564a76e210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f3400640 1 -- 172.21.15.59:0/3594615918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a76e1dcf0 con 0x564a76e22810 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.376+0000 7f12f2000640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e21620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.377+0000 7f12f3400640 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0be910 tx=0x7f12ec02bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e0f57d66a9fc19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.377+0000 7f12dea00640 1 -- 172.21.15.59:0/3594615918 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0bf310 con 0x564a76e22810 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.377+0000 7f12dea00640 1 -- 172.21.15.59:0/3594615918 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec0bec10 con 0x564a76e22810 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.377+0000 7f12dea00640 1 -- 172.21.15.59:0/3594615918 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004b40 con 0x564a76e22810 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 -- 172.21.15.59:0/3594615918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 msgr2=0x564a76e335d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0be910 tx=0x7f12ec02bef0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 -- 172.21.15.59:0/3594615918 shutdown_connections 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e335d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e21620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 --2- 172.21.15.59:0/3594615918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 -- 172.21.15.59:0/3594615918 >> 172.21.15.59:0/3594615918 conn(0x564a76e235f0 msgr2=0x564a76e209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 -- 172.21.15.59:0/3594615918 shutdown_connections 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.378+0000 7f12fc363740 1 -- 172.21.15.59:0/3594615918 wait complete. 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 -- start start 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 Processor -- start 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e30d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41ac600 con 0x564a76de9a50 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41abf30 con 0x564a76e21e20 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41a9170 con 0x564a76e22810 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12fc363740 10 client.0 _ll_register_callbacks cb 0x7f12e4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.379+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e30d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50874/0 (socket says 172.21.15.59:50874) 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2a00640 1 -- 172.21.15.59:0/1762056025 learned_addr learned my addr 172.21.15.59:0/1762056025 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f3400640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 -- 172.21.15.59:0/1762056025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 msgr2=0x564a76e31d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e31d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 -- 172.21.15.59:0/1762056025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 msgr2=0x564a76e307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 -- 172.21.15.59:0/1762056025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a76e32310 con 0x564a76e21e20 2024-04-02T17:15:17.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12f2000640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e30d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564a76d837f0 tx=0x7f12e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c8f6935045246b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.380+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0031070 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f12e000fd80 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec0bec10 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12e0004030 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12fc363740 10 client.26063 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564a76e33680 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec0bec10 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.381+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f12e00036f0 con 0x564a76e21e20 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 10 client.26063 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 10 client.26063 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 20 client.26063 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e27980 0x564a76ea0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 4 client.26063 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:15:17.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 --> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a76d94c30 con 0x564a76e27980 2024-04-02T17:15:17.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.382+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e27980 0x564a76ea0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12f2a00640 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e27980 0x564a76ea0910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7e050cf50340239 server_cookie=9895c2973de02e5a in_seq=0 out_seq=0 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12d3e00640 10 client.26063 ms_handle_connect on v2:172.21.15.154:6836/2695634895 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12d3e00640 1 -- 172.21.15.59:0/1762056025 <== mds.3 v2:172.21.15.154:6836/2695634895 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a76d94c30 con 0x564a76e27980 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12d3e00640 10 client.26063 handle_command_reply: tid=0 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 client.26063 shutdown 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 msgr2=0x564a76e30d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e30d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564a76d837f0 tx=0x7f12e0004730 comp rx=0 tx=0).stop 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 shutdown_connections 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:6836/2695634895,v1:172.21.15.154:6837/2695634895] conn(0x564a76e27980 0x564a76ea0910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a76e22810 0x564a76e31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a76e21e20 0x564a76e30d10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 --2- 172.21.15.59:0/1762056025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a76de9a50 0x564a76e307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 >> 172.21.15.59:0/1762056025 conn(0x564a76e235f0 msgr2=0x564a76e21260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12d3e00640 0 client.26063 ms_handle_reset on v2:172.21.15.154:6836/2695634895 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 shutdown_connections 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 1 -- 172.21.15.59:0/1762056025 wait complete. 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.383+0000 7f12fc363740 20 client.26063 trim_cache size 0 max 16384 2024-04-02T17:15:17.389 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 msgr2=0x7f12d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 0x7f12d80a60f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x564a76e23070 tx=0x7f12e0009830 comp rx=0 tx=0).stop 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 msgr2=0x7f12f41a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12ec036040 tx=0x7f12ec0337b0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 shutdown_connections 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f12d80a3c30 0x7f12d80a60f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12f4103590 0x7f12f41a1420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f4102bd0 0x7f12f419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 --2- 172.21.15.59:0/927451703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12f41019e0 0x7f12f4199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 >> 172.21.15.59:0/927451703 conn(0x7f12f40fcf70 msgr2=0x7f12f4073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 shutdown_connections 2024-04-02T17:15:17.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.384+0000 7f12f3e00640 1 -- 172.21.15.59:0/927451703 wait complete. 2024-04-02T17:15:17.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 -- 172.21.15.59:0/323063132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0074560 msgr2=0x7f96f00749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 --2- 172.21.15.59:0/323063132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0074560 0x7f96f00749e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009930 tx=0x7f96d802bc30 comp rx=0 tx=0).stop 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 -- 172.21.15.59:0/323063132 shutdown_connections 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 --2- 172.21.15.59:0/323063132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f010e0e0 0x7f96f01104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 --2- 172.21.15.59:0/323063132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0074560 0x7f96f00749e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 --2- 172.21.15.59:0/323063132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0075b40 0x7f96f0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 -- 172.21.15.59:0/323063132 >> 172.21.15.59:0/323063132 conn(0x7f96f00fd0a0 msgr2=0x7f96f00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 -- 172.21.15.59:0/323063132 shutdown_connections 2024-04-02T17:15:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.627+0000 7f96f7200640 1 -- 172.21.15.59:0/323063132 wait complete. 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.628+0000 7f96f7200640 1 Processor -- start 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.628+0000 7f96f7200640 1 -- start start 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.628+0000 7f96f7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.628+0000 7f96f7200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 0x7f96f019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96f7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f010e0e0 0x7f96f01a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96f7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f0113da0 con 0x7f96f0074560 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96f7200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96f0113c20 con 0x7f96f0075b40 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96f7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96f0113f20 con 0x7f96f010e0e0 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 0x7f96f019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50892/0 (socket says 172.21.15.59:50892) 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96efe00640 1 -- 172.21.15.59:0/260548524 learned_addr learned my addr 172.21.15.59:0/260548524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96f5200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f010e0e0 0x7f96f01a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 -- 172.21.15.59:0/260548524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f010e0e0 msgr2=0x7f96f01a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f010e0e0 0x7f96f01a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 -- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 msgr2=0x7f96f019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96e7400640 1 -- 172.21.15.59:0/260548524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f01a60f0 con 0x7f96f0075b40 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.629+0000 7f96efe00640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96e7400640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 0x7f96f019ea10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009230 tx=0x7f96d802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96ed600640 1 -- 172.21.15.59:0/260548524 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8041070 con 0x7f96f0075b40 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96ed600640 1 -- 172.21.15.59:0/260548524 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d8004450 con 0x7f96f0075b40 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f01a6380 con 0x7f96f0075b40 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96ed600640 1 -- 172.21.15.59:0/260548524 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8034440 con 0x7f96f0075b40 2024-04-02T17:15:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.630+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f01a6870 con 0x7f96f0075b40 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.633+0000 7f96f9e2f740 1 Processor -- start 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.633+0000 7f96ed600640 1 -- 172.21.15.59:0/260548524 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f96d8044070 con 0x7f96f0075b40 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.633+0000 7f96f9e2f740 1 -- start start 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x5622758fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562275836400 con 0x5622758fa7d0 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56227581af10 con 0x5622758f5660 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56227585a6f0 con 0x5622758f4c70 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96efe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x5622758fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96efe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54260/0 (socket says 172.21.15.59:54260) 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96f5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.634+0000 7f96efe00640 1 -- 172.21.15.59:0/203691378 learned_addr learned my addr 172.21.15.59:0/203691378 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 -- 172.21.15.59:0/203691378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 msgr2=0x5622758f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 -- 172.21.15.59:0/203691378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 msgr2=0x5622758fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x5622758fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 -- 172.21.15.59:0/203691378 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227556e3f0 con 0x5622758f5660 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96e7400640 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x5622758fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96efe00640 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96e000f8d0 tx=0x7f96e000fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f48e48c9828d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96e5600640 1 -- 172.21.15.59:0/203691378 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011c20 con 0x5622758f5660 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96e5600640 1 -- 172.21.15.59:0/203691378 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0011dc0 con 0x5622758f5660 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96e5600640 1 -- 172.21.15.59:0/203691378 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0018830 con 0x5622758f5660 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.635+0000 7f96ed600640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 0x7f96d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96ed600640 1 -- 172.21.15.59:0/260548524 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f96d80f1e50 con 0x7f96f0075b40 2024-04-02T17:15:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f5200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 0x7f96d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 -- 172.21.15.59:0/203691378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 msgr2=0x5622758fa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96e000f8d0 tx=0x7f96e000fd90 comp rx=0 tx=0).stop 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 -- 172.21.15.59:0/203691378 shutdown_connections 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x5622758fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758fa160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/203691378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 -- 172.21.15.59:0/203691378 >> 172.21.15.59:0/203691378 conn(0x5622758bca50 msgr2=0x5622758f05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 -- 172.21.15.59:0/203691378 shutdown_connections 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f5200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 0x7f96d40a6070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5622758f5ec0 tx=0x7f96e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.636+0000 7f96f9e2f740 1 -- 172.21.15.59:0/203691378 wait complete. 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 -- start start 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 Processor -- start 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x562275906460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5622758566a0 con 0x5622758fa7d0 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562275802980 con 0x5622758f5660 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.637+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562275815cb0 con 0x5622758f4c70 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96f9e2f740 10 client.0 _ll_register_callbacks cb 0x5622758156a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96e7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96f5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x562275906460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47142/0 (socket says 172.21.15.59:47142) 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 -- 172.21.15.59:0/448761087 learned_addr learned my addr 172.21.15.59:0/448761087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 -- 172.21.15.59:0/448761087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 msgr2=0x5622758ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 -- 172.21.15.59:0/448761087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 msgr2=0x562275906460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x562275906460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 -- 172.21.15.59:0/448761087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622756387d0 con 0x5622758f4c70 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96e7400640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96efe00640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009700 tx=0x7f96e0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae873813a4d0530 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.638+0000 7f96de000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0026070 con 0x5622758f4c70 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e0011dc0 con 0x5622758f4c70 2024-04-02T17:15:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f96e0004c40 con 0x5622758f4c70 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96e0004e40 con 0x5622758f4c70 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96f9e2f740 10 client.26104 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5622759097d0 con 0x5622758f4c70 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.639+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0011dc0 con 0x5622758f4c70 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f96e00187c0 con 0x5622758f4c70 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 10 client.26104 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 10 client.26104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 20 client.26104 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x56227590ac90 0x56227590d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 4 client.26104 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56227581a040 con 0x56227590ac90 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.640+0000 7f96e7400640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x56227590ac90 0x56227590d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.641+0000 7f96e7400640 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x56227590ac90 0x56227590d0c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=510886e84981526d server_cookie=a9871f5e261ee641 in_seq=0 out_seq=0 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.641+0000 7f96de000640 10 client.26104 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96de000640 1 -- 172.21.15.59:0/448761087 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56227581a040 con 0x56227590ac90 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96de000640 10 client.26104 handle_command_reply: tid=0 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 client.26104 shutdown 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 msgr2=0x5622758feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009700 tx=0x7f96e0003ba0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 shutdown_connections 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x56227590ac90 0x56227590d0c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x562275906460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96de000640 0 client.26104 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x5622758ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/448761087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x5622758feee0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 >> 172.21.15.59:0/448761087 conn(0x5622758bca50 msgr2=0x5622758ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 shutdown_connections 2024-04-02T17:15:17.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.642+0000 7f96f9e2f740 1 -- 172.21.15.59:0/448761087 wait complete. 2024-04-02T17:15:17.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.643+0000 7f96f9e2f740 20 client.26104 trim_cache size 0 max 16384 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.657+0000 7f96f9e2f740 1 Processor -- start 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 -- start start 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758f4c70 0x5622758ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f5660 0x562275906420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01af890 con 0x5622758f4c70 2024-04-02T17:15:17.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b1490 con 0x5622758fa7d0 2024-04-02T17:15:17.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01aff60 con 0x5622758f5660 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758f4c70 0x5622758ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.658+0000 7f96f5200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54274/0 (socket says 172.21.15.59:54274) 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 -- 172.21.15.59:0/99439006 learned_addr learned my addr 172.21.15.59:0/99439006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f5660 0x562275906420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 -- 172.21.15.59:0/99439006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f5660 msgr2=0x562275906420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f5660 0x562275906420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 -- 172.21.15.59:0/99439006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758f4c70 msgr2=0x5622758ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758f4c70 0x5622758ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 -- 172.21.15.59:0/99439006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622758f1330 con 0x5622758fa7d0 2024-04-02T17:15:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.659+0000 7f96f5200640 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004390 tx=0x7f96e801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cbf11a6b54a681 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96e4c00640 1 -- 172.21.15.59:0/99439006 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e802e070 con 0x5622758fa7d0 2024-04-02T17:15:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96e4c00640 1 -- 172.21.15.59:0/99439006 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e8004480 con 0x5622758fa7d0 2024-04-02T17:15:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96e4c00640 1 -- 172.21.15.59:0/99439006 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8029920 con 0x5622758fa7d0 2024-04-02T17:15:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 -- 172.21.15.59:0/99439006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 msgr2=0x5622758fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004390 tx=0x7f96e801b000 comp rx=0 tx=0).stop 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 -- 172.21.15.59:0/99439006 shutdown_connections 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758fa7d0 0x5622758fe500 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f5660 0x562275906420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/99439006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758f4c70 0x5622758ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 -- 172.21.15.59:0/99439006 >> 172.21.15.59:0/99439006 conn(0x5622758bca50 msgr2=0x5622758ef010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 -- 172.21.15.59:0/99439006 shutdown_connections 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.660+0000 7f96f9e2f740 1 -- 172.21.15.59:0/99439006 wait complete. 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 -- start start 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 Processor -- start 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x562275903860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x56227596f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b0d00 con 0x5622758fa7d0 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b0630 con 0x5622758f5660 2024-04-02T17:15:17.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01ad870 con 0x5622758f4c70 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.661+0000 7f96f9e2f740 10 client.0 _ll_register_callbacks cb 0x7f96e000f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47160/0 (socket says 172.21.15.59:47160) 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 -- 172.21.15.59:0/1264673173 learned_addr learned my addr 172.21.15.59:0/1264673173 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96e7400640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x562275903860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96f5200640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x56227596f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 -- 172.21.15.59:0/1264673173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 msgr2=0x562275903860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x562275903860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 -- 172.21.15.59:0/1264673173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 msgr2=0x56227596f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x56227596f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 -- 172.21.15.59:0/1264673173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622758f09b0 con 0x5622758f4c70 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96efe00640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96e000fdd0 tx=0x7f96e0018a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab865d5156d210f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.662+0000 7f96de000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:15:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0026070 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e8004480 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f96e0018e00 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96e00051e0 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96f9e2f740 10 client.26110 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562275970830 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.663+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e8004480 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f96e003e030 con 0x5622758f4c70 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 10 client.26110 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 10 client.26110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 20 client.26110 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5622759711e0 0x562275973600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 4 client.26110 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:15:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 --> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562275867c30 con 0x5622759711e0 2024-04-02T17:15:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.664+0000 7f96e7400640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5622759711e0 0x562275973600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96e7400640 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5622759711e0 0x562275973600 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94a38b9ca7b9d4bf server_cookie=ebc60db100454c25 in_seq=0 out_seq=0 2024-04-02T17:15:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96de000640 10 client.26110 ms_handle_connect on v2:172.21.15.164:6838/1157843957 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96de000640 1 -- 172.21.15.59:0/1264673173 <== mds.2 v2:172.21.15.164:6838/1157843957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562275867c30 con 0x5622759711e0 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96de000640 10 client.26110 handle_command_reply: tid=0 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 client.26110 shutdown 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 msgr2=0x562275903320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96e000fdd0 tx=0x7f96e0018a70 comp rx=0 tx=0).stop 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 shutdown_connections 2024-04-02T17:15:17.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:6838/1157843957,v1:172.21.15.164:6839/1157843957] conn(0x5622759711e0 0x562275973600 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5622758fa7d0 0x56227596f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622758f5660 0x562275903860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96de000640 0 client.26110 ms_handle_reset on v2:172.21.15.164:6838/1157843957 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 --2- 172.21.15.59:0/1264673173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622758f4c70 0x562275903320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 >> 172.21.15.59:0/1264673173 conn(0x5622758bca50 msgr2=0x5622758ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 shutdown_connections 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.665+0000 7f96f9e2f740 1 -- 172.21.15.59:0/1264673173 wait complete. 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f9e2f740 20 client.26110 trim_cache size 0 max 16384 2024-04-02T17:15:17.670 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 msgr2=0x7f96d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 0x7f96d40a6070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5622758f5ec0 tx=0x7f96e8005f90 comp rx=0 tx=0).stop 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 msgr2=0x7f96f019ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 0x7f96f019ea10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009230 tx=0x7f96d802c5d0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 shutdown_connections 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f010e0e0 0x7f96f01a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96f0075b40 0x7f96f019ea10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f96d40a3bb0 0x7f96d40a6070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 --2- 172.21.15.59:0/260548524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0074560 0x7f96f019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 >> 172.21.15.59:0/260548524 conn(0x7f96f00fd0a0 msgr2=0x7f96f0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 shutdown_connections 2024-04-02T17:15:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.666+0000 7f96f7200640 1 -- 172.21.15.59:0/260548524 wait complete. 2024-04-02T17:15:17.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T17:15:17.767 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:17 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2909188224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:15:17.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 -- 172.21.15.59:0/1270657599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc108120 msgr2=0x7f57dc108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 --2- 172.21.15.59:0/1270657599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc108120 0x7f57dc108540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009ca0 tx=0x7f57cc02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 -- 172.21.15.59:0/1270657599 shutdown_connections 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 --2- 172.21.15.59:0/1270657599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109cd0 0x7f57dc110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 --2- 172.21.15.59:0/1270657599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc109310 0x7f57dc109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 --2- 172.21.15.59:0/1270657599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc108120 0x7f57dc108540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 -- 172.21.15.59:0/1270657599 >> 172.21.15.59:0/1270657599 conn(0x7f57dc0770c0 msgr2=0x7f57dc079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 -- 172.21.15.59:0/1270657599 shutdown_connections 2024-04-02T17:15:17.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.912+0000 7f57e3a00640 1 -- 172.21.15.59:0/1270657599 wait complete. 2024-04-02T17:15:17.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 Processor -- start 2024-04-02T17:15:17.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 -- start start 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 0x7f57dc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 0x7f57dc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 0x7f57dc1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc113dd0 con 0x7f57dc109cd0 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc113c50 con 0x7f57dc108120 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.913+0000 7f57e3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc113f50 con 0x7f57dc109310 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 0x7f57dc1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 0x7f57dc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 0x7f57dc1a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50936/0 (socket says 172.21.15.59:50936) 2024-04-02T17:15:17.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1a00640 1 -- 172.21.15.59:0/3680648932 learned_addr learned my addr 172.21.15.59:0/3680648932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 0x7f57dc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 -- 172.21.15.59:0/3680648932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 msgr2=0x7f57dc19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 0x7f57dc19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 -- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 msgr2=0x7f57dc1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 0x7f57dc1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 -- 172.21.15.59:0/3680648932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc1a6050 con 0x7f57dc108120 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57d3e00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 0x7f57dc19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.914+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 0x7f57dc19e430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57cc037040 tx=0x7f57cc032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.915+0000 7f57da000640 1 -- 172.21.15.59:0/3680648932 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc041020 con 0x7f57dc108120 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.915+0000 7f57da000640 1 -- 172.21.15.59:0/3680648932 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57cc034cd0 con 0x7f57dc108120 2024-04-02T17:15:17.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.915+0000 7f57da000640 1 -- 172.21.15.59:0/3680648932 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc039040 con 0x7f57dc108120 2024-04-02T17:15:17.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.915+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1a62e0 con 0x7f57dc108120 2024-04-02T17:15:17.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.915+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a67f0 con 0x7f57dc108120 2024-04-02T17:15:17.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.918+0000 7f57da000640 1 -- 172.21.15.59:0/3680648932 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57cc044020 con 0x7f57dc108120 2024-04-02T17:15:17.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.918+0000 7f57e67bc740 1 Processor -- start 2024-04-02T17:15:17.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.918+0000 7f57e67bc740 1 -- start start 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d41c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565144c7b400 con 0x565144d39c70 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565144c5ff10 con 0x565144d3a660 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e67bc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565144c9f6f0 con 0x565144d3f7d0 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50946/0 (socket says 172.21.15.59:50946) 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 -- 172.21.15.59:0/1313236627 learned_addr learned my addr 172.21.15.59:0/1313236627 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57d3e00640 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d41c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 -- 172.21.15.59:0/1313236627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 msgr2=0x565144d41c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d41c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 -- 172.21.15.59:0/1313236627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 msgr2=0x565144d3f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d3f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.919+0000 7f57e1a00640 1 -- 172.21.15.59:0/1313236627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651449b33f0 con 0x565144d39c70 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57e1a00640 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009d90 tx=0x7f57d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a465286148ea04db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57da000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 0x7f57c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57d2000640 1 -- 172.21.15.59:0/1313236627 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d401e070 con 0x565144d39c70 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57d2000640 1 -- 172.21.15.59:0/1313236627 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d4019b10 con 0x565144d39c70 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57da000640 1 -- 172.21.15.59:0/3680648932 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f57cc0f2170 con 0x7f57dc108120 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57d2000640 1 -- 172.21.15.59:0/1313236627 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400fbd0 con 0x565144d39c70 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.920+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 0x7f57c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 -- 172.21.15.59:0/1313236627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 msgr2=0x565144d3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009d90 tx=0x7f57d4011810 comp rx=0 tx=0).stop 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e1000640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 0x7f57c00a60f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0badf0 tx=0x7f57cc0b8f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 -- 172.21.15.59:0/1313236627 shutdown_connections 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d41c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d3f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 --2- 172.21.15.59:0/1313236627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d3a090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 -- 172.21.15.59:0/1313236627 >> 172.21.15.59:0/1313236627 conn(0x565144d01a50 msgr2=0x565144d355e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 -- 172.21.15.59:0/1313236627 shutdown_connections 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.921+0000 7f57e67bc740 1 -- 172.21.15.59:0/1313236627 wait complete. 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 -- start start 2024-04-02T17:15:17.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 Processor -- start 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d44400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565144c9b6a0 con 0x565144d39c70 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565144c47980 con 0x565144d3a660 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565144c5acb0 con 0x565144d3f7d0 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.922+0000 7f57e67bc740 10 client.0 _ll_register_callbacks cb 0x565144c5a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50950/0 (socket says 172.21.15.59:50950) 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 -- 172.21.15.59:0/801905649 learned_addr learned my addr 172.21.15.59:0/801905649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57d3e00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d44400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 -- 172.21.15.59:0/801905649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 msgr2=0x565144d4b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 -- 172.21.15.59:0/801905649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 msgr2=0x565144d44400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d44400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 -- 172.21.15.59:0/801905649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565144a7d7d0 con 0x565144d39c70 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57e1a00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57d400f510 tx=0x7f57d4009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca7f71abc3fb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.923+0000 7f57caa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d402d060 con 0x565144d39c70 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f57d400fb50 con 0x565144d39c70 2024-04-02T17:15:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d4019b10 con 0x565144d39c70 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f57d401e070 con 0x565144d39c70 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57e67bc740 10 client.16008 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565144d4e7b0 con 0x565144d39c70 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.924+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d4019b10 con 0x565144d39c70 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f57d4031030 con 0x565144d39c70 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 10 client.16008 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 10 client.16008 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 20 client.16008 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144d4fc50 0x565144d52060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 4 client.16008 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565144c5f040 con 0x565144d4fc50 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.925+0000 7f57d3e00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144d4fc50 0x565144d52060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.926+0000 7f57caa00640 10 client.16008 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.926+0000 7f57d3e00640 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144d4fc50 0x565144d52060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6be02ae2e4c04584 server_cookie=f71a80e6fda34760 in_seq=0 out_seq=0 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.926+0000 7f57caa00640 1 -- 172.21.15.59:0/801905649 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565144c5f040 con 0x565144d4fc50 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.926+0000 7f57caa00640 10 client.16008 handle_command_reply: tid=0 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 client.16008 shutdown 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 msgr2=0x565144d43ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57d400f510 tx=0x7f57d4009760 comp rx=0 tx=0).stop 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 shutdown_connections 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144d4fc50 0x565144d52060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3a660 0x565144d44400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 --2- 172.21.15.59:0/801905649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d39c70 0x565144d43ec0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57caa00640 0 client.16008 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 >> 172.21.15.59:0/801905649 conn(0x565144d01a50 msgr2=0x565144d348a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 shutdown_connections 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 1 -- 172.21.15.59:0/801905649 wait complete. 2024-04-02T17:15:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.927+0000 7f57e67bc740 20 client.16008 trim_cache size 0 max 16384 2024-04-02T17:15:17.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 Processor -- start 2024-04-02T17:15:17.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 -- start start 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d4b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3f7d0 0x565144d38820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1af7a0 con 0x565144d3a660 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b13a0 con 0x565144d3f7d0 2024-04-02T17:15:17.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.942+0000 7f57e67bc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1afe70 con 0x565144d39c70 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47206/0 (socket says 172.21.15.59:47206) 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 -- 172.21.15.59:0/3804163308 learned_addr learned my addr 172.21.15.59:0/3804163308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57d3e00640 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d4b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3f7d0 0x565144d38820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 -- 172.21.15.59:0/3804163308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3f7d0 msgr2=0x565144d38820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3f7d0 0x565144d38820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 -- 172.21.15.59:0/3804163308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 msgr2=0x565144d4b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d4b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.943+0000 7f57e1a00640 1 -- 172.21.15.59:0/3804163308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565144d36330 con 0x565144d39c70 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57e1a00640 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57d40059f0 tx=0x7f57d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf28b8bc5886e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57d1600640 1 -- 172.21.15.59:0/3804163308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d402d070 con 0x565144d39c70 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57d1600640 1 -- 172.21.15.59:0/3804163308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d4004e40 con 0x565144d39c70 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57d1600640 1 -- 172.21.15.59:0/3804163308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400f7d0 con 0x565144d39c70 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57e67bc740 1 -- 172.21.15.59:0/3804163308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 msgr2=0x565144d441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.944+0000 7f57e67bc740 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57d40059f0 tx=0x7f57d4011810 comp rx=0 tx=0).stop 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 -- 172.21.15.59:0/3804163308 shutdown_connections 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d4b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d3f7d0 0x565144d38820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 --2- 172.21.15.59:0/3804163308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d39c70 0x565144d441e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 -- 172.21.15.59:0/3804163308 >> 172.21.15.59:0/3804163308 conn(0x565144d01a50 msgr2=0x565144d43ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 -- 172.21.15.59:0/3804163308 shutdown_connections 2024-04-02T17:15:17.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 -- 172.21.15.59:0/3804163308 wait complete. 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 -- start start 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.945+0000 7f57e67bc740 1 Processor -- start 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d39c70 0x565144d48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b0c10 con 0x565144d3a660 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b0540 con 0x565144d39c70 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1ad780 con 0x565144d3f7d0 2024-04-02T17:15:17.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e67bc740 10 client.0 _ll_register_callbacks cb 0x7f57c4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50968/0 (socket says 172.21.15.59:50968) 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 -- 172.21.15.59:0/12371917 learned_addr learned my addr 172.21.15.59:0/12371917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e1a00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47220/0 (socket says 172.21.15.59:47220) 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57e1a00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d39c70 0x565144d48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 -- 172.21.15.59:0/12371917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 msgr2=0x565144d4a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 -- 172.21.15.59:0/12371917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d39c70 msgr2=0x565144d48ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d39c70 0x565144d48ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.946+0000 7f57d3e00640 1 -- 172.21.15.59:0/12371917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565144d4a600 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57d3e00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009fd0 tx=0x7f57c4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdab6cc18c388c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57caa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4005020 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f57c40051c0 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565144cacc30 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f57c402e050 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.947+0000 7f57e67bc740 10 client.16014 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565144db6200 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565144cacc30 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f57c40274d0 con 0x565144d3a660 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 10 client.16014 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 10 client.16014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-02T17:15:17.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 20 client.16014 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144db6c30 0x565144db9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 4 client.16014 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 --> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565144ca6f80 con 0x565144db6c30 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.948+0000 7f57e1a00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144db6c30 0x565144db9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57e1a00640 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144db6c30 0x565144db9050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4d0471ed89c329f server_cookie=57dc24bb8c6ff16b in_seq=0 out_seq=0 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57caa00640 10 client.16014 ms_handle_connect on v2:172.21.15.59:6838/377916146 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57caa00640 1 -- 172.21.15.59:0/12371917 <== mds.1 v2:172.21.15.59:6838/377916146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1834 (crc 0 0 0) 0x565144ca6f80 con 0x565144db6c30 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57caa00640 10 client.16014 handle_command_reply: tid=0 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57e67bc740 1 client.16014 shutdown 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 msgr2=0x565144d49000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.949+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009fd0 tx=0x7f57c4012b00 comp rx=0 tx=0).stop 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 shutdown_connections 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:6838/377916146,v1:172.21.15.59:6839/377916146] conn(0x565144db6c30 0x565144db9050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565144d3a660 0x565144d49000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565144d3f7d0 0x565144d4a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 --2- 172.21.15.59:0/12371917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565144d39c70 0x565144d48ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57caa00640 0 client.16014 ms_handle_reset on v2:172.21.15.59:6838/377916146 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 >> 172.21.15.59:0/12371917 conn(0x565144d01a50 msgr2=0x565144d34010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 shutdown_connections 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 1 -- 172.21.15.59:0/12371917 wait complete. 2024-04-02T17:15:17.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e67bc740 20 client.16014 trim_cache size 0 max 16384 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 750899252, 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628308, 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.956 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 1114249323, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628305, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP" 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 2270859389, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628327, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.957 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 2352893128, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628326, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3110016284, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628328, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.958 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3116537478, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628306, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3170281817, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628304, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:15:17.959 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-02T17:15:17.960 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3261980844, 2024-04-02T17:15:17.960 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511628307, 2024-04-02T17:15:17.960 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT" 2024-04-02T17:15:17.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:15:17.960 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 msgr2=0x7f57c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 0x7f57c00a60f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0badf0 tx=0x7f57cc0b8f90 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 msgr2=0x7f57dc19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.950+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 0x7f57dc19e430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57cc037040 tx=0x7f57cc032ea0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 shutdown_connections 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57c00a3c30 0x7f57c00a60f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57dc109cd0 0x7f57dc1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57dc109310 0x7f57dc19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 --2- 172.21.15.59:0/3680648932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc108120 0x7f57dc19e430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 >> 172.21.15.59:0/3680648932 conn(0x7f57dc0770c0 msgr2=0x7f57dc10c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 shutdown_connections 2024-04-02T17:15:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:17.951+0000 7f57e3a00640 1 -- 172.21.15.59:0/3680648932 wait complete. 2024-04-02T17:15:17.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T17:15:18.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29139 3.87 MB/sec warmup 45 sec latency 1751.638 ms 2024-04-02T17:15:18.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:17 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2909188224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:15:18.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:17 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2909188224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.194+0000 7ffb55e00640 1 -- 172.21.15.59:0/864904512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 msgr2=0x7ffb50105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.194+0000 7ffb55e00640 1 --2- 172.21.15.59:0/864904512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb50105270 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009900 tx=0x7ffb3c02bc50 comp rx=0 tx=0).stop 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 -- 172.21.15.59:0/864904512 shutdown_connections 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 --2- 172.21.15.59:0/864904512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb50105270 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 --2- 172.21.15.59:0/864904512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb501057b0 0x7ffb5010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 --2- 172.21.15.59:0/864904512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb50103c00 0x7ffb50104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 -- 172.21.15.59:0/864904512 >> 172.21.15.59:0/864904512 conn(0x7ffb500ff170 msgr2=0x7ffb501015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 -- 172.21.15.59:0/864904512 shutdown_connections 2024-04-02T17:15:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.195+0000 7ffb55e00640 1 -- 172.21.15.59:0/864904512 wait complete. 2024-04-02T17:15:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.196+0000 7ffb55e00640 1 Processor -- start 2024-04-02T17:15:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 -- start start 2024-04-02T17:15:18.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb5019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 0x7ffb501a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5010f8d0 con 0x7ffb50104df0 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5010f750 con 0x7ffb501057b0 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb55e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5010fa50 con 0x7ffb50103c00 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.197+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47238/0 (socket says 172.21.15.59:47238) 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb54a00640 1 -- 172.21.15.59:0/1507732509 learned_addr learned my addr 172.21.15.59:0/1507732509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb5019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb55400640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 0x7ffb501a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 -- 172.21.15.59:0/1507732509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 msgr2=0x7ffb50199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 -- 172.21.15.59:0/1507732509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 msgr2=0x7ffb501a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 0x7ffb501a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb4fe00640 1 -- 172.21.15.59:0/1507732509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb501a1ad0 con 0x7ffb50104df0 2024-04-02T17:15:18.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb54a00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:18.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.198+0000 7ffb55400640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 0x7ffb501a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:18.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.199+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb5019a3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c036040 tx=0x7ffb3c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.199+0000 7ffb4d600640 1 -- 172.21.15.59:0/1507732509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c041070 con 0x7ffb50104df0 2024-04-02T17:15:18.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.200+0000 7ffb4d600640 1 -- 172.21.15.59:0/1507732509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb3c004450 con 0x7ffb50104df0 2024-04-02T17:15:18.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.200+0000 7ffb4d600640 1 -- 172.21.15.59:0/1507732509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c003e60 con 0x7ffb50104df0 2024-04-02T17:15:18.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.200+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb501a1d60 con 0x7ffb50104df0 2024-04-02T17:15:18.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.200+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb501a21d0 con 0x7ffb50104df0 2024-04-02T17:15:18.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.204+0000 7ffb4d600640 1 -- 172.21.15.59:0/1507732509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb3c044070 con 0x7ffb50104df0 2024-04-02T17:15:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.204+0000 7ffb5a182740 1 Processor -- start 2024-04-02T17:15:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 -- start start 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5637c232e400 con 0x5637c23f2980 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5637c2312f10 con 0x5637c23ed810 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb5a182740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637c23526f0 con 0x5637c23ece20 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb55400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb55400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47240/0 (socket says 172.21.15.59:47240) 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb55400640 1 -- 172.21.15.59:0/758873866 learned_addr learned my addr 172.21.15.59:0/758873866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.205+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 -- 172.21.15.59:0/758873866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 msgr2=0x5637c23f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 -- 172.21.15.59:0/758873866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 msgr2=0x5637c23f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 -- 172.21.15.59:0/758873866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c20663f0 con 0x5637c23ece20 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb54a00640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.206+0000 7ffb55400640 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb480117e0 tx=0x7ffb48011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5542f4f5e8ed734 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.207+0000 7ffb41c00640 1 -- 172.21.15.59:0/758873866 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011310 con 0x5637c23ece20 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.207+0000 7ffb41c00640 1 -- 172.21.15.59:0/758873866 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb48003e50 con 0x5637c23ece20 2024-04-02T17:15:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.207+0000 7ffb41c00640 1 -- 172.21.15.59:0/758873866 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48004550 con 0x5637c23ece20 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.207+0000 7ffb4d600640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 0x7ffb380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb4d600640 1 -- 172.21.15.59:0/1507732509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ffb3c0f1dd0 con 0x7ffb50104df0 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb54a00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 0x7ffb380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 -- 172.21.15.59:0/758873866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 msgr2=0x5637c23ed240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb480117e0 tx=0x7ffb48011ca0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 -- 172.21.15.59:0/758873866 shutdown_connections 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 --2- 172.21.15.59:0/758873866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23ed240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 -- 172.21.15.59:0/758873866 >> 172.21.15.59:0/758873866 conn(0x5637c23b4a50 msgr2=0x5637c23e8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 -- 172.21.15.59:0/758873866 shutdown_connections 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.208+0000 7ffb5a182740 1 -- 172.21.15.59:0/758873866 wait complete. 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb54a00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 0x7ffb380a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5637c23e96e0 tx=0x7ffb44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 -- start start 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 Processor -- start 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 0x5637c23f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5637c234e6a0 con 0x5637c23ece20 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5637c22fa980 con 0x5637c23ed810 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.209+0000 7ffb5a182740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637c230dcb0 con 0x5637c23f2980 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb5a182740 10 client.0 _ll_register_callbacks cb 0x5637c230d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 0x5637c23f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47248/0 (socket says 172.21.15.59:47248) 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 -- 172.21.15.59:0/957446915 learned_addr learned my addr 172.21.15.59:0/957446915 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb54a00640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 0x5637c23f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51008/0 (socket says 172.21.15.59:51008) 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 -- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 msgr2=0x5637c23f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 -- 172.21.15.59:0/957446915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 msgr2=0x5637c23f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 0x5637c23f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb55400640 1 -- 172.21.15.59:0/957446915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c21307d0 con 0x5637c23f2980 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.210+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.211+0000 7ffb55400640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800a010 tx=0x7ffb480113c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a644a145ae5fe654 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.211+0000 7ffb36000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.211+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4802a060 con 0x5637c23f2980 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.211+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb48003e50 con 0x5637c23f2980 2024-04-02T17:15:18.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.211+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ffb48004a40 con 0x5637c23f2980 2024-04-02T17:15:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb48004c40 con 0x5637c23f2980 2024-04-02T17:15:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 10 client.26140 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637c2401980 con 0x5637c23f2980 2024-04-02T17:15:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb48003e50 con 0x5637c23f2980 2024-04-02T17:15:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ffb4802e030 con 0x5637c23f2980 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 10 client.26140 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 20 client.26140 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2402e90 0x5637c24052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 4 client.26140 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.212+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637c2312040 con 0x5637c2402e90 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.213+0000 7ffb54a00640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2402e90 0x5637c24052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.213+0000 7ffb54a00640 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2402e90 0x5637c24052f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7742fbabc2ce6 server_cookie=77b9faf4472f6b3b in_seq=0 out_seq=0 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.213+0000 7ffb36000640 10 client.26140 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb36000640 1 -- 172.21.15.59:0/957446915 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637c2312040 con 0x5637c2402e90 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb36000640 10 client.26140 handle_command_reply: tid=0 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 client.26140 shutdown 2024-04-02T17:15:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 msgr2=0x5637c23fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800a010 tx=0x7ffb480113c0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 shutdown_connections 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2402e90 0x5637c24052f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c23fe610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb36000640 0 client.26140 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 --2- 172.21.15.59:0/957446915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ece20 0x5637c23f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.214+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 >> 172.21.15.59:0/957446915 conn(0x5637c23b4a50 msgr2=0x5637c23e7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.215+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 shutdown_connections 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.215+0000 7ffb5a182740 1 -- 172.21.15.59:0/957446915 wait complete. 2024-04-02T17:15:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.215+0000 7ffb5a182740 20 client.26140 trim_cache size 0 max 16384 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.230+0000 7ffb5a182740 1 Processor -- start 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.230+0000 7ffb5a182740 1 -- start start 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501ab100 con 0x5637c23f2980 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501acd00 con 0x5637c23ed810 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb5a182740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501ab7d0 con 0x5637c23ece20 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51016/0 (socket says 172.21.15.59:51016) 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 -- 172.21.15.59:0/2490186947 learned_addr learned my addr 172.21.15.59:0/2490186947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb54a00640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 -- 172.21.15.59:0/2490186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 msgr2=0x5637c23e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 -- 172.21.15.59:0/2490186947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 msgr2=0x5637c23fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb55400640 1 -- 172.21.15.59:0/2490186947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c2405970 con 0x5637c23f2980 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.231+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.232+0000 7ffb55400640 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009f60 tx=0x7ffb480116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62bc8e0707775a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.232+0000 7ffb41200640 1 -- 172.21.15.59:0/2490186947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4802a070 con 0x5637c23f2980 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.232+0000 7ffb41200640 1 -- 172.21.15.59:0/2490186947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb480052c0 con 0x5637c23f2980 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.232+0000 7ffb41200640 1 -- 172.21.15.59:0/2490186947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb480040b0 con 0x5637c23f2980 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- 172.21.15.59:0/2490186947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 msgr2=0x5637c23eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009f60 tx=0x7ffb480116e0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- 172.21.15.59:0/2490186947 shutdown_connections 2024-04-02T17:15:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23f2980 0x5637c23eb9d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ed810 0x5637c23fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 --2- 172.21.15.59:0/2490186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23ece20 0x5637c23e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- 172.21.15.59:0/2490186947 >> 172.21.15.59:0/2490186947 conn(0x5637c23b4a50 msgr2=0x5637c23f7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- 172.21.15.59:0/2490186947 shutdown_connections 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- 172.21.15.59:0/2490186947 wait complete. 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.233+0000 7ffb5a182740 1 -- start start 2024-04-02T17:15:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 Processor -- start 2024-04-02T17:15:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ed810 0x5637c23fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c2467460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501ac570 con 0x5637c23ed810 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501abea0 con 0x5637c23ece20 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501a90e0 con 0x5637c23f2980 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb5a182740 10 client.0 _ll_register_callbacks cb 0x7ffb48003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ed810 0x5637c23fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb54a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:54398/0 (socket says 172.21.15.59:54398) 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb54a00640 1 -- 172.21.15.59:0/315648238 learned_addr learned my addr 172.21.15.59:0/315648238 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.234+0000 7ffb55400640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c2467460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 -- 172.21.15.59:0/315648238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 msgr2=0x5637c2467460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c2467460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 -- 172.21.15.59:0/315648238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ed810 msgr2=0x5637c23fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ed810 0x5637c23fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 -- 172.21.15.59:0/315648238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c23e8d30 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb54a00640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5637c23ee280 tx=0x7ffb440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba92c2330806efb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb36000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402f070 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb480052c0 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.235+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ffb44004110 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb4403b050 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 10 client.26117 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637c2468af0 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb480052c0 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ffb44003a60 con 0x5637c23ece20 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 10 client.26117 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 20 client.26117 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2469570 0x5637c246b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 4 client.26117 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.236+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637c235fc30 con 0x5637c2469570 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2469570 0x5637c246b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb4fe00640 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2469570 0x5637c246b990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad6bf4725f644f2 server_cookie=924bf9262a76d897 in_seq=0 out_seq=0 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb36000640 10 client.26117 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb36000640 1 -- 172.21.15.59:0/315648238 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637c235fc30 con 0x5637c2469570 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb36000640 10 client.26117 handle_command_reply: tid=0 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.237+0000 7ffb5a182740 1 client.26117 shutdown 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 msgr2=0x5637c23fb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5637c23ee280 tx=0x7ffb440046c0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 shutdown_connections 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5637c2469570 0x5637c246b990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637c23ed810 0x5637c23fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637c23f2980 0x5637c2467460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 --2- 172.21.15.59:0/315648238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5637c23ece20 0x5637c23fb460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb36000640 0 client.26117 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 >> 172.21.15.59:0/315648238 conn(0x5637c23b4a50 msgr2=0x5637c23e7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 shutdown_connections 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 1 -- 172.21.15.59:0/315648238 wait complete. 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.238+0000 7ffb5a182740 20 client.26117 trim_cache size 0 max 16384 2024-04-02T17:15:18.242 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 msgr2=0x7ffb380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 0x7ffb380a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5637c23e96e0 tx=0x7ffb44005f90 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 msgr2=0x7ffb5019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb5019a3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c036040 tx=0x7ffb3c02c670 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 shutdown_connections 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ffb380a3c30 0x7ffb380a60f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb50104df0 0x7ffb5019a3f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb501057b0 0x7ffb501a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 --2- 172.21.15.59:0/1507732509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb50103c00 0x7ffb50199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 >> 172.21.15.59:0/1507732509 conn(0x7ffb500ff170 msgr2=0x7ffb501080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 shutdown_connections 2024-04-02T17:15:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:15:18.239+0000 7ffb55e00640 1 -- 172.21.15.59:0/1507732509 wait complete. 2024-04-02T17:15:18.256 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 750899252, 'ino': 1099511628308, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 1114249323, 'ino': 1099511628305, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 2270859389, 'ino': 1099511628327, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 2352893128, 'ino': 1099511628326, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 3110016284, 'ino': 1099511628328, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 3116537478, 'ino': 1099511628306, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 3170281817, 'ino': 1099511628304, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 3261980844, 'ino': 1099511628307, 'path': '/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT'}] 2024-04-02T17:15:18.256 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-04-02T17:15:19.034 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29207 3.79 MB/sec warmup 46 sec latency 1860.516 ms 2024-04-02T17:15:19.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:18 smithi059 ceph-mon[31664]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-04-02T17:15:19.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:18 smithi164 ceph-mon[36499]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-04-02T17:15:19.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:18 smithi154 ceph-mon[36479]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 148 KiB/s wr, 52 op/s 2024-04-02T17:15:20.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29207 3.71 MB/sec warmup 47 sec latency 1809.673 ms 2024-04-02T17:15:21.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29207 3.63 MB/sec warmup 48 sec latency 2809.823 ms 2024-04-02T17:15:21.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:20 smithi059 ceph-mon[31664]: pgmap v456: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-04-02T17:15:21.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:20 smithi164 ceph-mon[36499]: pgmap v456: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-04-02T17:15:21.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:20 smithi154 ceph-mon[36479]: pgmap v456: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-04-02T17:15:22.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29207 3.56 MB/sec warmup 49 sec latency 3809.955 ms 2024-04-02T17:15:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 29207 3.49 MB/sec warmup 50 sec latency 4810.101 ms 2024-04-02T17:15:23.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:22 smithi059 ceph-mon[31664]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 192 KiB/s wr, 59 op/s 2024-04-02T17:15:23.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:22 smithi164 ceph-mon[36499]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 192 KiB/s wr, 59 op/s 2024-04-02T17:15:23.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:22 smithi154 ceph-mon[36479]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 192 KiB/s wr, 59 op/s 2024-04-02T17:15:24.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 30753 3.55 MB/sec warmup 51 sec latency 4924.096 ms 2024-04-02T17:15:25.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 32108 3.69 MB/sec warmup 52 sec latency 14.371 ms 2024-04-02T17:15:25.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:24 smithi059 ceph-mon[31664]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 195 KiB/s wr, 60 op/s 2024-04-02T17:15:25.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:24 smithi164 ceph-mon[36499]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 195 KiB/s wr, 60 op/s 2024-04-02T17:15:25.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:24 smithi154 ceph-mon[36479]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 195 KiB/s wr, 60 op/s 2024-04-02T17:15:26.035 INFO:tasks.workunit.client.0.smithi059.stdout: 1 32424 3.64 MB/sec warmup 53 sec latency 22.563 ms 2024-04-02T17:15:27.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 32687 3.58 MB/sec warmup 54 sec latency 582.907 ms 2024-04-02T17:15:27.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:26 smithi059 ceph-mon[31664]: pgmap v459: 97 pgs: 97 active+clean; 24 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-04-02T17:15:27.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:26 smithi164 ceph-mon[36499]: pgmap v459: 97 pgs: 97 active+clean; 24 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-04-02T17:15:27.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:26 smithi154 ceph-mon[36479]: pgmap v459: 97 pgs: 97 active+clean; 24 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-04-02T17:15:28.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 32687 3.51 MB/sec warmup 55 sec latency 1583.065 ms 2024-04-02T17:15:29.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 33931 3.55 MB/sec warmup 56 sec latency 1691.957 ms 2024-04-02T17:15:29.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:28 smithi059 ceph-mon[31664]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 273 KiB/s wr, 77 op/s 2024-04-02T17:15:29.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:28 smithi164 ceph-mon[36499]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 273 KiB/s wr, 77 op/s 2024-04-02T17:15:29.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:28 smithi154 ceph-mon[36479]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 273 KiB/s wr, 77 op/s 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 -- 172.21.15.59:0/3649848736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 msgr2=0x7fc178104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 --2- 172.21.15.59:0/3649848736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178104170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc168009860 tx=0x7fc16802bb70 comp rx=0 tx=0).stop 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 -- 172.21.15.59:0/3649848736 shutdown_connections 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 --2- 172.21.15.59:0/3649848736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 0x7fc1781053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 --2- 172.21.15.59:0/3649848736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc17810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 --2- 172.21.15.59:0/3649848736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178104170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 -- 172.21.15.59:0/3649848736 >> 172.21.15.59:0/3649848736 conn(0x7fc1780ff2c0 msgr2=0x7fc178101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 -- 172.21.15.59:0/3649848736 shutdown_connections 2024-04-02T17:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.439+0000 7fc17f400640 1 -- 172.21.15.59:0/3649848736 wait complete. 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.440+0000 7fc17f400640 1 Processor -- start 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.440+0000 7fc17f400640 1 -- start start 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 0x7fc17819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc1781a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810f9a0 con 0x7fc178104f40 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810f820 con 0x7fc178105900 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc17810fb20 con 0x7fc178103d50 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60686/0 (socket says 172.21.15.59:60686) 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 -- 172.21.15.59:0/3328471362 learned_addr learned my addr 172.21.15.59:0/3328471362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc1781a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc177e00640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 0x7fc17819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 -- 172.21.15.59:0/3328471362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 msgr2=0x7fc1781a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc1781a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 -- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 msgr2=0x7fc17819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 0x7fc17819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17ca00640 1 -- 172.21.15.59:0/3328471362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1781a1be0 con 0x7fc178103d50 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.441+0000 7fc17d400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc1781a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1680349e0 tx=0x7fc168032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc175600640 1 -- 172.21.15.59:0/3328471362 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168041020 con 0x7fc178103d50 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc175600640 1 -- 172.21.15.59:0/3328471362 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16802c9f0 con 0x7fc178103d50 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc175600640 1 -- 172.21.15.59:0/3328471362 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168039040 con 0x7fc178103d50 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1781a1e70 con 0x7fc178103d50 2024-04-02T17:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.442+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781a2280 con 0x7fc178103d50 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc175600640 1 -- 172.21.15.59:0/3328471362 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc168044020 con 0x7fc178103d50 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 Processor -- start 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 -- start start 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55af47ab3d70 con 0x55af47b4e8c0 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55af47a5c2a0 con 0x55af47b4ded0 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc182181740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55af47aafd20 con 0x55af47b53a30 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.446+0000 7fc17d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc17d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38420/0 (socket says 172.21.15.59:38420) 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc17d400640 1 -- 172.21.15.59:0/218220702 learned_addr learned my addr 172.21.15.59:0/218220702 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 -- 172.21.15.59:0/218220702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 msgr2=0x55af47b4e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b4e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 -- 172.21.15.59:0/218220702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 msgr2=0x55af47b533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 -- 172.21.15.59:0/218220702 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af47a70840 con 0x55af47b53a30 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc17ca00640 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.447+0000 7fc177e00640 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b55ec0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc160005fd0 tx=0x7fc1600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69342b3e1b6d09f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc16dc00640 1 -- 172.21.15.59:0/218220702 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160003c30 con 0x55af47b53a30 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc16dc00640 1 -- 172.21.15.59:0/218220702 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc160003dd0 con 0x55af47b53a30 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc16dc00640 1 -- 172.21.15.59:0/218220702 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16000fb00 con 0x55af47b53a30 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc175600640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 0x7fc1580a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc175600640 1 -- 172.21.15.59:0/3328471362 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc1680f1c70 con 0x7fc178103d50 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc17d400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 0x7fc1580a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc182181740 1 -- 172.21.15.59:0/218220702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 msgr2=0x55af47b55ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.448+0000 7fc182181740 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b55ec0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc160005fd0 tx=0x7fc1600118d0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 -- 172.21.15.59:0/218220702 shutdown_connections 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b55ec0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 --2- 172.21.15.59:0/218220702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b4e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 -- 172.21.15.59:0/218220702 >> 172.21.15.59:0/218220702 conn(0x55af47b49440 msgr2=0x55af47b49840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 -- 172.21.15.59:0/218220702 shutdown_connections 2024-04-02T17:15:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc17d400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 0x7fc1580a6110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55af47b4f120 tx=0x7fc170009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 -- 172.21.15.59:0/218220702 wait complete. 2024-04-02T17:15:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.449+0000 7fc182181740 1 -- start start 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 Processor -- start 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 0x55af47b5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55af47b569e0 con 0x55af47b53a30 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55af47a6f660 con 0x55af47b4ded0 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55af47b56b60 con 0x55af47b4e8c0 2024-04-02T17:15:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc182181740 10 client.0 _ll_register_callbacks cb 0x55af47a6f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc177e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc177e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60698/0 (socket says 172.21.15.59:60698) 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc177e00640 1 -- 172.21.15.59:0/2862865615 learned_addr learned my addr 172.21.15.59:0/2862865615 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.450+0000 7fc17d400640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 0x55af47b5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 -- 172.21.15.59:0/2862865615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 msgr2=0x55af47b588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 -- 172.21.15.59:0/2862865615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 msgr2=0x55af47b5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 0x55af47b5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 -- 172.21.15.59:0/2862865615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af47a7ee80 con 0x55af47b4ded0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17d400640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 0x55af47b5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc177e00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc17ca00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b583b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1680ef760 tx=0x7fc1680045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb17e75b30503496 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc166000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680bdcc0 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc1680ba560 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.451+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc160003dd0 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1680bd850 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 10 client.26129 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af477c8210 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc160003dd0 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fc1680f9040 con 0x55af47b4ded0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 10 client.26129 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 20 client.26129 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47b64230 0x55af47b666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 4 client.26129 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.452+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af47a63cd0 con 0x55af47b64230 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.453+0000 7fc177e00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47b64230 0x55af47b666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.453+0000 7fc177e00640 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47b64230 0x55af47b666d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452f5ec7bd32c9bb server_cookie=a9037f34a51d5d3b in_seq=0 out_seq=0 2024-04-02T17:15:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.453+0000 7fc166000640 10 client.26129 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.454+0000 7fc166000640 1 -- 172.21.15.59:0/2862865615 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af47a63cd0 con 0x55af47b64230 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.454+0000 7fc166000640 10 client.26129 handle_command_reply: tid=0 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.454+0000 7fc182181740 1 client.26129 shutdown 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 msgr2=0x55af47b583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b583b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1680ef760 tx=0x7fc1680045f0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 shutdown_connections 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47b64230 0x55af47b666d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b53a30 0x55af47b5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b4e8c0 0x55af47b588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 --2- 172.21.15.59:0/2862865615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b583b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 >> 172.21.15.59:0/2862865615 conn(0x55af47b49440 msgr2=0x55af47b48640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc166000640 0 client.26129 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 shutdown_connections 2024-04-02T17:15:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 1 -- 172.21.15.59:0/2862865615 wait complete. 2024-04-02T17:15:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.455+0000 7fc182181740 20 client.26129 trim_cache size 0 max 16384 2024-04-02T17:15:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 Processor -- start 2024-04-02T17:15:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 -- start start 2024-04-02T17:15:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ab230 con 0x55af47b4e8c0 2024-04-02T17:15:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ace30 con 0x55af47b4ded0 2024-04-02T17:15:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc182181740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ab900 con 0x55af47b53a30 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc17d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.473+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60708/0 (socket says 172.21.15.59:60708) 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 -- 172.21.15.59:0/3694329799 learned_addr learned my addr 172.21.15.59:0/3694329799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc177e00640 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 -- 172.21.15.59:0/3694329799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 msgr2=0x55af47b58850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b58850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 -- 172.21.15.59:0/3694329799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 msgr2=0x55af47b5f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 -- 172.21.15.59:0/3694329799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af47894620 con 0x55af47b53a30 2024-04-02T17:15:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc177e00640 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.474+0000 7fc17d400640 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc17000f820 tx=0x7fc170017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50deba0bd9974505 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.475+0000 7fc16d200640 1 -- 172.21.15.59:0/3694329799 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170005020 con 0x55af47b53a30 2024-04-02T17:15:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.475+0000 7fc16d200640 1 -- 172.21.15.59:0/3694329799 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700056a0 con 0x55af47b53a30 2024-04-02T17:15:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.475+0000 7fc16d200640 1 -- 172.21.15.59:0/3694329799 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17001d540 con 0x55af47b53a30 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- 172.21.15.59:0/3694329799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 msgr2=0x55af47b4cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc17000f820 tx=0x7fc170017000 comp rx=0 tx=0).stop 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- 172.21.15.59:0/3694329799 shutdown_connections 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b4cdb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 --2- 172.21.15.59:0/3694329799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b58850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- 172.21.15.59:0/3694329799 >> 172.21.15.59:0/3694329799 conn(0x55af47b49440 msgr2=0x55af47b4c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- 172.21.15.59:0/3694329799 shutdown_connections 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- 172.21.15.59:0/3694329799 wait complete. 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.476+0000 7fc182181740 1 -- start start 2024-04-02T17:15:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 Processor -- start 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781ac6a0 con 0x55af47b4e8c0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781abfd0 con 0x55af47b4ded0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781a9210 con 0x55af47b53a30 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc182181740 10 client.0 _ll_register_callbacks cb 0x7fc1680f5440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc17ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38448/0 (socket says 172.21.15.59:38448) 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc17ca00640 1 -- 172.21.15.59:0/3868243785 learned_addr learned my addr 172.21.15.59:0/3868243785 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.477+0000 7fc17d400640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc177e00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 -- 172.21.15.59:0/3868243785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 msgr2=0x55af47b5e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b5e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 -- 172.21.15.59:0/3868243785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 msgr2=0x55af47b5d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 -- 172.21.15.59:0/3868243785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af47b5e9a0 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc177e00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17d400640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc17ca00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1680ba350 tx=0x7fc168032910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96087b61b59935c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc166000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680bda30 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fc1680c68d0 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.478+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1700056a0 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1680bdcc0 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 10 client.26135 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af47b5fc20 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700056a0 con 0x55af47b4ded0 2024-04-02T17:15:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fc16803c030 con 0x55af47b4ded0 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 10 client.26135 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 20 client.26135 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47bcae70 0x55af47bcd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 4 client.26135 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af47ac12b0 con 0x55af47bcae70 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.479+0000 7fc177e00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47bcae70 0x55af47bcd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.480+0000 7fc177e00640 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47bcae70 0x55af47bcd290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3db67c323a914a server_cookie=be89738569db85b9 in_seq=0 out_seq=0 2024-04-02T17:15:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.480+0000 7fc166000640 10 client.26135 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.505+0000 7fc166000640 1 -- 172.21.15.59:0/3868243785 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af47ac12b0 con 0x55af47bcae70 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.505+0000 7fc166000640 10 client.26135 handle_command_reply: tid=0 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.505+0000 7fc182181740 1 client.26135 shutdown 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 msgr2=0x55af47b5ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1680ba350 tx=0x7fc168032910 comp rx=0 tx=0).stop 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 shutdown_connections 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55af47bcae70 0x55af47bcd290 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af47b4e8c0 0x55af47b5d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af47b53a30 0x55af47b5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 --2- 172.21.15.59:0/3868243785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55af47b4ded0 0x55af47b5ce60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 >> 172.21.15.59:0/3868243785 conn(0x55af47b49440 msgr2=0x55af47b58640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc166000640 0 client.26135 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 shutdown_connections 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 1 -- 172.21.15.59:0/3868243785 wait complete. 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.506+0000 7fc182181740 20 client.26135 trim_cache size 0 max 16384 2024-04-02T17:15:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:15:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:15:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:15:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:15:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 msgr2=0x7fc1580a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 0x7fc1580a6110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55af47b4f120 tx=0x7fc170009260 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 msgr2=0x7fc178199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1680349e0 tx=0x7fc168032430 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 shutdown_connections 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1580a3c50 0x7fc1580a6110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc178104f40 0x7fc17819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc178105900 0x7fc1781a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 --2- 172.21.15.59:0/3328471362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc178103d50 0x7fc178199fc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 >> 172.21.15.59:0/3328471362 conn(0x7fc1780ff2c0 msgr2=0x7fc178108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 shutdown_connections 2024-04-02T17:15:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:29.507+0000 7fc17f400640 1 -- 172.21.15.59:0/3328471362 wait complete. 2024-04-02T17:15:30.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 35582 3.70 MB/sec warmup 57 sec latency 18.262 ms 2024-04-02T17:15:31.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 35957 3.66 MB/sec warmup 58 sec latency 21.908 ms 2024-04-02T17:15:31.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:30 smithi059 ceph-mon[31664]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 222 KiB/s wr, 61 op/s 2024-04-02T17:15:31.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:30 smithi164 ceph-mon[36499]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 222 KiB/s wr, 61 op/s 2024-04-02T17:15:31.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:30 smithi154 ceph-mon[36479]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 222 KiB/s wr, 61 op/s 2024-04-02T17:15:32.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 36174 3.60 MB/sec warmup 59 sec latency 649.113 ms 2024-04-02T17:15:33.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 36174 3.54 MB/sec warmup 60 sec latency 1649.254 ms 2024-04-02T17:15:33.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:32 smithi059 ceph-mon[31664]: pgmap v462: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 288 KiB/s wr, 74 op/s 2024-04-02T17:15:33.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:32 smithi164 ceph-mon[36499]: pgmap v462: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 288 KiB/s wr, 74 op/s 2024-04-02T17:15:33.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:32 smithi154 ceph-mon[36479]: pgmap v462: 97 pgs: 97 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 288 KiB/s wr, 74 op/s 2024-04-02T17:15:34.036 INFO:tasks.workunit.client.0.smithi059.stdout: 1 37124 3.56 MB/sec warmup 61 sec latency 1758.366 ms 2024-04-02T17:15:35.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 38833 3.65 MB/sec warmup 62 sec latency 13.462 ms 2024-04-02T17:15:35.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:34 smithi059 ceph-mon[31664]: pgmap v463: 97 pgs: 97 active+clean; 26 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 338 KiB/s wr, 92 op/s 2024-04-02T17:15:35.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:34 smithi164 ceph-mon[36499]: pgmap v463: 97 pgs: 97 active+clean; 26 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 338 KiB/s wr, 92 op/s 2024-04-02T17:15:35.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:34 smithi154 ceph-mon[36479]: pgmap v463: 97 pgs: 97 active+clean; 26 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 338 KiB/s wr, 92 op/s 2024-04-02T17:15:36.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 39458 3.67 MB/sec warmup 63 sec latency 31.928 ms 2024-04-02T17:15:37.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 39778 3.62 MB/sec warmup 64 sec latency 353.085 ms 2024-04-02T17:15:37.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:36 smithi059 ceph-mon[31664]: pgmap v464: 97 pgs: 97 active+clean; 27 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 380 KiB/s wr, 101 op/s 2024-04-02T17:15:37.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:36 smithi164 ceph-mon[36499]: pgmap v464: 97 pgs: 97 active+clean; 27 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 380 KiB/s wr, 101 op/s 2024-04-02T17:15:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:36 smithi154 ceph-mon[36479]: pgmap v464: 97 pgs: 97 active+clean; 27 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 380 KiB/s wr, 101 op/s 2024-04-02T17:15:38.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 39778 3.56 MB/sec warmup 65 sec latency 1353.215 ms 2024-04-02T17:15:39.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 39846 3.51 MB/sec warmup 66 sec latency 1460.985 ms 2024-04-02T17:15:39.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:38 smithi059 ceph-mon[31664]: pgmap v465: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 416 KiB/s wr, 113 op/s 2024-04-02T17:15:39.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:38 smithi164 ceph-mon[36499]: pgmap v465: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 416 KiB/s wr, 113 op/s 2024-04-02T17:15:39.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:38 smithi154 ceph-mon[36479]: pgmap v465: 97 pgs: 97 active+clean; 28 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 416 KiB/s wr, 113 op/s 2024-04-02T17:15:40.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 39846 3.46 MB/sec warmup 67 sec latency 1775.687 ms 2024-04-02T17:15:41.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 41493 3.51 MB/sec warmup 68 sec latency 1813.393 ms 2024-04-02T17:15:41.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:40 smithi059 ceph-mon[31664]: pgmap v466: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 395 KiB/s wr, 98 op/s 2024-04-02T17:15:41.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:40 smithi164 ceph-mon[36499]: pgmap v466: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 395 KiB/s wr, 98 op/s 2024-04-02T17:15:41.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:40 smithi154 ceph-mon[36479]: pgmap v466: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 395 KiB/s wr, 98 op/s 2024-04-02T17:15:42.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 42821 3.62 MB/sec warmup 69 sec latency 36.921 ms 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 -- 172.21.15.59:0/1007254732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1056d0 msgr2=0x7f251c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 --2- 172.21.15.59:0/1007254732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1056d0 0x7f251c105b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2514009ca0 tx=0x7f251402c100 comp rx=0 tx=0).stop 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 -- 172.21.15.59:0/1007254732 shutdown_connections 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 --2- 172.21.15.59:0/1007254732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1056d0 0x7f251c105b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 --2- 172.21.15.59:0/1007254732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c102d20 0x7f251c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 --2- 172.21.15.59:0/1007254732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c1003b0 0x7f251c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 -- 172.21.15.59:0/1007254732 >> 172.21.15.59:0/1007254732 conn(0x7f251c0fbfb0 msgr2=0x7f251c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 -- 172.21.15.59:0/1007254732 shutdown_connections 2024-04-02T17:15:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.658+0000 7f2522200640 1 -- 172.21.15.59:0/1007254732 wait complete. 2024-04-02T17:15:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.659+0000 7f2522200640 1 Processor -- start 2024-04-02T17:15:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 -- start start 2024-04-02T17:15:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c102d20 0x7f251c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 0x7f251c1a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251c113de0 con 0x7f251c1003b0 2024-04-02T17:15:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f251c113c60 con 0x7f251c102d20 2024-04-02T17:15:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f2522200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f251c113f60 con 0x7f251c1056d0 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f251b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f251be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 0x7f251c1a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f251b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47460/0 (socket says 172.21.15.59:47460) 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f251b400640 1 -- 172.21.15.59:0/1609063605 learned_addr learned my addr 172.21.15.59:0/1609063605 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.660+0000 7f251aa00640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c102d20 0x7f251c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 -- 172.21.15.59:0/1609063605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c102d20 msgr2=0x7f251c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c102d20 0x7f251c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 -- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 msgr2=0x7f251c19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 -- 172.21.15.59:0/1609063605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c1a5f00 con 0x7f251c1056d0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251b400640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f251be00640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 0x7f251c1a5890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2514037040 tx=0x7f2514035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.661+0000 7f2513e00640 1 -- 172.21.15.59:0/1609063605 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514039040 con 0x7f251c1056d0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.662+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c1a6190 con 0x7f251c1056d0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.662+0000 7f2513e00640 1 -- 172.21.15.59:0/1609063605 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2514044070 con 0x7f251c1056d0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.662+0000 7f2513e00640 1 -- 172.21.15.59:0/1609063605 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140044e0 con 0x7f251c1056d0 2024-04-02T17:15:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.662+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a66a0 con 0x7f251c1056d0 2024-04-02T17:15:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.665+0000 7f2513e00640 1 -- 172.21.15.59:0/1609063605 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2514032890 con 0x7f251c1056d0 2024-04-02T17:15:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.665+0000 7f2524f63740 1 Processor -- start 2024-04-02T17:15:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.665+0000 7f2524f63740 1 -- start start 2024-04-02T17:15:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x5587854ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x558785506ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558785464d70 con 0x5587854feed0 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55878540d2a0 con 0x558785504a30 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f2524f63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558785460d20 con 0x5587854ff8c0 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f251b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f251b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60666/0 (socket says 172.21.15.59:60666) 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f251b400640 1 -- 172.21.15.59:0/1617313969 learned_addr learned my addr 172.21.15.59:0/1617313969 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.666+0000 7f251aa00640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x5587854ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251aa00640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x558785506ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 -- 172.21.15.59:0/1617313969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 msgr2=0x558785506ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x558785506ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 -- 172.21.15.59:0/1617313969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 msgr2=0x5587854ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x5587854ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 -- 172.21.15.59:0/1617313969 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558785421840 con 0x5587854ff8c0 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251aa00640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x5587854ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f251b400640 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f250c010580 tx=0x7f250c010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267bff091359fe60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f2507e00640 1 -- 172.21.15.59:0/1617313969 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c00e890 con 0x5587854ff8c0 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f2507e00640 1 -- 172.21.15.59:0/1617313969 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c01c480 con 0x5587854ff8c0 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.667+0000 7f2507e00640 1 -- 172.21.15.59:0/1617313969 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0116a0 con 0x5587854ff8c0 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 -- 172.21.15.59:0/1617313969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 msgr2=0x5587855043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f250c010580 tx=0x7f250c010a40 comp rx=0 tx=0).stop 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 -- 172.21.15.59:0/1617313969 shutdown_connections 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x558785506ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x5587855043c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 --2- 172.21.15.59:0/1617313969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x5587854ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 -- 172.21.15.59:0/1617313969 >> 172.21.15.59:0/1617313969 conn(0x5587854fa440 msgr2=0x5587854fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 -- 172.21.15.59:0/1617313969 shutdown_connections 2024-04-02T17:15:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.668+0000 7f2524f63740 1 -- 172.21.15.59:0/1617313969 wait complete. 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.669+0000 7f2524f63740 1 -- start start 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.669+0000 7f2524f63740 1 Processor -- start 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x558785509440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x558785509980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2513e00640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 0x7f25000a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587855079e0 con 0x5587854feed0 2024-04-02T17:15:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558785420660 con 0x558785504a30 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558785507b60 con 0x5587854ff8c0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2524f63740 10 client.0 _ll_register_callbacks cb 0x558785420050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f251be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f251be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:41696/0 (socket says 172.21.15.59:41696) 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f251be00640 1 -- 172.21.15.59:0/269643069 learned_addr learned my addr 172.21.15.59:0/269643069 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f2513e00640 1 -- 172.21.15.59:0/1609063605 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f25140f2ab0 con 0x7f251c1056d0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f251aa00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x558785509440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.670+0000 7f251b400640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 0x7f25000a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251aa00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x558785509980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 -- 172.21.15.59:0/269643069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 msgr2=0x558785509980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x558785509980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 -- 172.21.15.59:0/269643069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 msgr2=0x558785509440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x558785509440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 -- 172.21.15.59:0/269643069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878542fe80 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251aa00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x558785509980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f251be00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25140324c0 tx=0x7f2514031670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba86d64ceaabca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f2504c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140be630 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2514004680 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.671+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c01c480 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f251b400640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 0x7f25000a60a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2514087990 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2524f63740 10 client.26144 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558785179210 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c01c480 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f251403c030 con 0x558785504a30 2024-04-02T17:15:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2524f63740 10 client.26144 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.672+0000 7f2524f63740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f2524f63740 20 client.26144 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558785515310 0x5587855177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f2524f63740 4 client.26144 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558785414cd0 con 0x558785515310 2024-04-02T17:15:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f251aa00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558785515310 0x5587855177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f251aa00640 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558785515310 0x5587855177d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c405775778d7d2 server_cookie=13665203f78c16b5 in_seq=0 out_seq=0 2024-04-02T17:15:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.673+0000 7f2504c00640 10 client.26144 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.674+0000 7f2504c00640 1 -- 172.21.15.59:0/269643069 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558785414cd0 con 0x558785515310 2024-04-02T17:15:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.674+0000 7f2504c00640 10 client.26144 handle_command_reply: tid=0 2024-04-02T17:15:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.674+0000 7f2524f63740 1 client.26144 shutdown 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.674+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 msgr2=0x5587855109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.674+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25140324c0 tx=0x7f2514031670 comp rx=0 tx=0).stop 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 shutdown_connections 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558785515310 0x5587855177d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587855109c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854ff8c0 0x558785509980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 --2- 172.21.15.59:0/269643069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854feed0 0x558785509440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2504c00640 0 client.26144 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 >> 172.21.15.59:0/269643069 conn(0x5587854fa440 msgr2=0x5587854f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 shutdown_connections 2024-04-02T17:15:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 1 -- 172.21.15.59:0/269643069 wait complete. 2024-04-02T17:15:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.675+0000 7f2524f63740 20 client.26144 trim_cache size 0 max 16384 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 Processor -- start 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 -- start start 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 0x5587854f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587854f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1af5f0 con 0x5587854ff8c0 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1b11f0 con 0x558785504a30 2024-04-02T17:15:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f2524f63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1afcc0 con 0x5587854feed0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f251aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f251b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 0x5587854f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f251aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47494/0 (socket says 172.21.15.59:47494) 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f251aa00640 1 -- 172.21.15.59:0/539825780 learned_addr learned my addr 172.21.15.59:0/539825780 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.696+0000 7f251be00640 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587854f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 -- 172.21.15.59:0/539825780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 msgr2=0x5587854f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587854f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 -- 172.21.15.59:0/539825780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 msgr2=0x5587855108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 -- 172.21.15.59:0/539825780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558785245620 con 0x5587854feed0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251aa00640 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f251b400640 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 0x5587854f8db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5587854fb3b0 tx=0x7f250c022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa87541fab56c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.697+0000 7f2507400640 1 -- 172.21.15.59:0/539825780 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c01ce00 con 0x5587854feed0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2507400640 1 -- 172.21.15.59:0/539825780 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c003ad0 con 0x5587854feed0 2024-04-02T17:15:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2507400640 1 -- 172.21.15.59:0/539825780 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c00eb90 con 0x5587854feed0 2024-04-02T17:15:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 -- 172.21.15.59:0/539825780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 msgr2=0x5587854f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 0x5587854f8db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5587854fb3b0 tx=0x7f250c022000 comp rx=0 tx=0).stop 2024-04-02T17:15:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 -- 172.21.15.59:0/539825780 shutdown_connections 2024-04-02T17:15:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x5587855108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558785504a30 0x5587854f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 --2- 172.21.15.59:0/539825780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587854feed0 0x5587854f8db0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.698+0000 7f2524f63740 1 -- 172.21.15.59:0/539825780 >> 172.21.15.59:0/539825780 conn(0x5587854fa440 msgr2=0x5587854fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.699+0000 7f2524f63740 1 -- 172.21.15.59:0/539825780 shutdown_connections 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.699+0000 7f2524f63740 1 -- 172.21.15.59:0/539825780 wait complete. 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.699+0000 7f2524f63740 1 -- start start 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.699+0000 7f2524f63740 1 Processor -- start 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x55878550e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 0x55878550f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1b0a60 con 0x5587854ff8c0 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1b0390 con 0x5587854feed0 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ad5d0 con 0x558785504a30 2024-04-02T17:15:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f2524f63740 10 client.0 _ll_register_callbacks cb 0x7f251400b230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f251b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f251b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:41708/0 (socket says 172.21.15.59:41708) 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f251b400640 1 -- 172.21.15.59:0/1916528648 learned_addr learned my addr 172.21.15.59:0/1916528648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f251be00640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 0x55878550f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.700+0000 7f251aa00640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x55878550e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 -- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 msgr2=0x55878550ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 -- 172.21.15.59:0/1916528648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 msgr2=0x55878550e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x55878550e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 -- 172.21.15.59:0/1916528648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878550fa10 con 0x558785504a30 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251b400640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f251be00640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 0x55878550f420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25140bec60 tx=0x7f2514045ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672adf6c42ba8b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f2504c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140bf130 con 0x558785504a30 2024-04-02T17:15:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f25140c74b0 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.701+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c003ad0 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25140314b0 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 10 client.26179 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55878557b950 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c003ad0 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f25140b9450 con 0x558785504a30 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 10 client.26179 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 20 client.26179 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55878557c170 0x55878557e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 4 client.26179 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587854722b0 con 0x55878557c170 2024-04-02T17:15:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.702+0000 7f251b400640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55878557c170 0x55878557e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.703+0000 7f251b400640 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55878557c170 0x55878557e590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d937c78c554a870 server_cookie=10b69fcec89171a2 in_seq=0 out_seq=0 2024-04-02T17:15:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.703+0000 7f2504c00640 10 client.26179 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2504c00640 1 -- 172.21.15.59:0/1916528648 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587854722b0 con 0x55878557c170 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2504c00640 10 client.26179 handle_command_reply: tid=0 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 client.26179 shutdown 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 msgr2=0x55878550f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 0x55878550f420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25140bec60 tx=0x7f2514045ec0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 shutdown_connections 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55878557c170 0x55878557e590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587854ff8c0 0x55878550e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2504c00640 0 client.26179 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558785504a30 0x55878550f420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 --2- 172.21.15.59:0/1916528648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5587854feed0 0x55878550ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 >> 172.21.15.59:0/1916528648 conn(0x5587854fa440 msgr2=0x5587854fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 shutdown_connections 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.726+0000 7f2524f63740 1 -- 172.21.15.59:0/1916528648 wait complete. 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2524f63740 20 client.26179 trim_cache size 0 max 16384 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:15:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:15:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:15:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:15:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 msgr2=0x7f25000a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 0x7f25000a60a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c013040 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 msgr2=0x7f251c1a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 0x7f251c1a5890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2514037040 tx=0x7f2514035d10 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 shutdown_connections 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f251c1056d0 0x7f251c1a5890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f251c102d20 0x7f251c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f25000a3be0 0x7f25000a60a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 --2- 172.21.15.59:0/1609063605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c1003b0 0x7f251c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.727+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 >> 172.21.15.59:0/1609063605 conn(0x7f251c0fbfb0 msgr2=0x7f251c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.728+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 shutdown_connections 2024-04-02T17:15:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:42.728+0000 7f2522200640 1 -- 172.21.15.59:0/1609063605 wait complete. 2024-04-02T17:15:43.037 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43145 3.58 MB/sec warmup 70 sec latency 22.523 ms 2024-04-02T17:15:43.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:42 smithi059 ceph-mon[31664]: pgmap v467: 97 pgs: 97 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 584 KiB/s wr, 107 op/s 2024-04-02T17:15:43.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:42 smithi164 ceph-mon[36499]: pgmap v467: 97 pgs: 97 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 584 KiB/s wr, 107 op/s 2024-04-02T17:15:43.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:42 smithi154 ceph-mon[36479]: pgmap v467: 97 pgs: 97 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 584 KiB/s wr, 107 op/s 2024-04-02T17:15:44.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43264 3.53 MB/sec warmup 71 sec latency 852.563 ms 2024-04-02T17:15:45.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43264 3.48 MB/sec warmup 72 sec latency 1852.701 ms 2024-04-02T17:15:45.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:44 smithi059 ceph-mon[31664]: pgmap v468: 97 pgs: 97 active+clean; 29 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 951 KiB/s wr, 122 op/s 2024-04-02T17:15:45.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:44 smithi164 ceph-mon[36499]: pgmap v468: 97 pgs: 97 active+clean; 29 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 951 KiB/s wr, 122 op/s 2024-04-02T17:15:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:44 smithi154 ceph-mon[36479]: pgmap v468: 97 pgs: 97 active+clean; 29 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 951 KiB/s wr, 122 op/s 2024-04-02T17:15:46.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43264 3.43 MB/sec warmup 73 sec latency 2852.880 ms 2024-04-02T17:15:47.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43264 3.39 MB/sec warmup 74 sec latency 3852.986 ms 2024-04-02T17:15:47.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:46 smithi059 ceph-mon[31664]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-02T17:15:47.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:46 smithi164 ceph-mon[36499]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-02T17:15:47.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:46 smithi154 ceph-mon[36479]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-02T17:15:48.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43264 3.34 MB/sec warmup 75 sec latency 4853.129 ms 2024-04-02T17:15:49.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43388 3.30 MB/sec warmup 76 sec latency 4960.813 ms 2024-04-02T17:15:49.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:48 smithi059 ceph-mon[31664]: pgmap v470: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-04-02T17:15:49.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:48 smithi164 ceph-mon[36499]: pgmap v470: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-04-02T17:15:49.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:48 smithi154 ceph-mon[36479]: pgmap v470: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-04-02T17:15:50.038 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43388 3.26 MB/sec warmup 77 sec latency 1679.543 ms 2024-04-02T17:15:51.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43388 3.22 MB/sec warmup 78 sec latency 2679.700 ms 2024-04-02T17:15:51.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:50 smithi059 ceph-mon[31664]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-04-02T17:15:51.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:50 smithi164 ceph-mon[36499]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-04-02T17:15:51.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:50 smithi154 ceph-mon[36479]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-04-02T17:15:52.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43388 3.18 MB/sec warmup 79 sec latency 3679.905 ms 2024-04-02T17:15:53.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 43388 3.14 MB/sec warmup 80 sec latency 4680.069 ms 2024-04-02T17:15:53.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:52 smithi059 ceph-mon[31664]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T17:15:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:52 smithi164 ceph-mon[36499]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T17:15:53.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:52 smithi154 ceph-mon[36479]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T17:15:54.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 44936 3.18 MB/sec warmup 81 sec latency 4787.088 ms 2024-04-02T17:15:55.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46303 3.27 MB/sec warmup 82 sec latency 13.820 ms 2024-04-02T17:15:55.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:54 smithi059 ceph-mon[31664]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T17:15:55.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:54 smithi164 ceph-mon[36499]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T17:15:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:54 smithi154 ceph-mon[36479]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2616776578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 msgr2=0x7fe2dc0749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2616776578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc0749e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009890 tx=0x7fe2c402bb70 comp rx=0 tx=0).stop 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2616776578 shutdown_connections 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2616776578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc0749e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2616776578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc10e100 0x7fe2dc1104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.897+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2616776578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc075b40 0x7fe2dc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.898+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2616776578 >> 172.21.15.59:0/2616776578 conn(0x7fe2dc0fd0c0 msgr2=0x7fe2dc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.898+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2616776578 shutdown_connections 2024-04-02T17:15:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.898+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2616776578 wait complete. 2024-04-02T17:15:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 Processor -- start 2024-04-02T17:15:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 -- start start 2024-04-02T17:15:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc10e100 0x7fe2dc1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc113dc0 con 0x7fe2dc074560 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc113c40 con 0x7fe2dc075b40 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.899+0000 7fe2e3c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc113f40 con 0x7fe2dc10e100 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57446/0 (socket says 172.21.15.59:57446) 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 -- 172.21.15.59:0/2287693999 learned_addr learned my addr 172.21.15.59:0/2287693999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2e1c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc10e100 0x7fe2dc1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 -- 172.21.15.59:0/2287693999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc10e100 msgr2=0x7fe2dc1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc10e100 0x7fe2dc1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 -- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 msgr2=0x7fe2dc19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 -- 172.21.15.59:0/2287693999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a5fe0 con 0x7fe2dc075b40 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.900+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c4032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.901+0000 7fe2d9600640 1 -- 172.21.15.59:0/2287693999 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4041020 con 0x7fe2dc075b40 2024-04-02T17:15:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.901+0000 7fe2d9600640 1 -- 172.21.15.59:0/2287693999 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c402c9f0 con 0x7fe2dc075b40 2024-04-02T17:15:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.901+0000 7fe2d9600640 1 -- 172.21.15.59:0/2287693999 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4039040 con 0x7fe2dc075b40 2024-04-02T17:15:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.901+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a6270 con 0x7fe2dc075b40 2024-04-02T17:15:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.901+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a6760 con 0x7fe2dc075b40 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 Processor -- start 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2d9600640 1 -- 172.21.15.59:0/2287693999 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe2c4044020 con 0x7fe2dc075b40 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 -- start start 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a6599dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a658fbd70 con 0x556a65995ed0 2024-04-02T17:15:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a658a42a0 con 0x556a659968c0 2024-04-02T17:15:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e686c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a658f7d20 con 0x556a6599ba30 2024-04-02T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57458/0 (socket says 172.21.15.59:57458) 2024-04-02T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2e1200640 1 -- 172.21.15.59:0/2686935587 learned_addr learned my addr 172.21.15.59:0/2686935587 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.906+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a6599dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 -- 172.21.15.59:0/2686935587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 msgr2=0x556a6599dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a6599dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 -- 172.21.15.59:0/2686935587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 msgr2=0x556a659962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 -- 172.21.15.59:0/2686935587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a658b8840 con 0x556a659968c0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a6599dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40104a0 tx=0x7fe2d4010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67937c9c6967cd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2d9600640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 0x7fe2c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.907+0000 7fe2d2000640 1 -- 172.21.15.59:0/2686935587 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4017070 con 0x556a659968c0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.908+0000 7fe2d2000640 1 -- 172.21.15.59:0/2686935587 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4011be0 con 0x556a659968c0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.908+0000 7fe2d2000640 1 -- 172.21.15.59:0/2686935587 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400ed90 con 0x556a659968c0 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.908+0000 7fe2d9600640 1 -- 172.21.15.59:0/2287693999 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe2c40f1da0 con 0x7fe2dc075b40 2024-04-02T17:15:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.908+0000 7fe2e1c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 0x7fe2c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 -- 172.21.15.59:0/2686935587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 msgr2=0x556a6599b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e1c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 0x7fe2c00a6190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x556a65997120 tx=0x7fe2cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40104a0 tx=0x7fe2d4010960 comp rx=0 tx=0).stop 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 -- 172.21.15.59:0/2686935587 shutdown_connections 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a6599dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a659968c0 0x556a6599b3c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2686935587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 -- 172.21.15.59:0/2686935587 >> 172.21.15.59:0/2686935587 conn(0x556a65991440 msgr2=0x556a65991840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 -- 172.21.15.59:0/2686935587 shutdown_connections 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.909+0000 7fe2e686c740 1 -- 172.21.15.59:0/2686935587 wait complete. 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 -- start start 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 Processor -- start 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a659968c0 0x556a659a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a659a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a6599e9e0 con 0x556a65995ed0 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a658b7660 con 0x556a6599ba30 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a6599eb60 con 0x556a659968c0 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.910+0000 7fe2e686c740 10 client.0 _ll_register_callbacks cb 0x556a658b7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a659a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a659a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57460/0 (socket says 172.21.15.59:57460) 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1c00640 1 -- 172.21.15.59:0/1906091792 learned_addr learned my addr 172.21.15.59:0/1906091792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 -- 172.21.15.59:0/1906091792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a659968c0 msgr2=0x556a659a08c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a659968c0 0x556a659a08c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 -- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 msgr2=0x556a659a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a659a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.911+0000 7fe2e1200640 1 -- 172.21.15.59:0/1906091792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a658c6e80 con 0x556a65995ed0 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659a0380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x556a659923b0 tx=0x7fe2d400efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c0916a9c153d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2caa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4017040 con 0x556a65995ed0 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe2d4004a40 con 0x556a65995ed0 2024-04-02T17:15:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d4011be0 con 0x556a65995ed0 2024-04-02T17:15:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe2d4004c40 con 0x556a65995ed0 2024-04-02T17:15:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2e686c740 10 client.16077 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.912+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a65610210 con 0x556a65995ed0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4011be0 con 0x556a65995ed0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fe2d4005440 con 0x556a65995ed0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 10 client.16077 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 20 client.16077 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a659abf40 0x556a659ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 4 client.16077 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a658abcd0 con 0x556a659abf40 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.913+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a659abf40 0x556a659ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.914+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a659abf40 0x556a659ae3e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ca2ce2776937f3 server_cookie=f8225461cb1e5afd in_seq=0 out_seq=0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.914+0000 7fe2caa00640 10 client.16077 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.914+0000 7fe2caa00640 1 -- 172.21.15.59:0/1906091792 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a658abcd0 con 0x556a659abf40 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2caa00640 10 client.16077 handle_command_reply: tid=0 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 client.16077 shutdown 2024-04-02T17:15:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 msgr2=0x556a659a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659a0380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x556a659923b0 tx=0x7fe2d400efc0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 shutdown_connections 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a659abf40 0x556a659ae3e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a659a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a659968c0 0x556a659a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1906091792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a65995ed0 0x556a659a0380 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 >> 172.21.15.59:0/1906091792 conn(0x556a65991440 msgr2=0x556a65990640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2caa00640 0 client.16077 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 shutdown_connections 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.915+0000 7fe2e686c740 1 -- 172.21.15.59:0/1906091792 wait complete. 2024-04-02T17:15:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.916+0000 7fe2e686c740 20 client.16077 trim_cache size 0 max 16384 2024-04-02T17:15:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 Processor -- start 2024-04-02T17:15:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 -- start start 2024-04-02T17:15:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 0x556a65994db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a6599f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1af700 con 0x556a659968c0 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b1300 con 0x556a6599ba30 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.937+0000 7fe2e686c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1afdd0 con 0x556a65995ed0 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37828/0 (socket says 172.21.15.59:37828) 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2dbe00640 1 -- 172.21.15.59:0/2348995622 learned_addr learned my addr 172.21.15.59:0/2348995622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 0x556a65994db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1c00640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a6599f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 -- 172.21.15.59:0/2348995622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 msgr2=0x556a6599f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a6599f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 -- 172.21.15.59:0/2348995622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 msgr2=0x556a659a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 -- 172.21.15.59:0/2348995622 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a656dc620 con 0x556a65995ed0 2024-04-02T17:15:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.938+0000 7fe2e1200640 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 0x556a65994db0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005810 tx=0x7fe2d4010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fe38bbd286adaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.939+0000 7fe2d1600640 1 -- 172.21.15.59:0/2348995622 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40049c0 con 0x556a65995ed0 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.939+0000 7fe2d1600640 1 -- 172.21.15.59:0/2348995622 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4029070 con 0x556a65995ed0 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.939+0000 7fe2d1600640 1 -- 172.21.15.59:0/2348995622 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400e8f0 con 0x556a65995ed0 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- 172.21.15.59:0/2348995622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 msgr2=0x556a65994db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 0x556a65994db0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005810 tx=0x7fe2d4010f90 comp rx=0 tx=0).stop 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- 172.21.15.59:0/2348995622 shutdown_connections 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a6599ba30 0x556a6599f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 --2- 172.21.15.59:0/2348995622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a65995ed0 0x556a65994db0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- 172.21.15.59:0/2348995622 >> 172.21.15.59:0/2348995622 conn(0x556a65991440 msgr2=0x556a659946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- 172.21.15.59:0/2348995622 shutdown_connections 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- 172.21.15.59:0/2348995622 wait complete. 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 -- start start 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.940+0000 7fe2e686c740 1 Processor -- start 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b0b70 con 0x556a659968c0 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b04a0 con 0x556a65995ed0 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1ad6e0 con 0x556a6599ba30 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e686c740 10 client.0 _ll_register_callbacks cb 0x7fe2d4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36778/0 (socket says 172.21.15.59:36778) 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57478/0 (socket says 172.21.15.59:57478) 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2e1200640 1 -- 172.21.15.59:0/1136483824 learned_addr learned my addr 172.21.15.59:0/1136483824 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.941+0000 7fe2dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37844/0 (socket says 172.21.15.59:37844) 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 -- 172.21.15.59:0/1136483824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 msgr2=0x556a659a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 -- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 msgr2=0x556a659a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 -- 172.21.15.59:0/1136483824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a659a6400 con 0x556a659968c0 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2e1c00640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2dbe00640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402cbc0 tx=0x7fe2c40f2130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392c7f7dcc1b125e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2caa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40bd980 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe2c40bc0c0 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.942+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d4029070 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe2c40be720 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 10 client.16083 fetch_fsmap learned FSMap version 31 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556a659a77e0 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4029070 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fe2c403c030 con 0x556a659968c0 2024-04-02T17:15:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 10 client.16083 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 20 client.16083 populate_metadata read hostname 'smithi059' 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a65a129f0 0x556a65a14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 4 client.16083 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.943+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a659092b0 con 0x556a65a129f0 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.944+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a65a129f0 0x556a65a14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.944+0000 7fe2e1200640 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a65a129f0 0x556a65a14e10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bfc48ed9060c4c server_cookie=7496531a9ffddef in_seq=0 out_seq=0 2024-04-02T17:15:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.944+0000 7fe2caa00640 10 client.16083 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.968+0000 7fe2caa00640 1 -- 172.21.15.59:0/1136483824 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a659092b0 con 0x556a65a129f0 2024-04-02T17:15:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.968+0000 7fe2caa00640 10 client.16083 handle_command_reply: tid=0 2024-04-02T17:15:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 client.16083 shutdown 2024-04-02T17:15:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 msgr2=0x556a659a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402cbc0 tx=0x7fe2c40f2130 comp rx=0 tx=0).stop 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 shutdown_connections 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a65a129f0 0x556a65a14e10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a659968c0 0x556a659a4e00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a6599ba30 0x556a659a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 --2- 172.21.15.59:0/1136483824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a65995ed0 0x556a659a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 >> 172.21.15.59:0/1136483824 conn(0x556a65991440 msgr2=0x556a65994f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2caa00640 0 client.16083 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 shutdown_connections 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 1 -- 172.21.15.59:0/1136483824 wait complete. 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.969+0000 7fe2e686c740 20 client.16083 trim_cache size 0 max 16384 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:15:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:15:55.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:15:55.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.970+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 msgr2=0x7fe2c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.970+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 0x7fe2c00a6190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x556a65997120 tx=0x7fe2cc009260 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.970+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 msgr2=0x7fe2dc19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.970+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c4032430 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 shutdown_connections 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2dc10e100 0x7fe2dc1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2dc075b40 0x7fe2dc19e900 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe2c00a3cd0 0x7fe2c00a6190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 --2- 172.21.15.59:0/2287693999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2dc074560 0x7fe2dc19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 >> 172.21.15.59:0/2287693999 conn(0x7fe2dc0fd0c0 msgr2=0x7fe2dc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 shutdown_connections 2024-04-02T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:15:55.971+0000 7fe2e3c00640 1 -- 172.21.15.59:0/2287693999 wait complete. 2024-04-02T17:15:56.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46684 3.25 MB/sec warmup 83 sec latency 24.748 ms 2024-04-02T17:15:57.039 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46814 3.21 MB/sec warmup 84 sec latency 837.104 ms 2024-04-02T17:15:57.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:56 smithi059 ceph-mon[31664]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 673 KiB/s wr, 69 op/s 2024-04-02T17:15:57.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:56 smithi164 ceph-mon[36499]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 673 KiB/s wr, 69 op/s 2024-04-02T17:15:57.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:56 smithi154 ceph-mon[36479]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 673 KiB/s wr, 69 op/s 2024-04-02T17:15:58.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46814 3.17 MB/sec warmup 85 sec latency 1837.246 ms 2024-04-02T17:15:59.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46870 3.14 MB/sec warmup 86 sec latency 1944.250 ms 2024-04-02T17:15:59.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:15:58 smithi059 ceph-mon[31664]: pgmap v475: 97 pgs: 97 active+clean; 29 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 197 KiB/s wr, 62 op/s 2024-04-02T17:15:59.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:15:58 smithi164 ceph-mon[36499]: pgmap v475: 97 pgs: 97 active+clean; 29 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 197 KiB/s wr, 62 op/s 2024-04-02T17:15:59.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:15:58 smithi154 ceph-mon[36479]: pgmap v475: 97 pgs: 97 active+clean; 29 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 197 KiB/s wr, 62 op/s 2024-04-02T17:16:00.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46870 3.10 MB/sec warmup 87 sec latency 1798.244 ms 2024-04-02T17:16:01.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46902 3.07 MB/sec warmup 88 sec latency 2774.311 ms 2024-04-02T17:16:01.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:00 smithi059 ceph-mon[31664]: pgmap v476: 97 pgs: 97 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 186 KiB/s wr, 57 op/s 2024-04-02T17:16:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:00 smithi164 ceph-mon[36499]: pgmap v476: 97 pgs: 97 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 186 KiB/s wr, 57 op/s 2024-04-02T17:16:01.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:00 smithi154 ceph-mon[36479]: pgmap v476: 97 pgs: 97 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 186 KiB/s wr, 57 op/s 2024-04-02T17:16:02.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46938 3.03 MB/sec warmup 89 sec latency 966.757 ms 2024-04-02T17:16:03.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 46938 3.00 MB/sec warmup 90 sec latency 1966.918 ms 2024-04-02T17:16:03.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:02 smithi059 ceph-mon[31664]: pgmap v477: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-04-02T17:16:03.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:02 smithi164 ceph-mon[36499]: pgmap v477: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-04-02T17:16:03.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:02 smithi154 ceph-mon[36479]: pgmap v477: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 207 KiB/s wr, 61 op/s 2024-04-02T17:16:04.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 48448 3.04 MB/sec warmup 91 sec latency 2073.579 ms 2024-04-02T17:16:05.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 49796 3.12 MB/sec warmup 92 sec latency 16.520 ms 2024-04-02T17:16:05.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:04 smithi164 ceph-mon[36499]: pgmap v478: 97 pgs: 97 active+clean; 30 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 409 KiB/s wr, 69 op/s 2024-04-02T17:16:05.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:04 smithi154 ceph-mon[36479]: pgmap v478: 97 pgs: 97 active+clean; 30 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 409 KiB/s wr, 69 op/s 2024-04-02T17:16:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:04 smithi059 ceph-mon[31664]: pgmap v478: 97 pgs: 97 active+clean; 30 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 409 KiB/s wr, 69 op/s 2024-04-02T17:16:06.040 INFO:tasks.workunit.client.0.smithi059.stdout: 1 50154 3.10 MB/sec warmup 93 sec latency 19.998 ms 2024-04-02T17:16:07.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 51328 3.13 MB/sec warmup 94 sec latency 23.859 ms 2024-04-02T17:16:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:06 smithi164 ceph-mon[36499]: pgmap v479: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 806 KiB/s wr, 87 op/s 2024-04-02T17:16:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:06 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:16:07.101 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:06 smithi059 ceph-mon[31664]: pgmap v479: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 806 KiB/s wr, 87 op/s 2024-04-02T17:16:07.101 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:06 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:16:07.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:06 smithi154 ceph-mon[36479]: pgmap v479: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 806 KiB/s wr, 87 op/s 2024-04-02T17:16:07.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:06 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:16:08.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 53046 3.19 MB/sec warmup 95 sec latency 18.973 ms 2024-04-02T17:16:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:16:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:16:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:16:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:16:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:16:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:16:08.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:16:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:16:08.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:16:09.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 53648 3.21 MB/sec warmup 96 sec latency 23.433 ms 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 -- 172.21.15.59:0/3116776334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64001036e0 msgr2=0x7f6400109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 --2- 172.21.15.59:0/3116776334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64001036e0 0x7f6400109f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009be0 tx=0x7f63f802bef0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 -- 172.21.15.59:0/3116776334 shutdown_connections 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 --2- 172.21.15.59:0/3116776334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 0x7f64001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 --2- 172.21.15.59:0/3116776334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64001036e0 0x7f6400109f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 --2- 172.21.15.59:0/3116776334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400101b30 0x7f6400101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 -- 172.21.15.59:0/3116776334 >> 172.21.15.59:0/3116776334 conn(0x7f64000fd0c0 msgr2=0x7f64000ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 -- 172.21.15.59:0/3116776334 shutdown_connections 2024-04-02T17:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.134+0000 7f6406600640 1 -- 172.21.15.59:0/3116776334 wait complete. 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.135+0000 7f6406600640 1 Processor -- start 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 -- start start 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 0x7f640019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001036e0 0x7f64001a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d7e0 con 0x7f6400102d20 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d660 con 0x7f6400101b30 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f6406600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d960 con 0x7f64001036e0 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44062/0 (socket says 172.21.15.59:44062) 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f63ff400640 1 -- 172.21.15.59:0/1086350292 learned_addr learned my addr 172.21.15.59:0/1086350292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001036e0 0x7f64001a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.136+0000 7f63fea00640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 0x7f640019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 -- 172.21.15.59:0/1086350292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001036e0 msgr2=0x7f64001a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001036e0 0x7f64001a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 -- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 msgr2=0x7f640019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 0x7f640019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 -- 172.21.15.59:0/1086350292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64001a1b40 con 0x7f6400101b30 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63ff400640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e9f0 tx=0x7f63f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63f7e00640 1 -- 172.21.15.59:0/1086350292 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000ccc0 con 0x7f6400101b30 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63f7e00640 1 -- 172.21.15.59:0/1086350292 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f00035f0 con 0x7f6400101b30 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f63f7e00640 1 -- 172.21.15.59:0/1086350292 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000ce60 con 0x7f6400101b30 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.137+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64001a1e30 con 0x7f6400101b30 2024-04-02T17:16:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.138+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a22f0 con 0x7f6400101b30 2024-04-02T17:16:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.142+0000 7f640921c740 1 Processor -- start 2024-04-02T17:16:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.142+0000 7f640921c740 1 -- start start 2024-04-02T17:16:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f63f7e00640 1 -- 172.21.15.59:0/1086350292 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f63f00037a0 con 0x7f6400101b30 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55687d738d70 con 0x55687d7d38c0 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55687d6e12a0 con 0x55687d7d2ed0 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f640921c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55687d734d20 con 0x55687d7d8a30 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44064/0 (socket says 172.21.15.59:44064) 2024-04-02T17:16:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.143+0000 7f63ffe00640 1 -- 172.21.15.59:0/1384073554 learned_addr learned my addr 172.21.15.59:0/1384073554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ff400640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63fea00640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 -- 172.21.15.59:0/1384073554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 msgr2=0x55687d7daec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7daec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 -- 172.21.15.59:0/1384073554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 msgr2=0x55687d7d83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7d83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 -- 172.21.15.59:0/1384073554 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687d6f5840 con 0x55687d7d2ed0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ff400640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63fea00640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.144+0000 7f63ffe00640 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f63f802c500 tx=0x7f63f8009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81017e310e05a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63f4c00640 1 -- 172.21.15.59:0/1384073554 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8043030 con 0x55687d7d2ed0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63f4c00640 1 -- 172.21.15.59:0/1384073554 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8033580 con 0x55687d7d2ed0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63f4c00640 1 -- 172.21.15.59:0/1384073554 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f802c890 con 0x55687d7d2ed0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63f7e00640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 0x7f63e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63f7e00640 1 -- 172.21.15.59:0/1086350292 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f63f00d0610 con 0x7f6400101b30 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.145+0000 7f63ff400640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 0x7f63e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 -- 172.21.15.59:0/1384073554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 msgr2=0x55687d7d32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f63f802c500 tx=0x7f63f8009330 comp rx=0 tx=0).stop 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 -- 172.21.15.59:0/1384073554 shutdown_connections 2024-04-02T17:16:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 --2- 172.21.15.59:0/1384073554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7d32f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 -- 172.21.15.59:0/1384073554 >> 172.21.15.59:0/1384073554 conn(0x55687d7ce440 msgr2=0x55687d7ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 -- 172.21.15.59:0/1384073554 shutdown_connections 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f640921c740 1 -- 172.21.15.59:0/1384073554 wait complete. 2024-04-02T17:16:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.146+0000 7f63ff400640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 0x7f63e40a6190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ce420 tx=0x7f63f0003f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 -- start start 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 Processor -- start 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55687d7db980 con 0x55687d7d2ed0 2024-04-02T17:16:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55687d6f4660 con 0x55687d7d38c0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.147+0000 7f640921c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55687d7dbb00 con 0x55687d7d8a30 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f640921c740 10 client.0 _ll_register_callbacks cb 0x55687d6f4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44074/0 (socket says 172.21.15.59:44074) 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 -- 172.21.15.59:0/2451397917 learned_addr learned my addr 172.21.15.59:0/2451397917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 -- 172.21.15.59:0/2451397917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 msgr2=0x55687d7e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 -- 172.21.15.59:0/2451397917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 msgr2=0x55687d7dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 -- 172.21.15.59:0/2451397917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55687d703e80 con 0x55687d7d38c0 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63fea00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0136e0 tx=0x7f63ec013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5159427e30d50c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.148+0000 7f63e9600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec020060 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f63ec011dd0 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f8033580 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63ec01b430 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f640921c740 10 client.26180 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55687d44d210 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.149+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8033580 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f63ec024030 con 0x55687d7d38c0 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 10 client.26180 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 20 client.26180 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d7e9170 0x55687d7eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 4 client.26180 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55687d6e8cd0 con 0x55687d7e9170 2024-04-02T17:16:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.150+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d7e9170 0x55687d7eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.151+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d7e9170 0x55687d7eb610 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf693da0cef898c8 server_cookie=150ff3b0c0505b6c in_seq=0 out_seq=0 2024-04-02T17:16:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.151+0000 7f63e9600640 10 client.26180 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f63e9600640 1 -- 172.21.15.59:0/2451397917 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55687d6e8cd0 con 0x55687d7e9170 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f63e9600640 10 client.26180 handle_command_reply: tid=0 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 client.26180 shutdown 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 msgr2=0x55687d7dd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0136e0 tx=0x7f63ec013ba0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 shutdown_connections 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d7e9170 0x55687d7eb610 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 --2- 172.21.15.59:0/2451397917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f63e9600640 0 client.26180 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 >> 172.21.15.59:0/2451397917 conn(0x55687d7ce440 msgr2=0x55687d7cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.152+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 shutdown_connections 2024-04-02T17:16:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.153+0000 7f640921c740 1 -- 172.21.15.59:0/2451397917 wait complete. 2024-04-02T17:16:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.153+0000 7f640921c740 20 client.26180 trim_cache size 0 max 16384 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.172+0000 7f640921c740 1 Processor -- start 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 -- start start 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ab220 con 0x55687d7d38c0 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ace20 con 0x55687d7d2ed0 2024-04-02T17:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f640921c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ab8f0 con 0x55687d7d8a30 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44076/0 (socket says 172.21.15.59:44076) 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f63ffe00640 1 -- 172.21.15.59:0/2844436175 learned_addr learned my addr 172.21.15.59:0/2844436175 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.173+0000 7f63fea00640 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 -- 172.21.15.59:0/2844436175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 msgr2=0x55687d7e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 -- 172.21.15.59:0/2844436175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 msgr2=0x55687d7dd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 -- 172.21.15.59:0/2844436175 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687d519620 con 0x55687d7d2ed0 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63f8004ab0 tx=0x7f63f802bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e6117da577a13c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ebe00640 1 -- 172.21.15.59:0/2844436175 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8043040 con 0x55687d7d2ed0 2024-04-02T17:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ebe00640 1 -- 172.21.15.59:0/2844436175 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f802cd70 con 0x55687d7d2ed0 2024-04-02T17:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.174+0000 7f63ebe00640 1 -- 172.21.15.59:0/2844436175 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8034430 con 0x55687d7d2ed0 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 -- 172.21.15.59:0/2844436175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 msgr2=0x55687d7dd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63f8004ab0 tx=0x7f63f802bef0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 -- 172.21.15.59:0/2844436175 shutdown_connections 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7dd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 --2- 172.21.15.59:0/2844436175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7dd2c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 -- 172.21.15.59:0/2844436175 >> 172.21.15.59:0/2844436175 conn(0x55687d7ce440 msgr2=0x55687d7ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 -- 172.21.15.59:0/2844436175 shutdown_connections 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.175+0000 7f640921c740 1 -- 172.21.15.59:0/2844436175 wait complete. 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.176+0000 7f640921c740 1 -- start start 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.176+0000 7f640921c740 1 Processor -- start 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ac690 con 0x55687d7d38c0 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64001abfc0 con 0x55687d7d2ed0 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64001a9200 con 0x55687d7d8a30 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f640921c740 10 client.0 _ll_register_callbacks cb 0x7f63ec009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7e1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44086/0 (socket says 172.21.15.59:44086) 2024-04-02T17:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f63ffe00640 1 -- 172.21.15.59:0/2729992015 learned_addr learned my addr 172.21.15.59:0/2729992015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f63fea00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.177+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 -- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 msgr2=0x55687d7e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 -- 172.21.15.59:0/2729992015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 msgr2=0x55687d7e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 -- 172.21.15.59:0/2729992015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55687d7e36d0 con 0x55687d7d8a30 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63ffe00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e30e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f63f802c430 tx=0x7f63f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca39b66ffd0d6c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63e9600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8043040 con 0x55687d7d8a30 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f63f802c890 con 0x55687d7d8a30 2024-04-02T17:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.178+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f802cd70 con 0x55687d7d8a30 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.179+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63f802ca90 con 0x55687d7d8a30 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.179+0000 7f640921c740 10 client.26209 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.179+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55687d7e49e0 con 0x55687d7d8a30 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.179+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f802cd70 con 0x55687d7d8a30 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f63f80347d0 con 0x55687d7d8a30 2024-04-02T17:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 10 client.26209 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 20 client.26209 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d84f7c0 0x55687d851bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 4 client.26209 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55687d7462b0 con 0x55687d84f7c0 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.180+0000 7f63fea00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d84f7c0 0x55687d851bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.181+0000 7f63fea00640 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d84f7c0 0x55687d851bb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312cd5c5285a4faf server_cookie=260d8378ea2f964f in_seq=0 out_seq=0 2024-04-02T17:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.181+0000 7f63e9600640 10 client.26209 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.208+0000 7f63e9600640 1 -- 172.21.15.59:0/2729992015 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55687d7462b0 con 0x55687d84f7c0 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.208+0000 7f63e9600640 10 client.26209 handle_command_reply: tid=0 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.208+0000 7f640921c740 1 client.26209 shutdown 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 msgr2=0x55687d7e30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e30e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f63f802c430 tx=0x7f63f8004a40 comp rx=0 tx=0).stop 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 shutdown_connections 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55687d84f7c0 0x55687d851bb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55687d7d38c0 0x55687d7e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55687d7d8a30 0x55687d7e30e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f63e9600640 0 client.26209 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 --2- 172.21.15.59:0/2729992015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55687d7d2ed0 0x55687d7e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 >> 172.21.15.59:0/2729992015 conn(0x55687d7ce440 msgr2=0x55687d7dd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 shutdown_connections 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 1 -- 172.21.15.59:0/2729992015 wait complete. 2024-04-02T17:16:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.209+0000 7f640921c740 20 client.26209 trim_cache size 0 max 16384 2024-04-02T17:16:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:16:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:16:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:16:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:16:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 msgr2=0x7f63e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 0x7f63e40a6190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ce420 tx=0x7f63f0003f10 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 msgr2=0x7f6400199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e9f0 tx=0x7f63f000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 shutdown_connections 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3cd0 0x7f63e40a6190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400102d20 0x7f640019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001036e0 0x7f64001a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 --2- 172.21.15.59:0/1086350292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400101b30 0x7f6400199f20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 >> 172.21.15.59:0/1086350292 conn(0x7f64000fd0c0 msgr2=0x7f6400073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 shutdown_connections 2024-04-02T17:16:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:09.210+0000 7f6406600640 1 -- 172.21.15.59:0/1086350292 wait complete. 2024-04-02T17:16:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:09 smithi059 ceph-mon[31664]: pgmap v480: 97 pgs: 97 active+clean; 32 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-02T17:16:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:09 smithi164 ceph-mon[36499]: pgmap v480: 97 pgs: 97 active+clean; 32 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-02T17:16:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:09 smithi154 ceph-mon[36479]: pgmap v480: 97 pgs: 97 active+clean; 32 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 860 KiB/s wr, 106 op/s 2024-04-02T17:16:10.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 54226 3.18 MB/sec warmup 97 sec latency 59.886 ms 2024-04-02T17:16:11.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 55833 3.22 MB/sec warmup 98 sec latency 14.161 ms 2024-04-02T17:16:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:11 smithi164 ceph-mon[36499]: pgmap v481: 97 pgs: 97 active+clean; 31 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 853 KiB/s wr, 91 op/s 2024-04-02T17:16:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:11 smithi154 ceph-mon[36479]: pgmap v481: 97 pgs: 97 active+clean; 31 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 853 KiB/s wr, 91 op/s 2024-04-02T17:16:11.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:11 smithi059 ceph-mon[31664]: pgmap v481: 97 pgs: 97 active+clean; 31 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 853 KiB/s wr, 91 op/s 2024-04-02T17:16:12.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57016 3.30 MB/sec warmup 99 sec latency 24.121 ms 2024-04-02T17:16:13.041 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57291 3.27 MB/sec warmup 100 sec latency 33.150 ms 2024-04-02T17:16:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:13 smithi164 ceph-mon[36499]: pgmap v482: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 970 KiB/s wr, 111 op/s 2024-04-02T17:16:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:13 smithi154 ceph-mon[36479]: pgmap v482: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 970 KiB/s wr, 111 op/s 2024-04-02T17:16:13.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:13 smithi059 ceph-mon[31664]: pgmap v482: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 970 KiB/s wr, 111 op/s 2024-04-02T17:16:14.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57447 3.24 MB/sec warmup 101 sec latency 798.988 ms 2024-04-02T17:16:15.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57447 3.21 MB/sec warmup 102 sec latency 1799.122 ms 2024-04-02T17:16:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:15 smithi164 ceph-mon[36499]: pgmap v483: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T17:16:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:15 smithi154 ceph-mon[36479]: pgmap v483: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T17:16:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:15 smithi059 ceph-mon[31664]: pgmap v483: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T17:16:16.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57447 3.17 MB/sec warmup 103 sec latency 2799.254 ms 2024-04-02T17:16:17.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57447 3.14 MB/sec warmup 104 sec latency 3799.352 ms 2024-04-02T17:16:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:17 smithi164 ceph-mon[36499]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 870 KiB/s wr, 138 op/s 2024-04-02T17:16:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:17 smithi154 ceph-mon[36479]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 870 KiB/s wr, 138 op/s 2024-04-02T17:16:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:17 smithi059 ceph-mon[31664]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 870 KiB/s wr, 138 op/s 2024-04-02T17:16:18.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57447 3.11 MB/sec warmup 105 sec latency 4799.484 ms 2024-04-02T17:16:19.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57571 3.09 MB/sec warmup 106 sec latency 4907.939 ms 2024-04-02T17:16:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:19 smithi164 ceph-mon[36499]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 489 KiB/s wr, 124 op/s 2024-04-02T17:16:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:19 smithi154 ceph-mon[36479]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 489 KiB/s wr, 124 op/s 2024-04-02T17:16:19.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:19 smithi059 ceph-mon[31664]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 489 KiB/s wr, 124 op/s 2024-04-02T17:16:20.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57571 3.06 MB/sec warmup 107 sec latency 1657.499 ms 2024-04-02T17:16:21.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 57571 3.03 MB/sec warmup 108 sec latency 2657.643 ms 2024-04-02T17:16:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:21 smithi154 ceph-mon[36479]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 398 KiB/s wr, 97 op/s 2024-04-02T17:16:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:21 smithi059 ceph-mon[31664]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 398 KiB/s wr, 97 op/s 2024-04-02T17:16:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:21 smithi164 ceph-mon[36499]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 398 KiB/s wr, 97 op/s 2024-04-02T17:16:22.042 INFO:tasks.workunit.client.0.smithi059.stdout: 1 58268 3.05 MB/sec warmup 109 sec latency 3120.447 ms 2024-04-02T17:16:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 -- 172.21.15.59:0/3286407312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244104f40 msgr2=0x7fa2441053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 --2- 172.21.15.59:0/3286407312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244104f40 0x7fa2441053c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009900 tx=0x7fa22c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 -- 172.21.15.59:0/3286407312 shutdown_connections 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 --2- 172.21.15.59:0/3286407312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244105900 0x7fa24410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 --2- 172.21.15.59:0/3286407312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244104f40 0x7fa2441053c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 --2- 172.21.15.59:0/3286407312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244103d50 0x7fa244104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 -- 172.21.15.59:0/3286407312 >> 172.21.15.59:0/3286407312 conn(0x7fa2440ff2c0 msgr2=0x7fa244101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 -- 172.21.15.59:0/3286407312 shutdown_connections 2024-04-02T17:16:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.372+0000 7fa249600640 1 -- 172.21.15.59:0/3286407312 wait complete. 2024-04-02T17:16:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 Processor -- start 2024-04-02T17:16:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 -- start start 2024-04-02T17:16:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244104f40 0x7fa24419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244105900 0x7fa2441a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410f9b0 con 0x7fa244105900 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410f830 con 0x7fa244103d50 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.373+0000 7fa249600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410fb30 con 0x7fa244104f40 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36274/0 (socket says 172.21.15.59:36274) 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 -- 172.21.15.59:0/1924193079 learned_addr learned my addr 172.21.15.59:0/1924193079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa243400640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244105900 0x7fa2441a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242000640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244104f40 0x7fa24419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 -- 172.21.15.59:0/1924193079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244104f40 msgr2=0x7fa24419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244104f40 0x7fa24419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 -- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244105900 msgr2=0x7fa2441a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244105900 0x7fa2441a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 -- 172.21.15.59:0/1924193079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2441a1b20 con 0x7fa244103d50 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.374+0000 7fa242a00640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c970 tx=0x7fa23400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.375+0000 7fa23b400640 1 -- 172.21.15.59:0/1924193079 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234007c10 con 0x7fa244103d50 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.375+0000 7fa23b400640 1 -- 172.21.15.59:0/1924193079 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234007db0 con 0x7fa244103d50 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.375+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2441a1e10 con 0x7fa244103d50 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.375+0000 7fa23b400640 1 -- 172.21.15.59:0/1924193079 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400f940 con 0x7fa244103d50 2024-04-02T17:16:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.375+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2441a2280 con 0x7fa244103d50 2024-04-02T17:16:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 Processor -- start 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa23b400640 1 -- 172.21.15.59:0/1924193079 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa23400fb40 con 0x7fa244103d50 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 -- start start 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 0x56355d9832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d9883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 0x56355d98aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56355d8e8d70 con 0x56355d982ed0 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56355d8912a0 con 0x56355d988a30 2024-04-02T17:16:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.380+0000 7fa24c384740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56355d8e4d20 con 0x56355d9838c0 2024-04-02T17:16:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa243400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 0x56355d9832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa243400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 0x56355d9832f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35622/0 (socket says 172.21.15.59:35622) 2024-04-02T17:16:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa243400640 1 -- 172.21.15.59:0/3303835614 learned_addr learned my addr 172.21.15.59:0/3303835614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 0x56355d98aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242a00640 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d9883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 -- 172.21.15.59:0/3303835614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 msgr2=0x56355d9883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d9883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 -- 172.21.15.59:0/3303835614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 msgr2=0x56355d9832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 0x56355d9832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.381+0000 7fa242000640 1 -- 172.21.15.59:0/3303835614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355d8a5840 con 0x56355d988a30 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa242000640 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 0x56355d98aec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009230 tx=0x7fa22c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8de7e828b4f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa233400640 1 -- 172.21.15.59:0/3303835614 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c02ccc0 con 0x56355d988a30 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa233400640 1 -- 172.21.15.59:0/3303835614 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c005040 con 0x56355d988a30 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa233400640 1 -- 172.21.15.59:0/3303835614 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c004a60 con 0x56355d988a30 2024-04-02T17:16:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa23b400640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 0x7fa2280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa23b400640 1 -- 172.21.15.59:0/1924193079 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa2340d0760 con 0x7fa244103d50 2024-04-02T17:16:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.382+0000 7fa243400640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 0x7fa2280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 -- 172.21.15.59:0/3303835614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 msgr2=0x56355d98aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 0x56355d98aec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009230 tx=0x7fa22c005d50 comp rx=0 tx=0).stop 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 -- 172.21.15.59:0/3303835614 shutdown_connections 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d988a30 0x56355d98aec0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d9883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 --2- 172.21.15.59:0/3303835614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d982ed0 0x56355d9832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 -- 172.21.15.59:0/3303835614 >> 172.21.15.59:0/3303835614 conn(0x56355d97e440 msgr2=0x56355d97e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 -- 172.21.15.59:0/3303835614 shutdown_connections 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa24c384740 1 -- 172.21.15.59:0/3303835614 wait complete. 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.383+0000 7fa243400640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 0x7fa2280a6210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56355d984120 tx=0x7fa23c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.384+0000 7fa24c384740 1 -- start start 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.384+0000 7fa24c384740 1 Processor -- start 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d994910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56355d98b980 con 0x56355d988a30 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56355d8a4660 con 0x56355d9838c0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56355d98bb00 con 0x56355d982ed0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36292/0 (socket says 172.21.15.59:36292) 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa24c384740 10 client.0 _ll_register_callbacks cb 0x56355d8a4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242000640 1 -- 172.21.15.59:0/1444354489 learned_addr learned my addr 172.21.15.59:0/1444354489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48312/0 (socket says 172.21.15.59:48312) 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa243400640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d994910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 -- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 msgr2=0x56355d98d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 -- 172.21.15.59:0/1444354489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 msgr2=0x56355d994910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d994910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242a00640 1 -- 172.21.15.59:0/1444354489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355d8b3e80 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa242000640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.385+0000 7fa243400640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d994910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa242a00640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x56355d984330 tx=0x7fa234098d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db7af6b15dd17b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa227e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400fe30 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa2340a30e0 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22c005040 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa23409d110 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa24c384740 10 client.26221 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56355d5fd210 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.386+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c005040 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fa2340d2be0 con 0x56355d982ed0 2024-04-02T17:16:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 10 client.26221 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 20 client.26221 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d999220 0x56355d99b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 4 client.26221 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355d898cd0 con 0x56355d999220 2024-04-02T17:16:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.387+0000 7fa242000640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d999220 0x56355d99b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.388+0000 7fa242000640 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d999220 0x56355d99b6e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a478334f0d0cb66 server_cookie=eec127b2db29d9c8 in_seq=0 out_seq=0 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.388+0000 7fa227e00640 10 client.26221 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.388+0000 7fa227e00640 1 -- 172.21.15.59:0/1444354489 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56355d898cd0 con 0x56355d999220 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.388+0000 7fa227e00640 10 client.26221 handle_command_reply: tid=0 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 client.26221 shutdown 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 msgr2=0x56355d98d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x56355d984330 tx=0x7fa234098d70 comp rx=0 tx=0).stop 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 shutdown_connections 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d999220 0x56355d99b6e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d994910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d9838c0 0x56355d98d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 --2- 172.21.15.59:0/1444354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d982ed0 0x56355d98d390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 >> 172.21.15.59:0/1444354489 conn(0x56355d97e440 msgr2=0x56355d97d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa227e00640 0 client.26221 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 shutdown_connections 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 1 -- 172.21.15.59:0/1444354489 wait complete. 2024-04-02T17:16:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.389+0000 7fa24c384740 20 client.26221 trim_cache size 0 max 16384 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.408+0000 7fa24c384740 1 Processor -- start 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.408+0000 7fa24c384740 1 -- start start 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 0x56355d98d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d988a30 0x56355d97cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ab1a0 con 0x56355d9838c0 2024-04-02T17:16:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441acda0 con 0x56355d982ed0 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa24c384740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ab870 con 0x56355d988a30 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa243400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d988a30 0x56355d97cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36296/0 (socket says 172.21.15.59:36296) 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 -- 172.21.15.59:0/529700396 learned_addr learned my addr 172.21.15.59:0/529700396 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242000640 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 0x56355d98d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 -- 172.21.15.59:0/529700396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d988a30 msgr2=0x56355d97cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d988a30 0x56355d97cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 -- 172.21.15.59:0/529700396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 msgr2=0x56355d98d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 0x56355d98d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242a00640 1 -- 172.21.15.59:0/529700396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355d6c9620 con 0x56355d982ed0 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.409+0000 7fa242000640 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 0x56355d98d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.410+0000 7fa242a00640 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa23409c9a0 tx=0x7fa234000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9336feedb2979d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.410+0000 7fa232a00640 1 -- 172.21.15.59:0/529700396 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23409c550 con 0x56355d982ed0 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.410+0000 7fa232a00640 1 -- 172.21.15.59:0/529700396 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23409ca10 con 0x56355d982ed0 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.410+0000 7fa232a00640 1 -- 172.21.15.59:0/529700396 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2340bbe60 con 0x56355d982ed0 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 -- 172.21.15.59:0/529700396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 msgr2=0x56355d98cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa23409c9a0 tx=0x7fa234000bc0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 -- 172.21.15.59:0/529700396 shutdown_connections 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d9838c0 0x56355d98d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d988a30 0x56355d97cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 --2- 172.21.15.59:0/529700396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d98cfa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 -- 172.21.15.59:0/529700396 >> 172.21.15.59:0/529700396 conn(0x56355d97e440 msgr2=0x56355d984d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 -- 172.21.15.59:0/529700396 shutdown_connections 2024-04-02T17:16:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.411+0000 7fa24c384740 1 -- 172.21.15.59:0/529700396 wait complete. 2024-04-02T17:16:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 -- start start 2024-04-02T17:16:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 Processor -- start 2024-04-02T17:16:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d9fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ac610 con 0x56355d988a30 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441abf40 con 0x56355d982ed0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441a9180 con 0x56355d9838c0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa24c384740 10 client.0 _ll_register_callbacks cb 0x7fa2340d6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.412+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48326/0 (socket says 172.21.15.59:48326) 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:36312/0 (socket says 172.21.15.59:36312) 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242000640 1 -- 172.21.15.59:0/1258358396 learned_addr learned my addr 172.21.15.59:0/1258358396 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa243400640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d9fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 -- 172.21.15.59:0/1258358396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 msgr2=0x56355d991380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 -- 172.21.15.59:0/1258358396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 msgr2=0x56355d9fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d9fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 -- 172.21.15.59:0/1258358396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355d9fddc0 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa243400640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d9fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242000640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa242a00640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56355d983800 tx=0x7fa23409c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dba5caef54cf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa227e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23409c550 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa234087110 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.413+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23409ca10 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa234087350 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 10 client.26204 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56355d9fddc0 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23409ca10 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fa2340d0b90 con 0x56355d982ed0 2024-04-02T17:16:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 10 client.26204 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 10 client.26204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 20 client.26204 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d9ff6e0 0x56355da01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 4 client.26204 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.414+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56355d8f62b0 con 0x56355d9ff6e0 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.415+0000 7fa242000640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d9ff6e0 0x56355da01b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.415+0000 7fa242000640 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d9ff6e0 0x56355da01b00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a141d985500fae server_cookie=b1b599b9cc2b613e in_seq=0 out_seq=0 2024-04-02T17:16:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.415+0000 7fa227e00640 10 client.26204 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.439+0000 7fa227e00640 1 -- 172.21.15.59:0/1258358396 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56355d8f62b0 con 0x56355d9ff6e0 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.439+0000 7fa227e00640 10 client.26204 handle_command_reply: tid=0 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.439+0000 7fa24c384740 1 client.26204 shutdown 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.439+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 msgr2=0x56355d990e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.439+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56355d983800 tx=0x7fa23409c430 comp rx=0 tx=0).stop 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 shutdown_connections 2024-04-02T17:16:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56355d9ff6e0 0x56355da01b00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56355d988a30 0x56355d9fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56355d9838c0 0x56355d991380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 --2- 172.21.15.59:0/1258358396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56355d982ed0 0x56355d990e40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa227e00640 0 client.26204 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 >> 172.21.15.59:0/1258358396 conn(0x56355d97e440 msgr2=0x56355d98cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 shutdown_connections 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 1 -- 172.21.15.59:0/1258358396 wait complete. 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.440+0000 7fa24c384740 20 client.26204 trim_cache size 0 max 16384 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:16:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 msgr2=0x7fa2280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 0x7fa2280a6210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56355d984120 tx=0x7fa23c009500 comp rx=0 tx=0).stop 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 msgr2=0x7fa244199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c970 tx=0x7fa23400ce30 comp rx=0 tx=0).stop 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 shutdown_connections 2024-04-02T17:16:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa2280a3d50 0x7fa2280a6210 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244105900 0x7fa2441a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa244104f40 0x7fa24419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 --2- 172.21.15.59:0/1924193079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa244103d50 0x7fa244199f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 >> 172.21.15.59:0/1924193079 conn(0x7fa2440ff2c0 msgr2=0x7fa244108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 shutdown_connections 2024-04-02T17:16:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:22.441+0000 7fa249600640 1 -- 172.21.15.59:0/1924193079 wait complete. 2024-04-02T17:16:23.043 INFO:tasks.workunit.client.0.smithi059.stdout: 1 59965 3.09 MB/sec warmup 110 sec latency 21.029 ms 2024-04-02T17:16:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:23 smithi154 ceph-mon[36479]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 340 KiB/s wr, 92 op/s 2024-04-02T17:16:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:23 smithi059 ceph-mon[31664]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 340 KiB/s wr, 92 op/s 2024-04-02T17:16:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:23 smithi164 ceph-mon[36499]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 340 KiB/s wr, 92 op/s 2024-04-02T17:16:24.043 INFO:tasks.workunit.client.0.smithi059.stdout: 1 60689 3.12 MB/sec warmup 111 sec latency 22.446 ms 2024-04-02T17:16:25.043 INFO:tasks.workunit.client.0.smithi059.stdout: 1 60995 3.09 MB/sec warmup 112 sec latency 156.472 ms 2024-04-02T17:16:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:25 smithi154 ceph-mon[36479]: pgmap v488: 97 pgs: 97 active+clean; 30 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 523 KiB/s wr, 89 op/s 2024-04-02T17:16:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:25 smithi059 ceph-mon[31664]: pgmap v488: 97 pgs: 97 active+clean; 30 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 523 KiB/s wr, 89 op/s 2024-04-02T17:16:25.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:25 smithi164 ceph-mon[36499]: pgmap v488: 97 pgs: 97 active+clean; 30 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 523 KiB/s wr, 89 op/s 2024-04-02T17:16:26.043 INFO:tasks.workunit.client.0.smithi059.stdout: 1 60995 3.06 MB/sec warmup 113 sec latency 1156.584 ms 2024-04-02T17:16:27.043 INFO:tasks.workunit.client.0.smithi059.stdout: 1 60995 3.04 MB/sec warmup 114 sec latency 2156.737 ms 2024-04-02T17:16:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:27 smithi154 ceph-mon[36479]: pgmap v489: 97 pgs: 97 active+clean; 30 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 860 KiB/s wr, 86 op/s 2024-04-02T17:16:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:27 smithi059 ceph-mon[31664]: pgmap v489: 97 pgs: 97 active+clean; 30 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 860 KiB/s wr, 86 op/s 2024-04-02T17:16:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:27 smithi164 ceph-mon[36499]: pgmap v489: 97 pgs: 97 active+clean; 30 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 860 KiB/s wr, 86 op/s 2024-04-02T17:16:28.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 60995 3.01 MB/sec warmup 115 sec latency 3156.907 ms 2024-04-02T17:16:29.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 61051 2.98 MB/sec warmup 116 sec latency 3261.934 ms 2024-04-02T17:16:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:29 smithi154 ceph-mon[36479]: pgmap v490: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:16:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:29 smithi059 ceph-mon[31664]: pgmap v490: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:16:29.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:29 smithi164 ceph-mon[36499]: pgmap v490: 97 pgs: 97 active+clean; 30 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:16:30.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 61051 2.96 MB/sec warmup 117 sec latency 1805.772 ms 2024-04-02T17:16:31.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 61051 2.93 MB/sec warmup 118 sec latency 2805.920 ms 2024-04-02T17:16:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:31 smithi154 ceph-mon[36479]: pgmap v491: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1012 KiB/s wr, 90 op/s 2024-04-02T17:16:31.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:31 smithi059 ceph-mon[31664]: pgmap v491: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1012 KiB/s wr, 90 op/s 2024-04-02T17:16:31.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:31 smithi164 ceph-mon[36499]: pgmap v491: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1012 KiB/s wr, 90 op/s 2024-04-02T17:16:32.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 61051 2.91 MB/sec warmup 119 sec latency 3806.073 ms 2024-04-02T17:16:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:33 smithi154 ceph-mon[36479]: pgmap v492: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1001 KiB/s wr, 89 op/s 2024-04-02T17:16:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:33 smithi059 ceph-mon[31664]: pgmap v492: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1001 KiB/s wr, 89 op/s 2024-04-02T17:16:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:33 smithi164 ceph-mon[36499]: pgmap v492: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1001 KiB/s wr, 89 op/s 2024-04-02T17:16:34.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 61940 4.95 MB/sec execute 1 sec latency 4913.924 ms 2024-04-02T17:16:35.044 INFO:tasks.workunit.client.0.smithi059.stdout: 1 63523 6.54 MB/sec execute 2 sec latency 16.718 ms 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.592+0000 7f4600e00640 1 -- 172.21.15.59:0/3520951055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 msgr2=0x7f45fc1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.592+0000 7f4600e00640 1 --2- 172.21.15.59:0/3520951055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1103c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009be0 tx=0x7f45f402bef0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 -- 172.21.15.59:0/3520951055 shutdown_connections 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 --2- 172.21.15.59:0/3520951055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 --2- 172.21.15.59:0/3520951055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1103c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 --2- 172.21.15.59:0/3520951055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 -- 172.21.15.59:0/3520951055 >> 172.21.15.59:0/3520951055 conn(0x7f45fc076f40 msgr2=0x7f45fc079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 -- 172.21.15.59:0/3520951055 shutdown_connections 2024-04-02T17:16:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.593+0000 7f4600e00640 1 -- 172.21.15.59:0/3520951055 wait complete. 2024-04-02T17:16:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.594+0000 7f4600e00640 1 Processor -- start 2024-04-02T17:16:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.594+0000 7f4600e00640 1 -- start start 2024-04-02T17:16:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc113c60 con 0x7f45fc109170 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc113ae0 con 0x7f45fc109b30 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f4600e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc113de0 con 0x7f45fc107f80 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45280/0 (socket says 172.21.15.59:45280) 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fb400640 1 -- 172.21.15.59:0/647196987 learned_addr learned my addr 172.21.15.59:0/647196987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45f2a00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 -- 172.21.15.59:0/647196987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 msgr2=0x7f45fc19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 -- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 msgr2=0x7f45fc19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fbe00640 1 -- 172.21.15.59:0/647196987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1a5df0 con 0x7f45fc109b30 2024-04-02T17:16:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45f2a00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.595+0000 7f45fb400640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f45fbe00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1a5780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45f402c430 tx=0x7f45f40337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f45f8c00640 1 -- 172.21.15.59:0/647196987 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4043020 con 0x7f45fc109b30 2024-04-02T17:16:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f45f8c00640 1 -- 172.21.15.59:0/647196987 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f402ca60 con 0x7f45fc109b30 2024-04-02T17:16:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f45f8c00640 1 -- 172.21.15.59:0/647196987 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4034690 con 0x7f45fc109b30 2024-04-02T17:16:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1a6080 con 0x7f45fc109b30 2024-04-02T17:16:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.596+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1a6580 con 0x7f45fc109b30 2024-04-02T17:16:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.600+0000 7f45f8c00640 1 -- 172.21.15.59:0/647196987 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f45f4046020 con 0x7f45fc109b30 2024-04-02T17:16:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.600+0000 7f46051e7740 1 Processor -- start 2024-04-02T17:16:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.600+0000 7f46051e7740 1 -- start start 2024-04-02T17:16:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 0x55738b81c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b821c60 0x55738b8240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55738b781d70 con 0x55738b821c60 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55738b72a2a0 con 0x55738b81c100 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f46051e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55738b77dd20 con 0x55738b81caf0 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 0x55738b81c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45284/0 (socket says 172.21.15.59:45284) 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 0x55738b81c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60882/0 (socket says 172.21.15.59:60882) 2024-04-02T17:16:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45fb400640 1 -- 172.21.15.59:0/2390297490 learned_addr learned my addr 172.21.15.59:0/2390297490 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.601+0000 7f45f2a00640 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b821c60 0x55738b8240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 -- 172.21.15.59:0/2390297490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 msgr2=0x55738b81c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 0x55738b81c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 -- 172.21.15.59:0/2390297490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b821c60 msgr2=0x55738b8240f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b821c60 0x55738b8240f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 -- 172.21.15.59:0/2390297490 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738b73e840 con 0x55738b81caf0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45fb400640 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0115c0 tx=0x7f45ec011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b797736791da6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f8c00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 0x7f45e00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f0c00640 1 -- 172.21.15.59:0/2390297490 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec005020 con 0x55738b81caf0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f0c00640 1 -- 172.21.15.59:0/2390297490 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec0047a0 con 0x55738b81caf0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f0c00640 1 -- 172.21.15.59:0/2390297490 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00fa00 con 0x55738b81caf0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f8c00640 1 -- 172.21.15.59:0/647196987 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f45f40f2a50 con 0x7f45fc109b30 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.602+0000 7f45f2a00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 0x7f45e00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 -- 172.21.15.59:0/2390297490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 msgr2=0x55738b8215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0115c0 tx=0x7f45ec011a80 comp rx=0 tx=0).stop 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 -- 172.21.15.59:0/2390297490 shutdown_connections 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b821c60 0x55738b8240f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81caf0 0x55738b8215f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 --2- 172.21.15.59:0/2390297490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81c100 0x55738b81c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.603+0000 7f46051e7740 1 -- 172.21.15.59:0/2390297490 >> 172.21.15.59:0/2390297490 conn(0x55738b817670 msgr2=0x55738b817a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.604+0000 7f45f2a00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 0x7f45e00a60e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009fd0 tx=0x7f45e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.604+0000 7f46051e7740 1 -- 172.21.15.59:0/2390297490 shutdown_connections 2024-04-02T17:16:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.604+0000 7f46051e7740 1 -- 172.21.15.59:0/2390297490 wait complete. 2024-04-02T17:16:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 -- start start 2024-04-02T17:16:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 Processor -- start 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b826670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b826bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55738b824c10 con 0x55738b81caf0 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55738b73d660 con 0x55738b821c60 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55738b824d90 con 0x55738b81c100 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.605+0000 7f46051e7740 10 client.0 _ll_register_callbacks cb 0x55738b73d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b826670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b826bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60888/0 (socket says 172.21.15.59:60888) 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 -- 172.21.15.59:0/1587740710 learned_addr learned my addr 172.21.15.59:0/1587740710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 -- 172.21.15.59:0/1587740710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 msgr2=0x55738b826670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b826670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 -- 172.21.15.59:0/1587740710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 msgr2=0x55738b826bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b826bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 -- 172.21.15.59:0/1587740710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738b74ce80 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45fbe00640 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45f40bb0d0 tx=0x7f45f40bea40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2358304e5b6ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45e9600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40bf370 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f45f40df070 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.606+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ec0047a0 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45f40bec70 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 10 client.26216 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55738b496210 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec0047a0 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f45f4004b40 con 0x55738b821c60 2024-04-02T17:16:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 10 client.26216 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 20 client.26216 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b832520 0x55738b8349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 4 client.26216 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.607+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738b731cd0 con 0x55738b832520 2024-04-02T17:16:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.608+0000 7f45fb400640 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b832520 0x55738b8349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.608+0000 7f45fb400640 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b832520 0x55738b8349e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8372854f997f1b8 server_cookie=d3542ef22a5ed136 in_seq=0 out_seq=0 2024-04-02T17:16:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.608+0000 7f45e9600640 10 client.26216 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.609+0000 7f45e9600640 1 -- 172.21.15.59:0/1587740710 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55738b731cd0 con 0x55738b832520 2024-04-02T17:16:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.609+0000 7f45e9600640 10 client.26216 handle_command_reply: tid=0 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.609+0000 7f46051e7740 1 client.26216 shutdown 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.609+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 msgr2=0x55738b82dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.609+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45f40bb0d0 tx=0x7f45f40bea40 comp rx=0 tx=0).stop 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 shutdown_connections 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b832520 0x55738b8349e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b826bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82dbf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 --2- 172.21.15.59:0/1587740710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b826670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f45e9600640 0 client.26216 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 >> 172.21.15.59:0/1587740710 conn(0x55738b817670 msgr2=0x55738b816870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 shutdown_connections 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 1 -- 172.21.15.59:0/1587740710 wait complete. 2024-04-02T17:16:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.610+0000 7f46051e7740 20 client.26216 trim_cache size 0 max 16384 2024-04-02T17:16:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 Processor -- start 2024-04-02T17:16:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 -- start start 2024-04-02T17:16:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 0x55738b815fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b821c60 0x55738b816520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1af5b0 con 0x55738b81c100 2024-04-02T17:16:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1b11b0 con 0x55738b81caf0 2024-04-02T17:16:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.627+0000 7f46051e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1afc80 con 0x55738b821c60 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 0x55738b815fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60892/0 (socket says 172.21.15.59:60892) 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 -- 172.21.15.59:0/2594351125 learned_addr learned my addr 172.21.15.59:0/2594351125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45fbe00640 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b821c60 0x55738b816520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 -- 172.21.15.59:0/2594351125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b821c60 msgr2=0x55738b816520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b821c60 0x55738b816520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 -- 172.21.15.59:0/2594351125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 msgr2=0x55738b815fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 0x55738b815fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 -- 172.21.15.59:0/2594351125 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738b562620 con 0x55738b81caf0 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45fb400640 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 0x55738b815fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.628+0000 7f45f2a00640 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45fc19f6e0 tx=0x7f45e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677f712f21c376fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.629+0000 7f45ebe00640 1 -- 172.21.15.59:0/2594351125 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4005020 con 0x55738b81caf0 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.629+0000 7f45ebe00640 1 -- 172.21.15.59:0/2594351125 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40047a0 con 0x55738b81caf0 2024-04-02T17:16:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.629+0000 7f45ebe00640 1 -- 172.21.15.59:0/2594351125 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40209b0 con 0x55738b81caf0 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- 172.21.15.59:0/2594351125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 msgr2=0x55738b82dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45fc19f6e0 tx=0x7f45e40046c0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- 172.21.15.59:0/2594351125 shutdown_connections 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b821c60 0x55738b816520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b81caf0 0x55738b82dbb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 --2- 172.21.15.59:0/2594351125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81c100 0x55738b815fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- 172.21.15.59:0/2594351125 >> 172.21.15.59:0/2594351125 conn(0x55738b817670 msgr2=0x55738b81e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- 172.21.15.59:0/2594351125 shutdown_connections 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- 172.21.15.59:0/2594351125 wait complete. 2024-04-02T17:16:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.630+0000 7f46051e7740 1 -- start start 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 Processor -- start 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b82b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b82b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1b0a20 con 0x55738b81caf0 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1b0350 con 0x55738b821c60 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1ad590 con 0x55738b81c100 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.631+0000 7f46051e7740 10 client.0 _ll_register_callbacks cb 0x7f45f40ba7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b82b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60906/0 (socket says 172.21.15.59:60906) 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 -- 172.21.15.59:0/3538686582 learned_addr learned my addr 172.21.15.59:0/3538686582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fb400640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b82b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 -- 172.21.15.59:0/3538686582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 msgr2=0x55738b82b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b82b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 -- 172.21.15.59:0/3538686582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 msgr2=0x55738b82b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b82b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 -- 172.21.15.59:0/3538686582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738b82caf0 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45fbe00640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f45f40bee30 tx=0x7f45f4034f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39be803cacb1072a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45e9600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.632+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40be3e0 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f45f409dae0 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40047a0 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45f40bf370 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 10 client.26222 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55738b82de00 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40047a0 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f45f40c7b30 con 0x55738b821c60 2024-04-02T17:16:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 10 client.26222 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 20 client.26222 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b898fb0 0x55738b89b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 4 client.26222 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.633+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55738b78f2b0 con 0x55738b898fb0 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.634+0000 7f45fb400640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b898fb0 0x55738b89b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.634+0000 7f45fb400640 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b898fb0 0x55738b89b3d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8006f5ac91f3fc5 server_cookie=44292651c2821492 in_seq=0 out_seq=0 2024-04-02T17:16:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.634+0000 7f45e9600640 10 client.26222 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:35 smithi154 ceph-mon[36479]: pgmap v493: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1010 KiB/s wr, 91 op/s 2024-04-02T17:16:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f45e9600640 1 -- 172.21.15.59:0/3538686582 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55738b78f2b0 con 0x55738b898fb0 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f45e9600640 10 client.26222 handle_command_reply: tid=0 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 client.26222 shutdown 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 msgr2=0x55738b82c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f45f40bee30 tx=0x7f45f4034f90 comp rx=0 tx=0).stop 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 shutdown_connections 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55738b898fb0 0x55738b89b3d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55738b81caf0 0x55738b82b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55738b821c60 0x55738b82c500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 --2- 172.21.15.59:0/3538686582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55738b81c100 0x55738b82b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f45e9600640 0 client.26222 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 >> 172.21.15.59:0/3538686582 conn(0x55738b817670 msgr2=0x55738b81a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 shutdown_connections 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.660+0000 7f46051e7740 1 -- 172.21.15.59:0/3538686582 wait complete. 2024-04-02T17:16:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f46051e7740 20 client.26222 trim_cache size 0 max 16384 2024-04-02T17:16:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:16:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:16:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:16:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 msgr2=0x7f45e00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 0x7f45e00a60e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009fd0 tx=0x7f45e4009260 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 msgr2=0x7f45fc1a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1a5780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f45f402c430 tx=0x7f45f40337b0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 shutdown_connections 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f45e00a3c20 0x7f45e00a60e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45fc109170 0x7f45fc19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45fc109b30 0x7f45fc1a5780 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 --2- 172.21.15.59:0/647196987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45fc107f80 0x7f45fc19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.661+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 >> 172.21.15.59:0/647196987 conn(0x7f45fc076f40 msgr2=0x7f45fc10c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.662+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 shutdown_connections 2024-04-02T17:16:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:35.662+0000 7f4600e00640 1 -- 172.21.15.59:0/647196987 wait complete. 2024-04-02T17:16:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:35 smithi059 ceph-mon[31664]: pgmap v493: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1010 KiB/s wr, 91 op/s 2024-04-02T17:16:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:35 smithi164 ceph-mon[36499]: pgmap v493: 97 pgs: 97 active+clean; 30 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1010 KiB/s wr, 91 op/s 2024-04-02T17:16:36.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64262 6.29 MB/sec execute 3 sec latency 28.451 ms 2024-04-02T17:16:37.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64536 4.74 MB/sec execute 4 sec latency 319.821 ms 2024-04-02T17:16:37.624 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:37 smithi164 ceph-mon[36499]: pgmap v494: 97 pgs: 97 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 713 KiB/s wr, 76 op/s 2024-04-02T17:16:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:37 smithi154 ceph-mon[36479]: pgmap v494: 97 pgs: 97 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 713 KiB/s wr, 76 op/s 2024-04-02T17:16:37.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:37 smithi059 ceph-mon[31664]: pgmap v494: 97 pgs: 97 active+clean; 31 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 713 KiB/s wr, 76 op/s 2024-04-02T17:16:38.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64536 3.80 MB/sec execute 5 sec latency 1319.996 ms 2024-04-02T17:16:39.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64592 3.19 MB/sec execute 6 sec latency 1425.007 ms 2024-04-02T17:16:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:39 smithi154 ceph-mon[36479]: pgmap v495: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 374 KiB/s wr, 70 op/s 2024-04-02T17:16:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:39 smithi059 ceph-mon[31664]: pgmap v495: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 374 KiB/s wr, 70 op/s 2024-04-02T17:16:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:39 smithi164 ceph-mon[36499]: pgmap v495: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 374 KiB/s wr, 70 op/s 2024-04-02T17:16:40.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64592 2.73 MB/sec execute 7 sec latency 1807.752 ms 2024-04-02T17:16:41.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64592 2.39 MB/sec execute 8 sec latency 2807.917 ms 2024-04-02T17:16:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:41 smithi154 ceph-mon[36479]: pgmap v496: 97 pgs: 97 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 173 KiB/s wr, 55 op/s 2024-04-02T17:16:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:41 smithi059 ceph-mon[31664]: pgmap v496: 97 pgs: 97 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 173 KiB/s wr, 55 op/s 2024-04-02T17:16:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:41 smithi164 ceph-mon[36499]: pgmap v496: 97 pgs: 97 active+clean; 32 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 173 KiB/s wr, 55 op/s 2024-04-02T17:16:42.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64660 2.14 MB/sec execute 9 sec latency 3737.175 ms 2024-04-02T17:16:43.045 INFO:tasks.workunit.client.0.smithi059.stdout: 1 64660 1.93 MB/sec execute 10 sec latency 1018.749 ms 2024-04-02T17:16:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:43 smithi154 ceph-mon[36479]: pgmap v497: 97 pgs: 97 active+clean; 32 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 202 KiB/s wr, 60 op/s 2024-04-02T17:16:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:43 smithi059 ceph-mon[31664]: pgmap v497: 97 pgs: 97 active+clean; 32 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 202 KiB/s wr, 60 op/s 2024-04-02T17:16:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:43 smithi164 ceph-mon[36499]: pgmap v497: 97 pgs: 97 active+clean; 32 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 202 KiB/s wr, 60 op/s 2024-04-02T17:16:44.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 65849 2.21 MB/sec execute 11 sec latency 1126.468 ms 2024-04-02T17:16:45.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 67221 2.77 MB/sec execute 12 sec latency 19.334 ms 2024-04-02T17:16:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:45 smithi154 ceph-mon[36479]: pgmap v498: 97 pgs: 97 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 510 KiB/s wr, 70 op/s 2024-04-02T17:16:45.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:45 smithi059 ceph-mon[31664]: pgmap v498: 97 pgs: 97 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 510 KiB/s wr, 70 op/s 2024-04-02T17:16:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:45 smithi164 ceph-mon[36499]: pgmap v498: 97 pgs: 97 active+clean; 33 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 510 KiB/s wr, 70 op/s 2024-04-02T17:16:46.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 67808 2.92 MB/sec execute 13 sec latency 26.090 ms 2024-04-02T17:16:47.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68143 2.73 MB/sec execute 14 sec latency 341.033 ms 2024-04-02T17:16:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:47 smithi154 ceph-mon[36479]: pgmap v499: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-04-02T17:16:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:47 smithi059 ceph-mon[31664]: pgmap v499: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-04-02T17:16:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:47 smithi164 ceph-mon[36499]: pgmap v499: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 784 KiB/s wr, 86 op/s 2024-04-02T17:16:48.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68143 2.55 MB/sec execute 15 sec latency 1341.192 ms 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 -- 172.21.15.59:0/192050567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58109170 msgr2=0x7fee581095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 --2- 172.21.15.59:0/192050567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58109170 0x7fee581095f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee44009900 tx=0x7fee4402bc30 comp rx=0 tx=0).stop 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 -- 172.21.15.59:0/192050567 shutdown_connections 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 --2- 172.21.15.59:0/192050567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 --2- 172.21.15.59:0/192050567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58109170 0x7fee581095f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 --2- 172.21.15.59:0/192050567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58107f80 0x7fee581083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 -- 172.21.15.59:0/192050567 >> 172.21.15.59:0/192050567 conn(0x7fee58076f40 msgr2=0x7fee58079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 -- 172.21.15.59:0/192050567 shutdown_connections 2024-04-02T17:16:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.811+0000 7fee57e00640 1 -- 172.21.15.59:0/192050567 wait complete. 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.812+0000 7fee57e00640 1 Processor -- start 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.812+0000 7fee57e00640 1 -- start start 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 0x7fee5819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee58113c60 con 0x7fee58109b30 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee58113ae0 con 0x7fee58109170 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee58113de0 con 0x7fee58107f80 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee56000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee57400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee56000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:47766/0 (socket says 172.21.15.59:47766) 2024-04-02T17:16:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee56000640 1 -- 172.21.15.59:0/2058355008 learned_addr learned my addr 172.21.15.59:0/2058355008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.813+0000 7fee56a00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 0x7fee5819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 -- 172.21.15.59:0/2058355008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 msgr2=0x7fee5819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 0x7fee5819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 -- 172.21.15.59:0/2058355008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 msgr2=0x7fee5819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 -- 172.21.15.59:0/2058355008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee581a5e60 con 0x7fee58109b30 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee56000640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee56a00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 0x7fee5819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57400640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581a57f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fee5000da10 tx=0x7fee5000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee4f400640 1 -- 172.21.15.59:0/2058355008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000bd70 con 0x7fee58109b30 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee581a6150 con 0x7fee58109b30 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee4f400640 1 -- 172.21.15.59:0/2058355008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee50003e50 con 0x7fee58109b30 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.814+0000 7fee4f400640 1 -- 172.21.15.59:0/2058355008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500106a0 con 0x7fee58109b30 2024-04-02T17:16:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.815+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581a6620 con 0x7fee58109b30 2024-04-02T17:16:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 Processor -- start 2024-04-02T17:16:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee4f400640 1 -- 172.21.15.59:0/2058355008 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fee50003970 con 0x7fee58109b30 2024-04-02T17:16:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 -- start start 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3e1c60 0x559d5c3e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d5c341d70 con 0x559d5c3dc100 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559d5c2ea2a0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.818+0000 7fee60609740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559d5c33dd20 con 0x559d5c3e1c60 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60802/0 (socket says 172.21.15.59:60802) 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56000640 1 -- 172.21.15.59:0/3015991863 learned_addr learned my addr 172.21.15.59:0/3015991863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56000640 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3e1c60 0x559d5c3e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 -- 172.21.15.59:0/3015991863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3e1c60 msgr2=0x559d5c3e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3e1c60 0x559d5c3e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 -- 172.21.15.59:0/3015991863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 msgr2=0x559d5c3dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 -- 172.21.15.59:0/3015991863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5c2fe840 con 0x559d5c3dcaf0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56000640 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.819+0000 7fee56a00640 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e15f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fa0 tx=0x7fee480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a259c7c8e160d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee4f400640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 0x7fee3c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee43e00640 1 -- 172.21.15.59:0/3015991863 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004310 con 0x559d5c3dcaf0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee43e00640 1 -- 172.21.15.59:0/3015991863 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee480044b0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee43e00640 1 -- 172.21.15.59:0/3015991863 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480035f0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee4f400640 1 -- 172.21.15.59:0/2058355008 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fee500d0880 con 0x7fee58109b30 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee56000640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 0x7fee3c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee60609740 1 -- 172.21.15.59:0/3015991863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 msgr2=0x559d5c3e15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.820+0000 7fee60609740 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e15f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee48009fa0 tx=0x7fee480118d0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 -- 172.21.15.59:0/3015991863 shutdown_connections 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3e1c60 0x559d5c3e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e15f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 --2- 172.21.15.59:0/3015991863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dc100 0x559d5c3dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 -- 172.21.15.59:0/3015991863 >> 172.21.15.59:0/3015991863 conn(0x559d5c3d7670 msgr2=0x559d5c3d7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee56000640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 0x7fee3c0a6060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee4402ceb0 tx=0x7fee44005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 -- 172.21.15.59:0/3015991863 shutdown_connections 2024-04-02T17:16:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 -- 172.21.15.59:0/3015991863 wait complete. 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 -- start start 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.821+0000 7fee60609740 1 Processor -- start 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c3edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d5c3e4c10 con 0x559d5c3dcaf0 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559d5c2fd660 con 0x559d5c3e1c60 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559d5c3e4d90 con 0x559d5c3dc100 2024-04-02T17:16:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee60609740 10 client.0 _ll_register_callbacks cb 0x559d5c2fd050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60808/0 (socket says 172.21.15.59:60808) 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee56000640 1 -- 172.21.15.59:0/1768797006 learned_addr learned my addr 172.21.15.59:0/1768797006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee57400640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c3edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.822+0000 7fee56a00640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 -- 172.21.15.59:0/1768797006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 msgr2=0x559d5c3e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 -- 172.21.15.59:0/1768797006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 msgr2=0x559d5c3e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 -- 172.21.15.59:0/1768797006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5c30ce80 con 0x559d5c3e1c60 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee56000640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee57400640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c3edb70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b630 tx=0x7fee5009d1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a78b4095b04fe4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee40c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:16:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500a5710 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fee50004000 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee480044b0 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee500b6020 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee60609740 10 client.26234 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.823+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d5c056210 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee480044b0 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fee50024060 con 0x559d5c3e1c60 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 10 client.26234 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 20 client.26234 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c3f24d0 0x559d5c3f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 4 client.26234 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.824+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d5c2f1cd0 con 0x559d5c3f24d0 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.825+0000 7fee56a00640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c3f24d0 0x559d5c3f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.825+0000 7fee56a00640 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c3f24d0 0x559d5c3f48e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c1cbaa013e8bb6 server_cookie=15d2c265a86679e0 in_seq=0 out_seq=0 2024-04-02T17:16:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.825+0000 7fee40c00640 10 client.26234 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee40c00640 1 -- 172.21.15.59:0/1768797006 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d5c2f1cd0 con 0x559d5c3f24d0 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee40c00640 10 client.26234 handle_command_reply: tid=0 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 client.26234 shutdown 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 msgr2=0x559d5c3edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c3edb70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b630 tx=0x7fee5009d1a0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 shutdown_connections 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c3f24d0 0x559d5c3f48e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c3edb70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 --2- 172.21.15.59:0/1768797006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 >> 172.21.15.59:0/1768797006 conn(0x559d5c3d7670 msgr2=0x559d5c3d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.826+0000 7fee40c00640 0 client.26234 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.827+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 shutdown_connections 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.827+0000 7fee60609740 1 -- 172.21.15.59:0/1768797006 wait complete. 2024-04-02T17:16:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.827+0000 7fee60609740 20 client.26234 trim_cache size 0 max 16384 2024-04-02T17:16:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.843+0000 7fee60609740 1 Processor -- start 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.843+0000 7fee60609740 1 -- start start 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dcaf0 0x559d5c3edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee581af5d0 con 0x559d5c3e1c60 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b11d0 con 0x559d5c3dc100 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee60609740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee581afca0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee57400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee57400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60820/0 (socket says 172.21.15.59:60820) 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee57400640 1 -- 172.21.15.59:0/2373032745 learned_addr learned my addr 172.21.15.59:0/2373032745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee56000640 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dcaf0 0x559d5c3edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.844+0000 7fee56a00640 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 -- 172.21.15.59:0/2373032745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dcaf0 msgr2=0x559d5c3edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dcaf0 0x559d5c3edb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 -- 172.21.15.59:0/2373032745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dc100 msgr2=0x559d5c3e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 -- 172.21.15.59:0/2373032745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5c122620 con 0x559d5c3e1c60 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee57400640 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee5009cc60 tx=0x7fee5009c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c0d823165a2739 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee43400640 1 -- 172.21.15.59:0/2373032745 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50014070 con 0x559d5c3e1c60 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee43400640 1 -- 172.21.15.59:0/2373032745 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee500a5710 con 0x559d5c3e1c60 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.845+0000 7fee43400640 1 -- 172.21.15.59:0/2373032745 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50099110 con 0x559d5c3e1c60 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- 172.21.15.59:0/2373032745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 msgr2=0x559d5c3e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee5009cc60 tx=0x7fee5009c870 comp rx=0 tx=0).stop 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- 172.21.15.59:0/2373032745 shutdown_connections 2024-04-02T17:16:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3e1c60 0x559d5c3e6b30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dcaf0 0x559d5c3edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 --2- 172.21.15.59:0/2373032745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3dc100 0x559d5c3e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- 172.21.15.59:0/2373032745 >> 172.21.15.59:0/2373032745 conn(0x559d5c3d7670 msgr2=0x559d5c3d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- 172.21.15.59:0/2373032745 shutdown_connections 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- 172.21.15.59:0/2373032745 wait complete. 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.846+0000 7fee60609740 1 -- start start 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 Processor -- start 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c456a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b0a40 con 0x559d5c3dcaf0 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b0370 con 0x559d5c3e1c60 2024-04-02T17:16:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ad5b0 con 0x559d5c3dc100 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee60609740 10 client.0 _ll_register_callbacks cb 0x7fee5009c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee56000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60830/0 (socket says 172.21.15.59:60830) 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee56000640 1 -- 172.21.15.59:0/1665167351 learned_addr learned my addr 172.21.15.59:0/1665167351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.847+0000 7fee56a00640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee57400640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c456a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 -- 172.21.15.59:0/1665167351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 msgr2=0x559d5c3ea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3ea800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 -- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 msgr2=0x559d5c456a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c456a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 -- 172.21.15.59:0/1665167351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5c3edda0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee56000640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee4402c8e0 tx=0x7fee4402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1e40743bbcbb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee40c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.848+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440055a0 con 0x559d5c3dcaf0 2024-04-02T17:16:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fee44043480 con 0x559d5c3dcaf0 2024-04-02T17:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee500a5710 con 0x559d5c3dcaf0 2024-04-02T17:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee44043680 con 0x559d5c3dcaf0 2024-04-02T17:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 10 client.16149 fetch_fsmap learned FSMap version 31 2024-04-02T17:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d5c458070 con 0x559d5c3dcaf0 2024-04-02T17:16:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee500a5710 con 0x559d5c3dcaf0 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fee44034930 con 0x559d5c3dcaf0 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 10 client.16149 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 20 client.16149 populate_metadata read hostname 'smithi059' 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c458b60 0x559d5c45af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 4 client.16149 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.849+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d5c34f2b0 con 0x559d5c458b60 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.850+0000 7fee56a00640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c458b60 0x559d5c45af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.850+0000 7fee56a00640 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c458b60 0x559d5c45af80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8a61fe3c264e6a server_cookie=e1aab30594f749b4 in_seq=0 out_seq=0 2024-04-02T17:16:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.850+0000 7fee40c00640 10 client.16149 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.876+0000 7fee40c00640 1 -- 172.21.15.59:0/1665167351 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d5c34f2b0 con 0x559d5c458b60 2024-04-02T17:16:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.876+0000 7fee40c00640 10 client.16149 handle_command_reply: tid=0 2024-04-02T17:16:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.876+0000 7fee60609740 1 client.16149 shutdown 2024-04-02T17:16:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.876+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 msgr2=0x559d5c3ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee4402c8e0 tx=0x7fee4402bc30 comp rx=0 tx=0).stop 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 shutdown_connections 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559d5c458b60 0x559d5c45af80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d5c3dcaf0 0x559d5c3ead40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559d5c3e1c60 0x559d5c456a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 --2- 172.21.15.59:0/1665167351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559d5c3dc100 0x559d5c3ea800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee40c00640 0 client.16149 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 >> 172.21.15.59:0/1665167351 conn(0x559d5c3d7670 msgr2=0x559d5c3e65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 shutdown_connections 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 1 -- 172.21.15.59:0/1665167351 wait complete. 2024-04-02T17:16:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.877+0000 7fee60609740 20 client.16149 trim_cache size 0 max 16384 2024-04-02T17:16:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:16:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:16:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:16:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 msgr2=0x7fee3c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 0x7fee3c0a6060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee4402ceb0 tx=0x7fee44005b70 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 msgr2=0x7fee581a57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581a57f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fee5000da10 tx=0x7fee5000ded0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 shutdown_connections 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fee3c0a3ba0 0x7fee3c0a6060 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58109b30 0x7fee581a57f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee58109170 0x7fee5819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 --2- 172.21.15.59:0/2058355008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee58107f80 0x7fee5819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 >> 172.21.15.59:0/2058355008 conn(0x7fee58076f40 msgr2=0x7fee5810c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:16:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 shutdown_connections 2024-04-02T17:16:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:16:48.878+0000 7fee57e00640 1 -- 172.21.15.59:0/2058355008 wait complete. 2024-04-02T17:16:49.046 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68211 2.40 MB/sec execute 16 sec latency 1442.697 ms 2024-04-02T17:16:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:49 smithi154 ceph-mon[36479]: pgmap v500: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 866 KiB/s wr, 102 op/s 2024-04-02T17:16:49.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:49 smithi059 ceph-mon[31664]: pgmap v500: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 866 KiB/s wr, 102 op/s 2024-04-02T17:16:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:49 smithi164 ceph-mon[36499]: pgmap v500: 97 pgs: 97 active+clean; 34 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 866 KiB/s wr, 102 op/s 2024-04-02T17:16:50.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68211 2.26 MB/sec execute 17 sec latency 1791.770 ms 2024-04-02T17:16:50.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:50 smithi059 ceph-mon[31664]: pgmap v501: 97 pgs: 97 active+clean; 33 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 799 KiB/s wr, 85 op/s 2024-04-02T17:16:50.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:50 smithi164 ceph-mon[36499]: pgmap v501: 97 pgs: 97 active+clean; 33 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 799 KiB/s wr, 85 op/s 2024-04-02T17:16:50.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:50 smithi154 ceph-mon[36479]: pgmap v501: 97 pgs: 97 active+clean; 33 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 799 KiB/s wr, 85 op/s 2024-04-02T17:16:51.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68211 2.13 MB/sec execute 18 sec latency 2791.990 ms 2024-04-02T17:16:52.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68211 2.02 MB/sec execute 19 sec latency 3792.150 ms 2024-04-02T17:16:53.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 68211 1.92 MB/sec execute 20 sec latency 4792.290 ms 2024-04-02T17:16:53.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:52 smithi059 ceph-mon[31664]: pgmap v502: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 815 KiB/s wr, 89 op/s 2024-04-02T17:16:53.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:52 smithi164 ceph-mon[36499]: pgmap v502: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 815 KiB/s wr, 89 op/s 2024-04-02T17:16:53.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:52 smithi154 ceph-mon[36479]: pgmap v502: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 815 KiB/s wr, 89 op/s 2024-04-02T17:16:54.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 69511 2.08 MB/sec execute 21 sec latency 4900.467 ms 2024-04-02T17:16:55.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 70806 2.39 MB/sec execute 22 sec latency 15.781 ms 2024-04-02T17:16:55.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:54 smithi059 ceph-mon[31664]: pgmap v503: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 790 KiB/s wr, 89 op/s 2024-04-02T17:16:55.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:54 smithi164 ceph-mon[36499]: pgmap v503: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 790 KiB/s wr, 89 op/s 2024-04-02T17:16:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:54 smithi154 ceph-mon[36479]: pgmap v503: 97 pgs: 97 active+clean; 33 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 790 KiB/s wr, 89 op/s 2024-04-02T17:16:56.047 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71360 2.48 MB/sec execute 23 sec latency 23.015 ms 2024-04-02T17:16:57.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71629 2.38 MB/sec execute 24 sec latency 451.563 ms 2024-04-02T17:16:57.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:56 smithi059 ceph-mon[31664]: pgmap v504: 97 pgs: 97 active+clean; 33 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 516 KiB/s wr, 86 op/s 2024-04-02T17:16:57.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:56 smithi164 ceph-mon[36499]: pgmap v504: 97 pgs: 97 active+clean; 33 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 516 KiB/s wr, 86 op/s 2024-04-02T17:16:57.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:56 smithi154 ceph-mon[36479]: pgmap v504: 97 pgs: 97 active+clean; 33 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 516 KiB/s wr, 86 op/s 2024-04-02T17:16:58.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71629 2.29 MB/sec execute 25 sec latency 1451.689 ms 2024-04-02T17:16:59.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71685 2.21 MB/sec execute 26 sec latency 1556.373 ms 2024-04-02T17:16:59.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:16:58 smithi059 ceph-mon[31664]: pgmap v505: 97 pgs: 97 active+clean; 34 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 314 KiB/s wr, 86 op/s 2024-04-02T17:16:59.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:16:58 smithi164 ceph-mon[36499]: pgmap v505: 97 pgs: 97 active+clean; 34 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 314 KiB/s wr, 86 op/s 2024-04-02T17:16:59.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:16:58 smithi154 ceph-mon[36479]: pgmap v505: 97 pgs: 97 active+clean; 34 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 314 KiB/s wr, 86 op/s 2024-04-02T17:17:00.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71685 2.12 MB/sec execute 27 sec latency 1798.618 ms 2024-04-02T17:17:01.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71685 2.05 MB/sec execute 28 sec latency 2798.722 ms 2024-04-02T17:17:01.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:00 smithi059 ceph-mon[31664]: pgmap v506: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 64 op/s 2024-04-02T17:17:01.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:00 smithi164 ceph-mon[36499]: pgmap v506: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 64 op/s 2024-04-02T17:17:01.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:00 smithi154 ceph-mon[36479]: pgmap v506: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 64 op/s 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 -- 172.21.15.59:0/2181659440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99601036e0 msgr2=0x7f9960109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 --2- 172.21.15.59:0/2181659440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99601036e0 0x7f9960109f70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9958009be0 tx=0x7f995802bef0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 -- 172.21.15.59:0/2181659440 shutdown_connections 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 --2- 172.21.15.59:0/2181659440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9960102d20 0x7f99601031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 --2- 172.21.15.59:0/2181659440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99601036e0 0x7f9960109f70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 --2- 172.21.15.59:0/2181659440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960101b30 0x7f9960101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 -- 172.21.15.59:0/2181659440 >> 172.21.15.59:0/2181659440 conn(0x7f99600fd0c0 msgr2=0x7f99600ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.022+0000 7f9967a00640 1 -- 172.21.15.59:0/2181659440 shutdown_connections 2024-04-02T17:17:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.023+0000 7f9967a00640 1 -- 172.21.15.59:0/2181659440 wait complete. 2024-04-02T17:17:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.024+0000 7f9967a00640 1 Processor -- start 2024-04-02T17:17:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.024+0000 7f9967a00640 1 -- start start 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 0x7f9960197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99601036e0 0x7f996019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f996010d7e0 con 0x7f99601036e0 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f996010d660 con 0x7f9960101b30 2024-04-02T17:17:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9967a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f996010d960 con 0x7f9960102d20 2024-04-02T17:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f995fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 0x7f9960197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f995fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47148/0 (socket says 172.21.15.59:47148) 2024-04-02T17:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f995fe00640 1 -- 172.21.15.59:0/3896007356 learned_addr learned my addr 172.21.15.59:0/3896007356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99601036e0 0x7f996019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965000640 1 -- 172.21.15.59:0/3896007356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 msgr2=0x7f9960198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965000640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965000640 1 -- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99601036e0 msgr2=0x7f996019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.025+0000 7f9965000640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99601036e0 0x7f996019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f9965000640 1 -- 172.21.15.59:0/3896007356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996019f930 con 0x7f9960101b30 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f995fe00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f9965000640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 0x7f9960197d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9c0 tx=0x7f995400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f995d600640 1 -- 172.21.15.59:0/3896007356 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995400cd80 con 0x7f9960101b30 2024-04-02T17:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f995d600640 1 -- 172.21.15.59:0/3896007356 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9954010910 con 0x7f9960101b30 2024-04-02T17:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996019fc20 con 0x7f9960101b30 2024-04-02T17:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f995d600640 1 -- 172.21.15.59:0/3896007356 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954018980 con 0x7f9960101b30 2024-04-02T17:17:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.026+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99601a01c0 con 0x7f9960101b30 2024-04-02T17:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 Processor -- start 2024-04-02T17:17:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 -- start start 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e53ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5560e3db1d70 con 0x5560e3e4c8c0 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5560e3d5a2a0 con 0x5560e3e4bed0 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f996a7ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560e3dadd20 con 0x5560e3e51a30 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f995d600640 1 -- 172.21.15.59:0/3896007356 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f995401f020 con 0x7f9960101b30 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f9965a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f9965a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40834/0 (socket says 172.21.15.59:40834) 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f9965a00640 1 -- 172.21.15.59:0/2391391057 learned_addr learned my addr 172.21.15.59:0/2391391057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.030+0000 7f995fe00640 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e53ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965000640 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 -- 172.21.15.59:0/2391391057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 msgr2=0x5560e3e53ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e53ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 -- 172.21.15.59:0/2391391057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 msgr2=0x5560e3e513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 -- 172.21.15.59:0/2391391057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e3d6e840 con 0x5560e3e4bed0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9965a00640 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f995802c500 tx=0x7f9958009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298a9218ead1b423 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9952000640 1 -- 172.21.15.59:0/2391391057 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958043030 con 0x5560e3e4bed0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9952000640 1 -- 172.21.15.59:0/2391391057 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958033580 con 0x5560e3e4bed0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.031+0000 7f9952000640 1 -- 172.21.15.59:0/2391391057 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995802c890 con 0x5560e3e4bed0 2024-04-02T17:17:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f995d600640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 0x7f99480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f9965000640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 0x7f99480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f995d600640 1 -- 172.21.15.59:0/3896007356 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f99540d0400 con 0x7f9960101b30 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 -- 172.21.15.59:0/2391391057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 msgr2=0x5560e3e4c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f995802c500 tx=0x7f9958009330 comp rx=0 tx=0).stop 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 -- 172.21.15.59:0/2391391057 shutdown_connections 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e53ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 --2- 172.21.15.59:0/2391391057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e4c2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.032+0000 7f996a7ce740 1 -- 172.21.15.59:0/2391391057 >> 172.21.15.59:0/2391391057 conn(0x5560e3e47440 msgr2=0x5560e3e47840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.033+0000 7f996a7ce740 1 -- 172.21.15.59:0/2391391057 shutdown_connections 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.033+0000 7f996a7ce740 1 -- 172.21.15.59:0/2391391057 wait complete. 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.033+0000 7f9965000640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 0x7f99480a6190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99540ce210 tx=0x7f995400efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.033+0000 7f996a7ce740 1 -- start start 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.033+0000 7f996a7ce740 1 Processor -- start 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5560e3e54980 con 0x5560e3e51a30 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5560e3d6d660 con 0x5560e3e4bed0 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560e3e54b00 con 0x5560e3e4c8c0 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f996a7ce740 10 client.0 _ll_register_callbacks cb 0x5560e3d6d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f9965a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f9965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f995fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f9965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e61cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57306/0 (socket says 172.21.15.59:57306) 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.034+0000 7f9965a00640 1 -- 172.21.15.59:0/1733336125 learned_addr learned my addr 172.21.15.59:0/1733336125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 -- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 msgr2=0x5560e3e5a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 -- 172.21.15.59:0/1733336125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 msgr2=0x5560e3e61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 -- 172.21.15.59:0/1733336125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e3d7ce80 con 0x5560e3e4c8c0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f9965a00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f995fe00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5ac90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f994c005c80 tx=0x7f994c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fb3e1db96cd659 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f9946a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c012110 con 0x5560e3e4c8c0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958033580 con 0x5560e3e4c8c0 2024-04-02T17:17:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.035+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f994c004050 con 0x5560e3e4c8c0 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f994c0122b0 con 0x5560e3e4c8c0 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 10 client.26275 fetch_fsmap learned FSMap version 31 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5560e3ac6210 con 0x5560e3e4c8c0 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958033580 con 0x5560e3e4c8c0 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f994c0104e0 con 0x5560e3e4c8c0 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 10 client.26275 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 10 client.26275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 20 client.26275 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3e66740 0x5560e3e68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 4 client.26275 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.036+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e3d61cd0 con 0x5560e3e66740 2024-04-02T17:17:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.037+0000 7f9965a00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3e66740 0x5560e3e68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.037+0000 7f9965a00640 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3e66740 0x5560e3e68b80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8260d1c13b54b611 server_cookie=415e0161ffe2d5fa in_seq=0 out_seq=0 2024-04-02T17:17:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.037+0000 7f9946a00640 10 client.26275 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.038+0000 7f9946a00640 1 -- 172.21.15.59:0/1733336125 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560e3d61cd0 con 0x5560e3e66740 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.038+0000 7f9946a00640 10 client.26275 handle_command_reply: tid=0 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.038+0000 7f996a7ce740 1 client.26275 shutdown 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 msgr2=0x5560e3e5ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5ac90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f994c005c80 tx=0x7f994c00f790 comp rx=0 tx=0).stop 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 shutdown_connections 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3e66740 0x5560e3e68b80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5ac90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 --2- 172.21.15.59:0/1733336125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f9946a00640 0 client.26275 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 >> 172.21.15.59:0/1733336125 conn(0x5560e3e47440 msgr2=0x5560e3e46640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 shutdown_connections 2024-04-02T17:17:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 1 -- 172.21.15.59:0/1733336125 wait complete. 2024-04-02T17:17:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.039+0000 7f996a7ce740 20 client.26275 trim_cache size 0 max 16384 2024-04-02T17:17:02.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71685 1.98 MB/sec execute 29 sec latency 3798.809 ms 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.055+0000 7f996a7ce740 1 Processor -- start 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.055+0000 7f996a7ce740 1 -- start start 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a6e50 con 0x5560e3e4c8c0 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a8a50 con 0x5560e3e4bed0 2024-04-02T17:17:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f996a7ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a7520 con 0x5560e3e51a30 2024-04-02T17:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f9965a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f9965a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40852/0 (socket says 172.21.15.59:40852) 2024-04-02T17:17:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f9965a00640 1 -- 172.21.15.59:0/3488460236 learned_addr learned my addr 172.21.15.59:0/3488460236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f995fe00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.056+0000 7f9965a00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 -- 172.21.15.59:0/3488460236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 msgr2=0x5560e3e592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 -- 172.21.15.59:0/3488460236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 msgr2=0x5560e3e61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 -- 172.21.15.59:0/3488460236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e3b92620 con 0x5560e3e4bed0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f995fe00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9965a00640 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9958003fd0 tx=0x7f995802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef03acf5f898fde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9951600640 1 -- 172.21.15.59:0/3488460236 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958043040 con 0x5560e3e4bed0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9951600640 1 -- 172.21.15.59:0/3488460236 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958044040 con 0x5560e3e4bed0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.057+0000 7f9951600640 1 -- 172.21.15.59:0/3488460236 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958053240 con 0x5560e3e4bed0 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 -- 172.21.15.59:0/3488460236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 msgr2=0x5560e3e58db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9958003fd0 tx=0x7f995802c890 comp rx=0 tx=0).stop 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 -- 172.21.15.59:0/3488460236 shutdown_connections 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e4c8c0 0x5560e3e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e51a30 0x5560e3e592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3488460236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e58db0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 -- 172.21.15.59:0/3488460236 >> 172.21.15.59:0/3488460236 conn(0x5560e3e47440 msgr2=0x5560e3e47820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 -- 172.21.15.59:0/3488460236 shutdown_connections 2024-04-02T17:17:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.058+0000 7f996a7ce740 1 -- 172.21.15.59:0/3488460236 wait complete. 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 -- start start 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 Processor -- start 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a82c0 con 0x5560e3e51a30 2024-04-02T17:17:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a7bf0 con 0x5560e3e4bed0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99601a4e30 con 0x5560e3e4c8c0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.059+0000 7f996a7ce740 10 client.0 _ll_register_callbacks cb 0x7f994c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57328/0 (socket says 172.21.15.59:57328) 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 -- 172.21.15.59:0/3216192834 learned_addr learned my addr 172.21.15.59:0/3216192834 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f995fe00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 -- 172.21.15.59:0/3216192834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 msgr2=0x5560e3e5f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 -- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 msgr2=0x5560e3e5f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 -- 172.21.15.59:0/3216192834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e3e60d20 con 0x5560e3e51a30 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9965a00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f99580099a0 tx=0x7f995802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4bd82e538cf8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.060+0000 7f9946a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958043070 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958044040 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f9958053bd0 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f99580336d0 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f996a7ce740 10 client.16167 fetch_fsmap learned FSMap version 31 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5560e3eccd00 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9958044040 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.061+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f9958068020 con 0x5560e3e51a30 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 10 client.16167 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 20 client.16167 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3ecd0c0 0x5560e3ecf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 4 client.16167 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560e3dbf2b0 con 0x5560e3ecd0c0 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.062+0000 7f995fe00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3ecd0c0 0x5560e3ecf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.063+0000 7f995fe00640 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3ecd0c0 0x5560e3ecf4b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c83693b5d15001 server_cookie=17eab7a405d4273 in_seq=0 out_seq=0 2024-04-02T17:17:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.063+0000 7f9946a00640 10 client.16167 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f9946a00640 1 -- 172.21.15.59:0/3216192834 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560e3dbf2b0 con 0x5560e3ecd0c0 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f9946a00640 10 client.16167 handle_command_reply: tid=0 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 client.16167 shutdown 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 msgr2=0x5560e3e60730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f99580099a0 tx=0x7f995802c5f0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 shutdown_connections 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5560e3ecd0c0 0x5560e3ecf4b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e3e51a30 0x5560e3e60730 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560e3e4c8c0 0x5560e3e5f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 --2- 172.21.15.59:0/3216192834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560e3e4bed0 0x5560e3e5f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f9946a00640 0 client.16167 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 >> 172.21.15.59:0/3216192834 conn(0x5560e3e47440 msgr2=0x5560e3e58db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 shutdown_connections 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.087+0000 7f996a7ce740 1 -- 172.21.15.59:0/3216192834 wait complete. 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f996a7ce740 20 client.16167 trim_cache size 0 max 16384 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:17:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 msgr2=0x7f99480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 0x7f99480a6190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99540ce210 tx=0x7f995400efb0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 msgr2=0x7f9960197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 0x7f9960197d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9c0 tx=0x7f995400ee80 comp rx=0 tx=0).stop 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 shutdown_connections 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99480a3cd0 0x7f99480a6190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99601036e0 0x7f996019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9960102d20 0x7f9960198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.088+0000 7f9967a00640 1 --2- 172.21.15.59:0/3896007356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9960101b30 0x7f9960197d20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.089+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 >> 172.21.15.59:0/3896007356 conn(0x7f99600fd0c0 msgr2=0x7f9960105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.089+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 shutdown_connections 2024-04-02T17:17:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:02.089+0000 7f9967a00640 1 -- 172.21.15.59:0/3896007356 wait complete. 2024-04-02T17:17:03.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:02 smithi059 ceph-mon[31664]: pgmap v507: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 219 KiB/s wr, 64 op/s 2024-04-02T17:17:03.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 71685 1.91 MB/sec execute 30 sec latency 4799.011 ms 2024-04-02T17:17:03.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:02 smithi164 ceph-mon[36499]: pgmap v507: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 219 KiB/s wr, 64 op/s 2024-04-02T17:17:03.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:02 smithi154 ceph-mon[36479]: pgmap v507: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 219 KiB/s wr, 64 op/s 2024-04-02T17:17:04.048 INFO:tasks.workunit.client.0.smithi059.stdout: 1 72522 2.01 MB/sec execute 31 sec latency 4903.813 ms 2024-04-02T17:17:05.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:04 smithi059 ceph-mon[31664]: pgmap v508: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 188 KiB/s wr, 61 op/s 2024-04-02T17:17:05.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 74182 2.20 MB/sec execute 32 sec latency 15.896 ms 2024-04-02T17:17:05.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:04 smithi164 ceph-mon[36499]: pgmap v508: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 188 KiB/s wr, 61 op/s 2024-04-02T17:17:05.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:04 smithi154 ceph-mon[36479]: pgmap v508: 97 pgs: 97 active+clean; 35 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 188 KiB/s wr, 61 op/s 2024-04-02T17:17:06.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 74904 2.31 MB/sec execute 33 sec latency 25.167 ms 2024-04-02T17:17:07.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:06 smithi059 ceph-mon[31664]: pgmap v509: 97 pgs: 97 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 227 KiB/s wr, 65 op/s 2024-04-02T17:17:07.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 75328 2.26 MB/sec execute 34 sec latency 24.959 ms 2024-04-02T17:17:07.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:06 smithi164 ceph-mon[36499]: pgmap v509: 97 pgs: 97 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 227 KiB/s wr, 65 op/s 2024-04-02T17:17:07.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:06 smithi154 ceph-mon[36479]: pgmap v509: 97 pgs: 97 active+clean; 35 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 227 KiB/s wr, 65 op/s 2024-04-02T17:17:08.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:07 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:17:08.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 76879 2.38 MB/sec execute 35 sec latency 28.783 ms 2024-04-02T17:17:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:07 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:17:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:07 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:17:09.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:08 smithi059 ceph-mon[31664]: pgmap v510: 97 pgs: 97 active+clean; 36 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 264 KiB/s wr, 78 op/s 2024-04-02T17:17:09.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 78233 2.61 MB/sec execute 36 sec latency 15.912 ms 2024-04-02T17:17:09.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:08 smithi164 ceph-mon[36499]: pgmap v510: 97 pgs: 97 active+clean; 36 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 264 KiB/s wr, 78 op/s 2024-04-02T17:17:09.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:08 smithi154 ceph-mon[36479]: pgmap v510: 97 pgs: 97 active+clean; 36 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 264 KiB/s wr, 78 op/s 2024-04-02T17:17:10.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:09 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:17:10.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:09 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:17:10.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:09 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:17:10.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 78529 2.58 MB/sec execute 37 sec latency 31.402 ms 2024-04-02T17:17:10.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:09 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:17:10.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:09 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:17:10.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:09 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:17:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:09 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:17:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:09 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:17:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:09 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:17:11.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 79065 2.53 MB/sec execute 38 sec latency 30.270 ms 2024-04-02T17:17:11.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:10 smithi154 ceph-mon[36479]: pgmap v511: 97 pgs: 97 active+clean; 37 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 250 KiB/s wr, 62 op/s 2024-04-02T17:17:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:10 smithi059 ceph-mon[31664]: pgmap v511: 97 pgs: 97 active+clean; 37 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 250 KiB/s wr, 62 op/s 2024-04-02T17:17:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:10 smithi164 ceph-mon[36499]: pgmap v511: 97 pgs: 97 active+clean; 37 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 250 KiB/s wr, 62 op/s 2024-04-02T17:17:12.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 80577 2.65 MB/sec execute 39 sec latency 25.232 ms 2024-04-02T17:17:13.049 INFO:tasks.workunit.client.0.smithi059.stdout: 1 81850 2.86 MB/sec execute 40 sec latency 34.001 ms 2024-04-02T17:17:13.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:12 smithi154 ceph-mon[36479]: pgmap v512: 97 pgs: 97 active+clean; 37 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 325 KiB/s wr, 80 op/s 2024-04-02T17:17:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:12 smithi059 ceph-mon[31664]: pgmap v512: 97 pgs: 97 active+clean; 37 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 325 KiB/s wr, 80 op/s 2024-04-02T17:17:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:12 smithi164 ceph-mon[36499]: pgmap v512: 97 pgs: 97 active+clean; 37 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 325 KiB/s wr, 80 op/s 2024-04-02T17:17:14.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 82097 2.79 MB/sec execute 41 sec latency 24.750 ms 2024-04-02T17:17:15.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 82426 2.74 MB/sec execute 42 sec latency 462.910 ms 2024-04-02T17:17:15.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:14 smithi154 ceph-mon[36479]: pgmap v513: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 401 KiB/s wr, 104 op/s 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 -- 172.21.15.59:0/3426457746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 msgr2=0x7fb860073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 --2- 172.21.15.59:0/3426457746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb860073f90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb850009930 tx=0x7fb85002bc30 comp rx=0 tx=0).stop 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 -- 172.21.15.59:0/3426457746 shutdown_connections 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 --2- 172.21.15.59:0/3426457746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 0x7fb8601104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 --2- 172.21.15.59:0/3426457746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb8600749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 --2- 172.21.15.59:0/3426457746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb860073f90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 -- 172.21.15.59:0/3426457746 >> 172.21.15.59:0/3426457746 conn(0x7fb8600fd0c0 msgr2=0x7fb8600ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 -- 172.21.15.59:0/3426457746 shutdown_connections 2024-04-02T17:17:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.237+0000 7fb865c00640 1 -- 172.21.15.59:0/3426457746 wait complete. 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.238+0000 7fb865c00640 1 Processor -- start 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 -- start start 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb86019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 0x7fb8601a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb860113dc0 con 0x7fb860075b40 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb860113c40 con 0x7fb860074560 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb865c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb860113f40 con 0x7fb86010e100 2024-04-02T17:17:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb85ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb85ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52820/0 (socket says 172.21.15.59:52820) 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.239+0000 7fb85ea00640 1 -- 172.21.15.59:0/2430041045 learned_addr learned my addr 172.21.15.59:0/2430041045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85e000640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb86019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85f400640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 0x7fb8601a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 -- 172.21.15.59:0/2430041045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 msgr2=0x7fb8601a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 0x7fb8601a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 -- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 msgr2=0x7fb86019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb86019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 -- 172.21.15.59:0/2430041045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8601a5fb0 con 0x7fb860074560 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85e000640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb86019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb85ea00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb850000c00 tx=0x7fb85002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb857400640 1 -- 172.21.15.59:0/2430041045 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850041070 con 0x7fb860074560 2024-04-02T17:17:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb857400640 1 -- 172.21.15.59:0/2430041045 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb850004450 con 0x7fb860074560 2024-04-02T17:17:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8601a6240 con 0x7fb860074560 2024-04-02T17:17:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.240+0000 7fb857400640 1 -- 172.21.15.59:0/2430041045 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850034440 con 0x7fb860074560 2024-04-02T17:17:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.241+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a6780 con 0x7fb860074560 2024-04-02T17:17:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb857400640 1 -- 172.21.15.59:0/2430041045 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb850044070 con 0x7fb860074560 2024-04-02T17:17:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 Processor -- start 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 -- start start 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a959f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626a94ffd70 con 0x5626a959fa30 2024-04-02T17:17:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626a94a82a0 con 0x5626a959a8c0 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.244+0000 7fb8689ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626a94fbd20 con 0x5626a9599ed0 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a959f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57754/0 (socket says 172.21.15.59:57754) 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 -- 172.21.15.59:0/3371301020 learned_addr learned my addr 172.21.15.59:0/3371301020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 -- 172.21.15.59:0/3371301020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 msgr2=0x5626a959f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a959f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 -- 172.21.15.59:0/3371301020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 msgr2=0x5626a95a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 -- 172.21.15.59:0/3371301020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a94bc840 con 0x5626a9599ed0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.245+0000 7fb85f400640 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb858009fd0 tx=0x7fb8580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea08eab236b0afa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb857400640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 0x7fb8440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb84be00640 1 -- 172.21.15.59:0/3371301020 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004d20 con 0x5626a9599ed0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb84be00640 1 -- 172.21.15.59:0/3371301020 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb858004050 con 0x5626a9599ed0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb84be00640 1 -- 172.21.15.59:0/3371301020 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858005280 con 0x5626a9599ed0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb857400640 1 -- 172.21.15.59:0/2430041045 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb8500f1fb0 con 0x7fb860074560 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb85ea00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 0x7fb8440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb8689ae740 1 -- 172.21.15.59:0/3371301020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 msgr2=0x5626a959a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.246+0000 7fb8689ae740 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb858009fd0 tx=0x7fb8580118d0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 -- 172.21.15.59:0/3371301020 shutdown_connections 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a959f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 --2- 172.21.15.59:0/3371301020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a959a2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 -- 172.21.15.59:0/3371301020 >> 172.21.15.59:0/3371301020 conn(0x5626a9595440 msgr2=0x5626a9595840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 -- 172.21.15.59:0/3371301020 shutdown_connections 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb85ea00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 0x7fb8440a6190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb85002cfd0 tx=0x7fb8500f59f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 -- 172.21.15.59:0/3371301020 wait complete. 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 -- start start 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.247+0000 7fb8689ae740 1 Processor -- start 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 0x5626a95a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959fa30 0x5626a95ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626a95a29e0 con 0x5626a959a8c0 2024-04-02T17:17:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626a94bb660 con 0x5626a9599ed0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626a95a2b60 con 0x5626a959fa30 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb8689ae740 10 client.0 _ll_register_callbacks cb 0x5626a94bb050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb85e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb85e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59182/0 (socket says 172.21.15.59:59182) 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb85e000640 1 -- 172.21.15.59:0/4080030441 learned_addr learned my addr 172.21.15.59:0/4080030441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb85f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959fa30 0x5626a95ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.248+0000 7fb85f400640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 0x5626a95a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 -- 172.21.15.59:0/4080030441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959fa30 msgr2=0x5626a95ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959fa30 0x5626a95ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 -- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 msgr2=0x5626a95a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 0x5626a95a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 -- 172.21.15.59:0/4080030441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a94cae80 con 0x5626a959a8c0 2024-04-02T17:17:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85f400640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 0x5626a95a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb85e000640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb84c012700 tx=0x7fb84c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb8e030462a01b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb848c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c011b50 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb84c011cf0 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb858004050 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb84c004020 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb8689ae740 10 client.16179 fetch_fsmap learned FSMap version 32 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.249+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626a9214210 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb858004050 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fb84c025050 con 0x5626a959a8c0 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 10 client.16179 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 20 client.16179 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a95b02e0 0x5626a95b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 4 client.16179 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:17:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.250+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626a94afcd0 con 0x5626a95b02e0 2024-04-02T17:17:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.251+0000 7fb85f400640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a95b02e0 0x5626a95b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.251+0000 7fb85f400640 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a95b02e0 0x5626a95b26f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4631e1edd13582f server_cookie=a5484e8b471a8be8 in_seq=0 out_seq=0 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.251+0000 7fb848c00640 10 client.16179 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.252+0000 7fb848c00640 1 -- 172.21.15.59:0/4080030441 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626a94afcd0 con 0x5626a95b02e0 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.252+0000 7fb848c00640 10 client.16179 handle_command_reply: tid=0 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.252+0000 7fb8689ae740 1 client.16179 shutdown 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.252+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 msgr2=0x5626a95a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.252+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb84c012700 tx=0x7fb84c012bc0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 shutdown_connections 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a95b02e0 0x5626a95b26f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959a8c0 0x5626a95a48d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959fa30 0x5626a95ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4080030441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a9599ed0 0x5626a95a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb848c00640 0 client.16179 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 >> 172.21.15.59:0/4080030441 conn(0x5626a9595440 msgr2=0x5626a9594640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 shutdown_connections 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 1 -- 172.21.15.59:0/4080030441 wait complete. 2024-04-02T17:17:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.253+0000 7fb8689ae740 20 client.16179 trim_cache size 0 max 16384 2024-04-02T17:17:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.269+0000 7fb8689ae740 1 Processor -- start 2024-04-02T17:17:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.269+0000 7fb8689ae740 1 -- start start 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 0x5626a9593db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 0x5626a95ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959fa30 0x5626a95942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601af6d0 con 0x5626a9599ed0 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b12a0 con 0x5626a959fa30 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb8689ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601afda0 con 0x5626a959a8c0 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 0x5626a9593db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 0x5626a9593db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59190/0 (socket says 172.21.15.59:59190) 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85f400640 1 -- 172.21.15.59:0/4064125545 learned_addr learned my addr 172.21.15.59:0/4064125545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 0x5626a95ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85f400640 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959fa30 0x5626a95942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 -- 172.21.15.59:0/4064125545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959fa30 msgr2=0x5626a95942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959fa30 0x5626a95942f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 -- 172.21.15.59:0/4064125545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 msgr2=0x5626a9593db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 0x5626a9593db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.270+0000 7fb85e000640 1 -- 172.21.15.59:0/4064125545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a92e0620 con 0x5626a959a8c0 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.271+0000 7fb85e000640 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 0x5626a95ab8d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0126d0 tx=0x7fb84c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ae23776e7dc066 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.271+0000 7fb84b400640 1 -- 172.21.15.59:0/4064125545 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0041c0 con 0x5626a959a8c0 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.271+0000 7fb84b400640 1 -- 172.21.15.59:0/4064125545 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c004360 con 0x5626a959a8c0 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.271+0000 7fb84b400640 1 -- 172.21.15.59:0/4064125545 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c01c910 con 0x5626a959a8c0 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- 172.21.15.59:0/4064125545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 msgr2=0x5626a95ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 0x5626a95ab8d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0126d0 tx=0x7fb84c001e70 comp rx=0 tx=0).stop 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- 172.21.15.59:0/4064125545 shutdown_connections 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959fa30 0x5626a95942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a959a8c0 0x5626a95ab8d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 --2- 172.21.15.59:0/4064125545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a9599ed0 0x5626a9593db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- 172.21.15.59:0/4064125545 >> 172.21.15.59:0/4064125545 conn(0x5626a9595440 msgr2=0x5626a9595820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- 172.21.15.59:0/4064125545 shutdown_connections 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- 172.21.15.59:0/4064125545 wait complete. 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 -- start start 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.272+0000 7fb8689ae740 1 Processor -- start 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a95a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a95a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0b40 con 0x5626a959fa30 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0470 con 0x5626a959a8c0 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ad6b0 con 0x5626a9599ed0 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb8689ae740 10 client.0 _ll_register_callbacks cb 0x7fb84c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb85f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a95a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb85f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb85f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a95a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57766/0 (socket says 172.21.15.59:57766) 2024-04-02T17:17:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb85f400640 1 -- 172.21.15.59:0/1725559270 learned_addr learned my addr 172.21.15.59:0/1725559270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.273+0000 7fb85e000640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a95a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 -- 172.21.15.59:0/1725559270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 msgr2=0x5626a95a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a95a9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 -- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 msgr2=0x5626a95a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a95a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 -- 172.21.15.59:0/1725559270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a95ab0a0 con 0x5626a959fa30 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85e000640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a95a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb85f400640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95aaab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb858011910 tx=0x7fb858009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9159aae0e7c4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb848c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858003c00 con 0x5626a959fa30 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb858003da0 con 0x5626a959fa30 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84c004360 con 0x5626a959fa30 2024-04-02T17:17:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.274+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb858004050 con 0x5626a959fa30 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 10 client.16185 fetch_fsmap learned FSMap version 32 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626a9616dc0 con 0x5626a959fa30 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84c004360 con 0x5626a959fa30 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fb858044020 con 0x5626a959fa30 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 20 client.16185 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a9617620 0x5626a9619a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 4 client.16185 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.275+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626a950d2b0 con 0x5626a9617620 2024-04-02T17:17:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.276+0000 7fb85e000640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a9617620 0x5626a9619a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.276+0000 7fb85e000640 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a9617620 0x5626a9619a40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c497b8e67a9d1a server_cookie=f42f12adac2dfb42 in_seq=0 out_seq=0 2024-04-02T17:17:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.276+0000 7fb848c00640 10 client.16185 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:14 smithi059 ceph-mon[31664]: pgmap v513: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 401 KiB/s wr, 104 op/s 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb848c00640 1 -- 172.21.15.59:0/1725559270 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626a950d2b0 con 0x5626a9617620 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb848c00640 10 client.16185 handle_command_reply: tid=0 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 client.16185 shutdown 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 msgr2=0x5626a95aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95aaab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb858011910 tx=0x7fb858009500 comp rx=0 tx=0).stop 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 shutdown_connections 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626a9617620 0x5626a9619a40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626a959fa30 0x5626a95aaab0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626a959a8c0 0x5626a95a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb848c00640 0 client.16185 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 --2- 172.21.15.59:0/1725559270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626a9599ed0 0x5626a95a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.303+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 >> 172.21.15.59:0/1725559270 conn(0x5626a9595440 msgr2=0x5626a95986a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.304+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 shutdown_connections 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.304+0000 7fb8689ae740 1 -- 172.21.15.59:0/1725559270 wait complete. 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.304+0000 7fb8689ae740 20 client.16185 trim_cache size 0 max 16384 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:17:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 msgr2=0x7fb8440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 0x7fb8440a6190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb85002cfd0 tx=0x7fb8500f59f0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 msgr2=0x7fb86019e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb850000c00 tx=0x7fb85002c640 comp rx=0 tx=0).stop 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 shutdown_connections 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8440a3cd0 0x7fb8440a6190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860075b40 0x7fb86019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb86010e100 0x7fb8601a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 --2- 172.21.15.59:0/2430041045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb860074560 0x7fb86019e390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 >> 172.21.15.59:0/2430041045 conn(0x7fb8600fd0c0 msgr2=0x7fb8601014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 shutdown_connections 2024-04-02T17:17:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:15.305+0000 7fb865c00640 1 -- 172.21.15.59:0/2430041045 wait complete. 2024-04-02T17:17:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:14 smithi164 ceph-mon[36499]: pgmap v513: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 401 KiB/s wr, 104 op/s 2024-04-02T17:17:16.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 82426 2.67 MB/sec execute 43 sec latency 1463.101 ms 2024-04-02T17:17:16.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:15 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:15 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:15 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:17.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 82426 2.61 MB/sec execute 44 sec latency 2463.202 ms 2024-04-02T17:17:17.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:16 smithi059 ceph-mon[31664]: pgmap v514: 97 pgs: 97 active+clean; 39 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 508 KiB/s wr, 117 op/s 2024-04-02T17:17:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:16 smithi164 ceph-mon[36499]: pgmap v514: 97 pgs: 97 active+clean; 39 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 508 KiB/s wr, 117 op/s 2024-04-02T17:17:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:16 smithi154 ceph-mon[36479]: pgmap v514: 97 pgs: 97 active+clean; 39 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 508 KiB/s wr, 117 op/s 2024-04-02T17:17:18.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 82615 2.56 MB/sec execute 45 sec latency 3285.380 ms 2024-04-02T17:17:19.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 84217 2.66 MB/sec execute 46 sec latency 20.698 ms 2024-04-02T17:17:19.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:18 smithi059 ceph-mon[31664]: pgmap v515: 97 pgs: 97 active+clean; 36 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 500 KiB/s wr, 117 op/s 2024-04-02T17:17:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:18 smithi164 ceph-mon[36499]: pgmap v515: 97 pgs: 97 active+clean; 36 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 500 KiB/s wr, 117 op/s 2024-04-02T17:17:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:18 smithi154 ceph-mon[36479]: pgmap v515: 97 pgs: 97 active+clean; 36 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 500 KiB/s wr, 117 op/s 2024-04-02T17:17:20.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85413 2.84 MB/sec execute 47 sec latency 41.699 ms 2024-04-02T17:17:21.050 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85726 2.78 MB/sec execute 48 sec latency 19.337 ms 2024-04-02T17:17:21.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:20 smithi059 ceph-mon[31664]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 973 KiB/s wr, 127 op/s 2024-04-02T17:17:21.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:20 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:21.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:20 smithi164 ceph-mon[36499]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 973 KiB/s wr, 127 op/s 2024-04-02T17:17:21.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:20 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:21.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:20 smithi154 ceph-mon[36479]: pgmap v516: 97 pgs: 97 active+clean; 38 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 973 KiB/s wr, 127 op/s 2024-04-02T17:17:21.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:20 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:17:22.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85843 2.73 MB/sec execute 49 sec latency 828.812 ms 2024-04-02T17:17:23.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85843 2.68 MB/sec execute 50 sec latency 1828.987 ms 2024-04-02T17:17:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:22 smithi059 ceph-mon[31664]: pgmap v517: 97 pgs: 97 active+clean; 38 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-02T17:17:23.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:22 smithi164 ceph-mon[36499]: pgmap v517: 97 pgs: 97 active+clean; 38 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-02T17:17:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:22 smithi154 ceph-mon[36479]: pgmap v517: 97 pgs: 97 active+clean; 38 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-02T17:17:24.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85967 2.63 MB/sec execute 51 sec latency 1930.586 ms 2024-04-02T17:17:25.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85967 2.58 MB/sec execute 52 sec latency 1682.859 ms 2024-04-02T17:17:25.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:24 smithi059 ceph-mon[31664]: pgmap v518: 97 pgs: 97 active+clean; 35 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-04-02T17:17:25.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:24 smithi164 ceph-mon[36499]: pgmap v518: 97 pgs: 97 active+clean; 35 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-04-02T17:17:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:24 smithi154 ceph-mon[36479]: pgmap v518: 97 pgs: 97 active+clean; 35 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-04-02T17:17:26.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85967 2.53 MB/sec execute 53 sec latency 2682.979 ms 2024-04-02T17:17:27.051 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85967 2.48 MB/sec execute 54 sec latency 3683.113 ms 2024-04-02T17:17:27.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:26 smithi059 ceph-mon[31664]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-02T17:17:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:26 smithi164 ceph-mon[36499]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-02T17:17:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:26 smithi154 ceph-mon[36479]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-02T17:17:28.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 85967 2.44 MB/sec execute 55 sec latency 4683.259 ms 2024-04-02T17:17:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 -- 172.21.15.59:0/1605543896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 msgr2=0x7f3fec0768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1605543896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec0768e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402bc50 comp rx=0 tx=0).stop 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 -- 172.21.15.59:0/1605543896 shutdown_connections 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1605543896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 0x7f3fec078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1605543896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 0x7f3fec077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1605543896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec0768e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 -- 172.21.15.59:0/1605543896 >> 172.21.15.59:0/1605543896 conn(0x7f3fec0ff2f0 msgr2=0x7f3fec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 -- 172.21.15.59:0/1605543896 shutdown_connections 2024-04-02T17:17:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.458+0000 7f3ff3600640 1 -- 172.21.15.59:0/1605543896 wait complete. 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.459+0000 7f3ff3600640 1 Processor -- start 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.459+0000 7f3ff3600640 1 -- start start 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 0x7f3fec19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 0x7f3fec1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec113df0 con 0x7f3fec0780a0 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec113c70 con 0x7f3fec0764c0 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff3600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec113f70 con 0x7f3fec077700 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 0x7f3fec1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:49122/0 (socket says 172.21.15.59:49122) 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3ff0c00640 1 -- 172.21.15.59:0/1124616519 learned_addr learned my addr 172.21.15.59:0/1124616519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.460+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 0x7f3fec19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 -- 172.21.15.59:0/1124616519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 msgr2=0x7f3fec19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 0x7f3fec19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 -- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 msgr2=0x7f3fec1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 0x7f3fec1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 -- 172.21.15.59:0/1124616519 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec1a5fd0 con 0x7f3fec0764c0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff0c00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009230 tx=0x7f3fd402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3fea000640 1 -- 172.21.15.59:0/1124616519 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4042020 con 0x7f3fec0764c0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3fea000640 1 -- 172.21.15.59:0/1124616519 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd4004450 con 0x7f3fec0764c0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.461+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec1a6260 con 0x7f3fec0764c0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.462+0000 7f3fea000640 1 -- 172.21.15.59:0/1124616519 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4003e20 con 0x7f3fec0764c0 2024-04-02T17:17:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.462+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1a6800 con 0x7f3fec0764c0 2024-04-02T17:17:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.465+0000 7f3fea000640 1 -- 172.21.15.59:0/1124616519 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3fd4045020 con 0x7f3fec0764c0 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.465+0000 7f3ff637d740 1 Processor -- start 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.465+0000 7f3ff637d740 1 -- start start 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef2f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eceee8fd70 con 0x55eceef2fa30 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eceee382a0 con 0x55eceef2a8c0 2024-04-02T17:17:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff637d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eceee8bd20 con 0x55eceef29ed0 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef2f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50786/0 (socket says 172.21.15.59:50786) 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 -- 172.21.15.59:0/2006058529 learned_addr learned my addr 172.21.15.59:0/2006058529 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 -- 172.21.15.59:0/2006058529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 msgr2=0x55eceef2f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef2f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 -- 172.21.15.59:0/2006058529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 msgr2=0x55eceef31ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef31ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.466+0000 7f3ff1600640 1 -- 172.21.15.59:0/2006058529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceee4c840 con 0x55eceef29ed0 2024-04-02T17:17:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3fea000640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3ff1600640 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009fd0 tx=0x7f3fe40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca12aac5f706039 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3fea000640 1 -- 172.21.15.59:0/1124616519 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3fd40f2410 con 0x7f3fec0764c0 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3ff0c00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3fe1c00640 1 -- 172.21.15.59:0/2006058529 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004d20 con 0x55eceef29ed0 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3fe1c00640 1 -- 172.21.15.59:0/2006058529 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4004050 con 0x55eceef29ed0 2024-04-02T17:17:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.467+0000 7f3fe1c00640 1 -- 172.21.15.59:0/2006058529 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4005260 con 0x55eceef29ed0 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 -- 172.21.15.59:0/2006058529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 msgr2=0x55eceef2a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009fd0 tx=0x7f3fe40118d0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff0c00640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40be330 tx=0x7f3fd40f5e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 -- 172.21.15.59:0/2006058529 shutdown_connections 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef31ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef2f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2006058529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2a2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 -- 172.21.15.59:0/2006058529 >> 172.21.15.59:0/2006058529 conn(0x55eceef25440 msgr2=0x55eceef25840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 -- 172.21.15.59:0/2006058529 shutdown_connections 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.468+0000 7f3ff637d740 1 -- 172.21.15.59:0/2006058529 wait complete. 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.469+0000 7f3ff637d740 1 -- start start 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.469+0000 7f3ff637d740 1 Processor -- start 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 0x55eceef349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2fa30 0x55eceef3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eceef329e0 con 0x55eceef2a8c0 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eceee4b660 con 0x55eceef2fa30 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eceef32b60 con 0x55eceef29ed0 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff637d740 10 client.0 _ll_register_callbacks cb 0x55eceee4b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 0x55eceef349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50800/0 (socket says 172.21.15.59:50800) 2024-04-02T17:17:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff1600640 1 -- 172.21.15.59:0/3893515798 learned_addr learned my addr 172.21.15.59:0/3893515798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.470+0000 7f3ff1600640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2fa30 0x55eceef3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 -- 172.21.15.59:0/3893515798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 msgr2=0x55eceef34470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 -- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2fa30 msgr2=0x55eceef3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2fa30 0x55eceef3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 -- 172.21.15.59:0/3893515798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceee5ae80 con 0x55eceef2a8c0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3ff1600640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 0x55eceef349b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3fec19f8c0 tx=0x7f3fdc012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3431e88b206d1204 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc004030 con 0x55eceef2a8c0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3fdc0041d0 con 0x55eceef2a8c0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.471+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe4004050 con 0x55eceef2a8c0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc004410 con 0x55eceef2a8c0 2024-04-02T17:17:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 10 client.16197 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eceeba4210 con 0x55eceef2a8c0 2024-04-02T17:17:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4004050 con 0x55eceef2a8c0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3fdc025050 con 0x55eceef2a8c0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 10 client.16197 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 20 client.16197 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceef403a0 0x55eceef42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 4 client.16197 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.472+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eceee3fcd0 con 0x55eceef403a0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.473+0000 7f3ff1600640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceef403a0 0x55eceef42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.473+0000 7f3ff1600640 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceef403a0 0x55eceef42800 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891681d024f14e49 server_cookie=7f12e03be4354fef in_seq=0 out_seq=0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.473+0000 7f3fda000640 10 client.16197 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.474+0000 7f3fda000640 1 -- 172.21.15.59:0/3893515798 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eceee3fcd0 con 0x55eceef403a0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.474+0000 7f3fda000640 10 client.16197 handle_command_reply: tid=0 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.474+0000 7f3ff637d740 1 client.16197 shutdown 2024-04-02T17:17:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.474+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 msgr2=0x55eceef349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.474+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 0x55eceef349b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3fec19f8c0 tx=0x7f3fdc012b60 comp rx=0 tx=0).stop 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 shutdown_connections 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceef403a0 0x55eceef42800 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2a8c0 0x55eceef349b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2fa30 0x55eceef3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 --2- 172.21.15.59:0/3893515798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef34470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3fda000640 0 client.16197 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 >> 172.21.15.59:0/3893515798 conn(0x55eceef25440 msgr2=0x55eceef24640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 shutdown_connections 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 1 -- 172.21.15.59:0/3893515798 wait complete. 2024-04-02T17:17:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.475+0000 7f3ff637d740 20 client.16197 trim_cache size 0 max 16384 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.493+0000 7f3ff637d740 1 Processor -- start 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.493+0000 7f3ff637d740 1 -- start start 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef2c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1af6f0 con 0x55eceef2fa30 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1b1230 con 0x55eceef2a8c0 2024-04-02T17:17:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff637d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1afdc0 con 0x55eceef29ed0 2024-04-02T17:17:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:49152/0 (socket says 172.21.15.59:49152) 2024-04-02T17:17:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3fe3e00640 1 -- 172.21.15.59:0/2044357290 learned_addr learned my addr 172.21.15.59:0/2044357290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3ff1600640 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef2c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.494+0000 7f3fe3e00640 1 -- 172.21.15.59:0/2044357290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 msgr2=0x55eceef2bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe3e00640 1 -- 172.21.15.59:0/2044357290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 msgr2=0x55eceef2c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef2c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe3e00640 1 -- 172.21.15.59:0/2044357290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eceec70620 con 0x55eceef2a8c0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3ff1600640 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef2c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc012420 tx=0x7f3fdc005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5c948008c233c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe1200640 1 -- 172.21.15.59:0/2044357290 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc003f90 con 0x55eceef2a8c0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe1200640 1 -- 172.21.15.59:0/2044357290 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fdc02b020 con 0x55eceef2a8c0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.495+0000 7f3fe1200640 1 -- 172.21.15.59:0/2044357290 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00fbd0 con 0x55eceef2a8c0 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 -- 172.21.15.59:0/2044357290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 msgr2=0x55eceef3b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc012420 tx=0x7f3fdc005640 comp rx=0 tx=0).stop 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 -- 172.21.15.59:0/2044357290 shutdown_connections 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef2fa30 0x55eceef2c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef3b9b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 --2- 172.21.15.59:0/2044357290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef29ed0 0x55eceef2bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 -- 172.21.15.59:0/2044357290 >> 172.21.15.59:0/2044357290 conn(0x55eceef25440 msgr2=0x55eceef32f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 -- 172.21.15.59:0/2044357290 shutdown_connections 2024-04-02T17:17:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.496+0000 7f3ff637d740 1 -- 172.21.15.59:0/2044357290 wait complete. 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 -- start start 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 Processor -- start 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef29ed0 0x55eceef38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 0x55eceef3a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1b0b60 con 0x55eceef29ed0 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1b0490 con 0x55eceef2a8c0 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1ad6d0 con 0x55eceef2fa30 2024-04-02T17:17:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.497+0000 7f3ff637d740 10 client.0 _ll_register_callbacks cb 0x7f3fdc009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:49166/0 (socket says 172.21.15.59:49166) 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 -- 172.21.15.59:0/983837750 learned_addr learned my addr 172.21.15.59:0/983837750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3ff1600640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef29ed0 0x55eceef38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3ff1600640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 0x55eceef3a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 -- 172.21.15.59:0/983837750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 msgr2=0x55eceef3a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 0x55eceef3a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 -- 172.21.15.59:0/983837750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef29ed0 msgr2=0x55eceef38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef29ed0 0x55eceef38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 -- 172.21.15.59:0/983837750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eceef3a6f0 con 0x55eceef2a8c0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3ff1600640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 0x55eceef3a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fe3e00640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc003c60 tx=0x7f3fdc00fd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8f60ff2c184972 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.498+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc035060 con 0x55eceef2a8c0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f3fdc003800 con 0x55eceef2a8c0 2024-04-02T17:17:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fdc02b020 con 0x55eceef2a8c0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc003a00 con 0x55eceef2a8c0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3ff637d740 10 client.26294 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eceef3ba00 con 0x55eceef2a8c0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.499+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fdc02b020 con 0x55eceef2a8c0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f3fdc039030 con 0x55eceef2a8c0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 10 client.26294 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 20 client.26294 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceefa6e60 0x55eceefa9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 4 client.26294 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eceee9d2b0 con 0x55eceefa6e60 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.500+0000 7f3ff1600640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceefa6e60 0x55eceefa9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.501+0000 7f3ff1600640 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceefa6e60 0x55eceefa9280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db272b84e49b14bb server_cookie=425c13bd4cb8b64c in_seq=0 out_seq=0 2024-04-02T17:17:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.501+0000 7f3fda000640 10 client.26294 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3fda000640 1 -- 172.21.15.59:0/983837750 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eceee9d2b0 con 0x55eceefa6e60 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3fda000640 10 client.26294 handle_command_reply: tid=0 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3ff637d740 1 client.26294 shutdown 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 msgr2=0x55eceef39140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc003c60 tx=0x7f3fdc00fd80 comp rx=0 tx=0).stop 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 shutdown_connections 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.525+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eceefa6e60 0x55eceefa9280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eceef2fa30 0x55eceef3a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eceef2a8c0 0x55eceef39140 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 --2- 172.21.15.59:0/983837750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eceef29ed0 0x55eceef38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3fda000640 0 client.26294 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 >> 172.21.15.59:0/983837750 conn(0x55eceef25440 msgr2=0x55eceef2bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 shutdown_connections 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 1 -- 172.21.15.59:0/983837750 wait complete. 2024-04-02T17:17:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.526+0000 7f3ff637d740 20 client.26294 trim_cache size 0 max 16384 2024-04-02T17:17:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:17:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:17:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:17:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 msgr2=0x7f3fcc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40be330 tx=0x7f3fd40f5e50 comp rx=0 tx=0).stop 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 msgr2=0x7f3fec19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009230 tx=0x7f3fd402c670 comp rx=0 tx=0).stop 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 shutdown_connections 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3fcc0a3cd0 0x7f3fcc0a6190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fec0780a0 0x7f3fec1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fec077700 0x7f3fec19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 --2- 172.21.15.59:0/1124616519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fec0764c0 0x7f3fec19e400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 >> 172.21.15.59:0/1124616519 conn(0x7f3fec0ff2f0 msgr2=0x7f3fec100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 shutdown_connections 2024-04-02T17:17:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:28.527+0000 7f3ff3600640 1 -- 172.21.15.59:0/1124616519 wait complete. 2024-04-02T17:17:29.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 87523 2.51 MB/sec execute 56 sec latency 4781.744 ms 2024-04-02T17:17:29.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:28 smithi059 ceph-mon[31664]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-02T17:17:29.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:28 smithi164 ceph-mon[36499]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-02T17:17:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:28 smithi154 ceph-mon[36479]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-02T17:17:30.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 88830 2.66 MB/sec execute 57 sec latency 15.893 ms 2024-04-02T17:17:31.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 89165 2.63 MB/sec execute 58 sec latency 27.216 ms 2024-04-02T17:17:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:30 smithi059 ceph-mon[31664]: pgmap v521: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-02T17:17:31.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:30 smithi164 ceph-mon[36499]: pgmap v521: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-02T17:17:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:30 smithi154 ceph-mon[36479]: pgmap v521: 97 pgs: 97 active+clean; 35 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-02T17:17:32.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 89516 2.60 MB/sec execute 59 sec latency 324.980 ms 2024-04-02T17:17:33.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 89516 2.55 MB/sec execute 60 sec latency 1325.121 ms 2024-04-02T17:17:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:32 smithi059 ceph-mon[31664]: pgmap v522: 97 pgs: 97 active+clean; 36 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1014 KiB/s wr, 92 op/s 2024-04-02T17:17:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:32 smithi164 ceph-mon[36499]: pgmap v522: 97 pgs: 97 active+clean; 36 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1014 KiB/s wr, 92 op/s 2024-04-02T17:17:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:32 smithi154 ceph-mon[36479]: pgmap v522: 97 pgs: 97 active+clean; 36 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1014 KiB/s wr, 92 op/s 2024-04-02T17:17:34.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 90845 2.60 MB/sec execute 61 sec latency 1423.258 ms 2024-04-02T17:17:35.052 INFO:tasks.workunit.client.0.smithi059.stdout: 1 92236 2.74 MB/sec execute 62 sec latency 15.392 ms 2024-04-02T17:17:35.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:34 smithi059 ceph-mon[31664]: pgmap v523: 97 pgs: 97 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 680 KiB/s wr, 100 op/s 2024-04-02T17:17:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:34 smithi164 ceph-mon[36499]: pgmap v523: 97 pgs: 97 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 680 KiB/s wr, 100 op/s 2024-04-02T17:17:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:34 smithi154 ceph-mon[36479]: pgmap v523: 97 pgs: 97 active+clean; 37 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 680 KiB/s wr, 100 op/s 2024-04-02T17:17:36.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 92674 2.73 MB/sec execute 63 sec latency 27.743 ms 2024-04-02T17:17:37.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 92936 2.69 MB/sec execute 64 sec latency 408.767 ms 2024-04-02T17:17:37.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:36 smithi059 ceph-mon[31664]: pgmap v524: 97 pgs: 97 active+clean; 37 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 259 KiB/s wr, 69 op/s 2024-04-02T17:17:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:36 smithi164 ceph-mon[36499]: pgmap v524: 97 pgs: 97 active+clean; 37 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 259 KiB/s wr, 69 op/s 2024-04-02T17:17:37.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:36 smithi154 ceph-mon[36479]: pgmap v524: 97 pgs: 97 active+clean; 37 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 259 KiB/s wr, 69 op/s 2024-04-02T17:17:38.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 92936 2.65 MB/sec execute 65 sec latency 1408.921 ms 2024-04-02T17:17:39.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 93060 2.61 MB/sec execute 66 sec latency 1508.143 ms 2024-04-02T17:17:39.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:38 smithi059 ceph-mon[31664]: pgmap v525: 97 pgs: 97 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 290 KiB/s wr, 75 op/s 2024-04-02T17:17:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:38 smithi164 ceph-mon[36499]: pgmap v525: 97 pgs: 97 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 290 KiB/s wr, 75 op/s 2024-04-02T17:17:39.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:38 smithi154 ceph-mon[36479]: pgmap v525: 97 pgs: 97 active+clean; 38 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 290 KiB/s wr, 75 op/s 2024-04-02T17:17:40.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 93060 2.57 MB/sec execute 67 sec latency 1690.683 ms 2024-04-02T17:17:41.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 93060 2.53 MB/sec execute 68 sec latency 2690.839 ms 2024-04-02T17:17:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:40 smithi059 ceph-mon[31664]: pgmap v526: 97 pgs: 97 active+clean; 38 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 91 op/s 2024-04-02T17:17:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:40 smithi164 ceph-mon[36499]: pgmap v526: 97 pgs: 97 active+clean; 38 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 91 op/s 2024-04-02T17:17:41.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:40 smithi154 ceph-mon[36479]: pgmap v526: 97 pgs: 97 active+clean; 38 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 91 op/s 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 -- 172.21.15.59:0/3206749747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 msgr2=0x7fa7981095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 --2- 172.21.15.59:0/3206749747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa7981095f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa788009900 tx=0x7fa78802bc40 comp rx=0 tx=0).stop 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 -- 172.21.15.59:0/3206749747 shutdown_connections 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 --2- 172.21.15.59:0/3206749747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798109b30 0x7fa7981103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 --2- 172.21.15.59:0/3206749747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa7981095f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 --2- 172.21.15.59:0/3206749747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798107f80 0x7fa7981083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 -- 172.21.15.59:0/3206749747 >> 172.21.15.59:0/3206749747 conn(0x7fa798076f40 msgr2=0x7fa798079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 -- 172.21.15.59:0/3206749747 shutdown_connections 2024-04-02T17:17:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.676+0000 7fa79de00640 1 -- 172.21.15.59:0/3206749747 wait complete. 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.677+0000 7fa79de00640 1 Processor -- start 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 -- start start 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798107f80 0x7fa79819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa79819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa798113c10 con 0x7fa798109b30 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa798113a90 con 0x7fa798109170 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa798113d90 con 0x7fa798107f80 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798107f80 0x7fa79819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43508/0 (socket says 172.21.15.59:43508) 2024-04-02T17:17:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa79d400640 1 -- 172.21.15.59:0/578887820 learned_addr learned my addr 172.21.15.59:0/578887820 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.678+0000 7fa797e00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa79819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 -- 172.21.15.59:0/578887820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798107f80 msgr2=0x7fa79819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798107f80 0x7fa79819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 -- 172.21.15.59:0/578887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 msgr2=0x7fa79819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa79819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 -- 172.21.15.59:0/578887820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7981a5e80 con 0x7fa798109b30 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79d400640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa79000eb10 tx=0x7fa79000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa795600640 1 -- 172.21.15.59:0/578887820 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790004b20 con 0x7fa798109b30 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa795600640 1 -- 172.21.15.59:0/578887820 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790004cc0 con 0x7fa798109b30 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7981a6170 con 0x7fa798109b30 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.679+0000 7fa795600640 1 -- 172.21.15.59:0/578887820 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790010760 con 0x7fa798109b30 2024-04-02T17:17:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.680+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7981a6690 con 0x7fa798109b30 2024-04-02T17:17:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.683+0000 7fa7a2230740 1 Processor -- start 2024-04-02T17:17:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.683+0000 7fa795600640 1 -- 172.21.15.59:0/578887820 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa790005020 con 0x7fa798109b30 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.683+0000 7fa7a2230740 1 -- start start 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f40520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7f480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556aa7ea5d70 con 0x556aa7f40100 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556aa7e4e2a0 con 0x556aa7f40af0 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa7a2230740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556aa7ea1d20 con 0x556aa7f45c60 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f40520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f40520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43514/0 (socket says 172.21.15.59:43514) 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa797e00640 1 -- 172.21.15.59:0/1008450694 learned_addr learned my addr 172.21.15.59:0/1008450694 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa797e00640 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7f480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 -- 172.21.15.59:0/1008450694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 msgr2=0x556aa7f480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7f480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 -- 172.21.15.59:0/1008450694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 msgr2=0x556aa7f40520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f40520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa79ca00640 1 -- 172.21.15.59:0/1008450694 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa7e62840 con 0x556aa7f40af0 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.684+0000 7fa797e00640 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7f480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa79ca00640 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f455f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009bf0 tx=0x7fa78c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1156c3da6bcf5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa795600640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 0x7fa7800a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa785c00640 1 -- 172.21.15.59:0/1008450694 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c009da0 con 0x556aa7f40af0 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa785c00640 1 -- 172.21.15.59:0/1008450694 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005020 con 0x556aa7f40af0 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa795600640 1 -- 172.21.15.59:0/578887820 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7900d0b50 con 0x7fa798109b30 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa785c00640 1 -- 172.21.15.59:0/1008450694 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c004bc0 con 0x556aa7f40af0 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.685+0000 7fa797e00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 0x7fa7800a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 -- 172.21.15.59:0/1008450694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 msgr2=0x556aa7f455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa797e00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 0x7fa7800a6060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x556aa7f41350 tx=0x7fa788005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f455f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009bf0 tx=0x7fa78c011810 comp rx=0 tx=0).stop 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 -- 172.21.15.59:0/1008450694 shutdown_connections 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7f480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f455f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1008450694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f40520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 -- 172.21.15.59:0/1008450694 >> 172.21.15.59:0/1008450694 conn(0x556aa7f3b670 msgr2=0x556aa7f3ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 -- 172.21.15.59:0/1008450694 shutdown_connections 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.686+0000 7fa7a2230740 1 -- 172.21.15.59:0/1008450694 wait complete. 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.687+0000 7fa7a2230740 1 -- start start 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.687+0000 7fa7a2230740 1 Processor -- start 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.687+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 0x556aa7f4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556aa7f48c10 con 0x556aa7f40af0 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556aa7e61660 con 0x556aa7f45c60 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556aa7f48d90 con 0x556aa7f40100 2024-04-02T17:17:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa7a2230740 10 client.0 _ll_register_callbacks cb 0x556aa7e61050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 0x556aa7f4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39340/0 (socket says 172.21.15.59:39340) 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 -- 172.21.15.59:0/1384973687 learned_addr learned my addr 172.21.15.59:0/1384973687 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f4a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:53378/0 (socket says 172.21.15.59:53378) 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 0x556aa7f4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43522/0 (socket says 172.21.15.59:43522) 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 -- 172.21.15.59:0/1384973687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 msgr2=0x556aa7f4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 -- 172.21.15.59:0/1384973687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 msgr2=0x556aa7f4ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 0x556aa7f4ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.688+0000 7fa79d400640 1 -- 172.21.15.59:0/1384973687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa7e70e80 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa79d400640 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7900a54b0 tx=0x7fa79009c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe70c0d9bf4b2425 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa77a000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790005510 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa79009c1c0 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c005020 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7900a1120 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa7a2230740 10 client.26306 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556aa7bba210 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.689+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005020 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa790098b50 con 0x556aa7f45c60 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 10 client.26306 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 10 client.26306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 20 client.26306 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7f563f0 0x556aa7f58820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 4 client.26306 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:17:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa7e55cd0 con 0x556aa7f563f0 2024-04-02T17:17:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.690+0000 7fa79ca00640 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7f563f0 0x556aa7f58820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.691+0000 7fa79ca00640 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7f563f0 0x556aa7f58820 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f2fe85da7e390e server_cookie=4cfe58f4690c29c0 in_seq=0 out_seq=0 2024-04-02T17:17:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.691+0000 7fa77a000640 10 client.26306 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.691+0000 7fa77a000640 1 -- 172.21.15.59:0/1384973687 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556aa7e55cd0 con 0x556aa7f563f0 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.691+0000 7fa77a000640 10 client.26306 handle_command_reply: tid=0 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.691+0000 7fa7a2230740 1 client.26306 shutdown 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 msgr2=0x556aa7f51be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7900a54b0 tx=0x7fa79009c790 comp rx=0 tx=0).stop 2024-04-02T17:17:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 shutdown_connections 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7f563f0 0x556aa7f58820 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40af0 0x556aa7f4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f45c60 0x556aa7f51be0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 --2- 172.21.15.59:0/1384973687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa77a000640 0 client.26306 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 >> 172.21.15.59:0/1384973687 conn(0x556aa7f3b670 msgr2=0x556aa7f3a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 shutdown_connections 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 1 -- 172.21.15.59:0/1384973687 wait complete. 2024-04-02T17:17:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.692+0000 7fa7a2230740 20 client.26306 trim_cache size 0 max 16384 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.708+0000 7fa7a2230740 1 Processor -- start 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 -- start start 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 0x556aa7f49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981af640 con 0x556aa7f45c60 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981b1180 con 0x556aa7f40af0 2024-04-02T17:17:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa7a2230740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981afd10 con 0x556aa7f40100 2024-04-02T17:17:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa797e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa797e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39356/0 (socket says 172.21.15.59:39356) 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:53388/0 (socket says 172.21.15.59:53388) 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa797e00640 1 -- 172.21.15.59:0/4112287142 learned_addr learned my addr 172.21.15.59:0/4112287142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.709+0000 7fa79d400640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 0x556aa7f49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 -- 172.21.15.59:0/4112287142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 msgr2=0x556aa7f51b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 -- 172.21.15.59:0/4112287142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 msgr2=0x556aa7f49670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 0x556aa7f49670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 -- 172.21.15.59:0/4112287142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa7c86620 con 0x556aa7f40100 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79d400640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 0x556aa7f49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa797e00640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa79ca00640 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x556aa7f3c5e0 tx=0x7fa78c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab59e6bd8912683b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa785200640 1 -- 172.21.15.59:0/4112287142 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c031070 con 0x556aa7f40100 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa785200640 1 -- 172.21.15.59:0/4112287142 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005a70 con 0x556aa7f40100 2024-04-02T17:17:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.710+0000 7fa785200640 1 -- 172.21.15.59:0/4112287142 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c02bdb0 con 0x556aa7f40100 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 -- 172.21.15.59:0/4112287142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 msgr2=0x556aa7f49130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x556aa7f3c5e0 tx=0x7fa78c011810 comp rx=0 tx=0).stop 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 -- 172.21.15.59:0/4112287142 shutdown_connections 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f45c60 0x556aa7f49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 --2- 172.21.15.59:0/4112287142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f40100 0x556aa7f49130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 -- 172.21.15.59:0/4112287142 >> 172.21.15.59:0/4112287142 conn(0x556aa7f3b670 msgr2=0x556aa7f42080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 -- 172.21.15.59:0/4112287142 shutdown_connections 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.711+0000 7fa7a2230740 1 -- 172.21.15.59:0/4112287142 wait complete. 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 -- start start 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 Processor -- start 2024-04-02T17:17:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f4eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981b0ab0 con 0x556aa7f40100 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981b03e0 con 0x556aa7f40af0 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7981ad620 con 0x556aa7f45c60 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa7a2230740 10 client.0 _ll_register_callbacks cb 0x7fa79009cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa79ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.712+0000 7fa79d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:53404/0 (socket says 172.21.15.59:53404) 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79d400640 1 -- 172.21.15.59:0/3641526660 learned_addr learned my addr 172.21.15.59:0/3641526660 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa797e00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f4eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 -- 172.21.15.59:0/3641526660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 msgr2=0x556aa7fba990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 -- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 msgr2=0x556aa7f4eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f4eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 -- 172.21.15.59:0/3641526660 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa7f4f550 con 0x556aa7f40100 2024-04-02T17:17:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79d400640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa79ca00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f4e970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556aa7f3c5e0 tx=0x7fa78c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba44be7f05a946 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.713+0000 7fa77a000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c03c060 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa78c004300 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c005a70 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa78c031040 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa7a2230740 10 client.16221 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556aa7fbc010 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.714+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005a70 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa78c040030 con 0x556aa7f40100 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 10 client.16221 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 20 client.16221 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7fbc770 0x556aa7fbeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 4 client.16221 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:17:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aa7eb32b0 con 0x556aa7fbc770 2024-04-02T17:17:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.715+0000 7fa797e00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7fbc770 0x556aa7fbeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.716+0000 7fa797e00640 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7fbc770 0x556aa7fbeb60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95e500f183e40ca server_cookie=e9c1c54039f68669 in_seq=0 out_seq=0 2024-04-02T17:17:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.716+0000 7fa77a000640 10 client.16221 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa77a000640 1 -- 172.21.15.59:0/3641526660 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556aa7eb32b0 con 0x556aa7fbc770 2024-04-02T17:17:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa77a000640 10 client.16221 handle_command_reply: tid=0 2024-04-02T17:17:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 client.16221 shutdown 2024-04-02T17:17:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 msgr2=0x556aa7f4e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f4e970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556aa7f3c5e0 tx=0x7fa78c0052f0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 shutdown_connections 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556aa7fbc770 0x556aa7fbeb60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556aa7f45c60 0x556aa7fba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556aa7f40af0 0x556aa7f4eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 --2- 172.21.15.59:0/3641526660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556aa7f40100 0x556aa7f4e970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 >> 172.21.15.59:0/3641526660 conn(0x556aa7f3b670 msgr2=0x556aa7f42080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.746+0000 7fa77a000640 0 client.16221 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.747+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 shutdown_connections 2024-04-02T17:17:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.747+0000 7fa7a2230740 1 -- 172.21.15.59:0/3641526660 wait complete. 2024-04-02T17:17:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.747+0000 7fa7a2230740 20 client.16221 trim_cache size 0 max 16384 2024-04-02T17:17:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:17:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:17:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:17:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:17:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.748+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 msgr2=0x7fa7800a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.748+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 0x7fa7800a6060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x556aa7f41350 tx=0x7fa788005b50 comp rx=0 tx=0).stop 2024-04-02T17:17:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.748+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 msgr2=0x7fa7981a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa79000eb10 tx=0x7fa79000c2d0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 shutdown_connections 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7800a3ba0 0x7fa7800a6060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa798109b30 0x7fa7981a5810 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa798109170 0x7fa79819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 --2- 172.21.15.59:0/578887820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa798107f80 0x7fa79819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 >> 172.21.15.59:0/578887820 conn(0x7fa798076f40 msgr2=0x7fa79810c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.749+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 shutdown_connections 2024-04-02T17:17:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:41.750+0000 7fa79de00640 1 -- 172.21.15.59:0/578887820 wait complete. 2024-04-02T17:17:42.053 INFO:tasks.workunit.client.0.smithi059.stdout: 1 93060 2.50 MB/sec execute 69 sec latency 3691.024 ms 2024-04-02T17:17:43.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 93060 2.46 MB/sec execute 70 sec latency 4691.159 ms 2024-04-02T17:17:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:42 smithi059 ceph-mon[31664]: pgmap v527: 97 pgs: 97 active+clean; 38 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 335 KiB/s wr, 90 op/s 2024-04-02T17:17:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:42 smithi164 ceph-mon[36499]: pgmap v527: 97 pgs: 97 active+clean; 38 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 335 KiB/s wr, 90 op/s 2024-04-02T17:17:43.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:42 smithi154 ceph-mon[36479]: pgmap v527: 97 pgs: 97 active+clean; 38 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 335 KiB/s wr, 90 op/s 2024-04-02T17:17:44.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 94252 2.50 MB/sec execute 71 sec latency 4789.880 ms 2024-04-02T17:17:45.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 95645 2.59 MB/sec execute 72 sec latency 16.907 ms 2024-04-02T17:17:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:45 smithi059 ceph-mon[31664]: pgmap v528: 97 pgs: 97 active+clean; 39 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 331 KiB/s wr, 91 op/s 2024-04-02T17:17:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:45 smithi164 ceph-mon[36499]: pgmap v528: 97 pgs: 97 active+clean; 39 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 331 KiB/s wr, 91 op/s 2024-04-02T17:17:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:45 smithi154 ceph-mon[36479]: pgmap v528: 97 pgs: 97 active+clean; 39 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 331 KiB/s wr, 91 op/s 2024-04-02T17:17:46.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 96185 2.62 MB/sec execute 73 sec latency 25.303 ms 2024-04-02T17:17:47.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 96607 2.59 MB/sec execute 74 sec latency 119.864 ms 2024-04-02T17:17:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:47 smithi059 ceph-mon[31664]: pgmap v529: 97 pgs: 97 active+clean; 39 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 247 KiB/s wr, 69 op/s 2024-04-02T17:17:47.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:47 smithi164 ceph-mon[36499]: pgmap v529: 97 pgs: 97 active+clean; 39 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 247 KiB/s wr, 69 op/s 2024-04-02T17:17:47.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:47 smithi154 ceph-mon[36479]: pgmap v529: 97 pgs: 97 active+clean; 39 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 247 KiB/s wr, 69 op/s 2024-04-02T17:17:48.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 96607 2.55 MB/sec execute 75 sec latency 1120.012 ms 2024-04-02T17:17:49.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 97917 2.59 MB/sec execute 76 sec latency 1218.253 ms 2024-04-02T17:17:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:49 smithi059 ceph-mon[31664]: pgmap v530: 97 pgs: 97 active+clean; 39 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 71 op/s 2024-04-02T17:17:49.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:49 smithi164 ceph-mon[36499]: pgmap v530: 97 pgs: 97 active+clean; 39 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 71 op/s 2024-04-02T17:17:49.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:49 smithi154 ceph-mon[36479]: pgmap v530: 97 pgs: 97 active+clean; 39 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 71 op/s 2024-04-02T17:17:50.054 INFO:tasks.workunit.client.0.smithi059.stdout: 1 99316 2.71 MB/sec execute 77 sec latency 16.196 ms 2024-04-02T17:17:51.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 99757 2.69 MB/sec execute 78 sec latency 33.793 ms 2024-04-02T17:17:51.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:51 smithi059 ceph-mon[31664]: pgmap v531: 97 pgs: 97 active+clean; 40 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 267 KiB/s wr, 81 op/s 2024-04-02T17:17:51.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:51 smithi164 ceph-mon[36499]: pgmap v531: 97 pgs: 97 active+clean; 40 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 267 KiB/s wr, 81 op/s 2024-04-02T17:17:51.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:51 smithi154 ceph-mon[36479]: pgmap v531: 97 pgs: 97 active+clean; 40 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 267 KiB/s wr, 81 op/s 2024-04-02T17:17:52.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100028 2.66 MB/sec execute 79 sec latency 351.786 ms 2024-04-02T17:17:53.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100028 2.63 MB/sec execute 80 sec latency 1351.917 ms 2024-04-02T17:17:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:53 smithi059 ceph-mon[31664]: pgmap v532: 97 pgs: 97 active+clean; 40 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 279 KiB/s wr, 78 op/s 2024-04-02T17:17:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:53 smithi164 ceph-mon[36499]: pgmap v532: 97 pgs: 97 active+clean; 40 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 279 KiB/s wr, 78 op/s 2024-04-02T17:17:53.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:53 smithi154 ceph-mon[36479]: pgmap v532: 97 pgs: 97 active+clean; 40 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 279 KiB/s wr, 78 op/s 2024-04-02T17:17:54.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100152 2.60 MB/sec execute 81 sec latency 1455.036 ms 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 -- 172.21.15.59:0/1213458281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f80103b80 msgr2=0x7f7f80103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 --2- 172.21.15.59:0/1213458281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f80103b80 0x7f7f80103fa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009860 tx=0x7f7f7402bb90 comp rx=0 tx=0).stop 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 -- 172.21.15.59:0/1213458281 shutdown_connections 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 --2- 172.21.15.59:0/1213458281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 --2- 172.21.15.59:0/1213458281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 0x7f7f801051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 --2- 172.21.15.59:0/1213458281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f80103b80 0x7f7f80103fa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 -- 172.21.15.59:0/1213458281 >> 172.21.15.59:0/1213458281 conn(0x7f7f800ff110 msgr2=0x7f7f80101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.900+0000 7f7f87200640 1 -- 172.21.15.59:0/1213458281 shutdown_connections 2024-04-02T17:17:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.901+0000 7f7f87200640 1 -- 172.21.15.59:0/1213458281 wait complete. 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.901+0000 7f7f87200640 1 Processor -- start 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 -- start start 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 0x7f7f8019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 0x7f7f801a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8010f7d0 con 0x7f7f8019f600 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8010f650 con 0x7f7f80104d70 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f87200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8010f950 con 0x7f7f80105730 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 0x7f7f801a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f85e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 0x7f7f8019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f85400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 0x7f7f801a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36500/0 (socket says 172.21.15.59:36500) 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.902+0000 7f7f86800640 1 -- 172.21.15.59:0/1876955527 learned_addr learned my addr 172.21.15.59:0/1876955527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 -- 172.21.15.59:0/1876955527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 msgr2=0x7f7f8019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 0x7f7f8019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 -- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 msgr2=0x7f7f801a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 0x7f7f801a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 -- 172.21.15.59:0/1876955527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a2060 con 0x7f7f80105730 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f85400640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8019a5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000e790 tx=0x7f7f7000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f7aa00640 1 -- 172.21.15.59:0/1876955527 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000eea0 con 0x7f7f80105730 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.903+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a2350 con 0x7f7f80105730 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.904+0000 7f7f7aa00640 1 -- 172.21.15.59:0/1876955527 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7000ccc0 con 0x7f7f80105730 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.904+0000 7f7f7aa00640 1 -- 172.21.15.59:0/1876955527 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70018760 con 0x7f7f80105730 2024-04-02T17:17:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.904+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a28f0 con 0x7f7f80105730 2024-04-02T17:17:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 Processor -- start 2024-04-02T17:17:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 -- start start 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c9934505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c9934530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c9933b0d70 con 0x55c993450c60 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c9933592a0 con 0x55c99344b100 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9933acd20 con 0x55c99344baf0 2024-04-02T17:17:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.907+0000 7f7f7aa00640 1 -- 172.21.15.59:0/1876955527 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f70010430 con 0x7f7f80105730 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f85400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c9934530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42154/0 (socket says 172.21.15.59:42154) 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 -- 172.21.15.59:0/2579570005 learned_addr learned my addr 172.21.15.59:0/2579570005 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f85e00640 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c9934505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 -- 172.21.15.59:0/2579570005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 msgr2=0x55c9934505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c9934505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 -- 172.21.15.59:0/2579570005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 msgr2=0x55c9934530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c9934530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 -- 172.21.15.59:0/2579570005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99336d840 con 0x55c99344b100 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.908+0000 7f7f86800640 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c011570 tx=0x7f7f7c011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdace6c2ee13e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f6f000640 1 -- 172.21.15.59:0/2579570005 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004b20 con 0x55c99344b100 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f6f000640 1 -- 172.21.15.59:0/2579570005 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c004cc0 con 0x55c99344b100 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f6f000640 1 -- 172.21.15.59:0/2579570005 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c005560 con 0x55c99344b100 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f7aa00640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 0x7f7f680a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f7aa00640 1 -- 172.21.15.59:0/1876955527 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7f700d0690 con 0x7f7f80105730 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f85e00640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 0x7f7f680a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2579570005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 msgr2=0x55c99344b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.909+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c011570 tx=0x7f7f7c011a30 comp rx=0 tx=0).stop 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2579570005 shutdown_connections 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c9934530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c9934505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2579570005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99344b520 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2579570005 >> 172.21.15.59:0/2579570005 conn(0x55c993446670 msgr2=0x55c993446a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2579570005 shutdown_connections 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2579570005 wait complete. 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 -- start start 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f85e00640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 0x7f7f680a6060 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f74037040 tx=0x7f7f740058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.910+0000 7f7f8b5c3740 1 Processor -- start 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 0x55c993455530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c993453c10 con 0x55c993450c60 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c99336c660 con 0x55c99344baf0 2024-04-02T17:17:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c993453d90 con 0x55c99344b100 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f8b5c3740 10 client.0 _ll_register_callbacks cb 0x55c99336c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f85400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f85e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 0x55c993455530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f85400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42156/0 (socket says 172.21.15.59:42156) 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.911+0000 7f7f85400640 1 -- 172.21.15.59:0/2456885779 learned_addr learned my addr 172.21.15.59:0/2456885779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 -- 172.21.15.59:0/2456885779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 msgr2=0x55c993455530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 0x55c993455530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 -- 172.21.15.59:0/2456885779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 msgr2=0x55c99345cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 -- 172.21.15.59:0/2456885779 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99337be80 con 0x55c99344baf0 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85e00640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 0x55c993455530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f86800640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f85400640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7f70098dc0 tx=0x7f7f70098cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f589297e2a096ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f63400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7009c960 con 0x55c99344baf0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7f70010720 con 0x55c99344baf0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c004cc0 con 0x55c99344baf0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f700a0c60 con 0x55c99344baf0 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f8b5c3740 10 client.26324 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.912+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9930c5210 con 0x55c99344baf0 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c004cc0 con 0x55c99344baf0 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f7f70087180 con 0x55c99344baf0 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 10 client.26324 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 20 client.26324 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934613a0 0x55c993463790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 4 client.26324 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.913+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c993360cd0 con 0x55c9934613a0 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.914+0000 7f7f85e00640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934613a0 0x55c993463790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.914+0000 7f7f85e00640 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934613a0 0x55c993463790 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8489bde0cc3061a server_cookie=5b1a2c8cba5e6bce in_seq=0 out_seq=0 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.914+0000 7f7f63400640 10 client.26324 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f63400640 1 -- 172.21.15.59:0/2456885779 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c993360cd0 con 0x55c9934613a0 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f63400640 10 client.26324 handle_command_reply: tid=0 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 client.26324 shutdown 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 msgr2=0x55c993455a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7f70098dc0 tx=0x7f7f70098cc0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 shutdown_connections 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934613a0 0x55c993463790 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344baf0 0x55c993455a70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/2456885779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344b100 0x55c993455530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 >> 172.21.15.59:0/2456885779 conn(0x55c993446670 msgr2=0x55c993445870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f63400640 0 client.26324 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 shutdown_connections 2024-04-02T17:17:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/2456885779 wait complete. 2024-04-02T17:17:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.915+0000 7f7f8b5c3740 20 client.26324 trim_cache size 0 max 16384 2024-04-02T17:17:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 Processor -- start 2024-04-02T17:17:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 -- start start 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c993450c60 0x55c993445520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ab8b0 con 0x55c99344b100 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ad3f0 con 0x55c993450c60 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.932+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801abf80 con 0x55c99344baf0 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36520/0 (socket says 172.21.15.59:36520) 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 -- 172.21.15.59:0/1109762878 learned_addr learned my addr 172.21.15.59:0/1109762878 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f86800640 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c993450c60 0x55c993445520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85400640 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 -- 172.21.15.59:0/1109762878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 msgr2=0x55c99345ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 -- 172.21.15.59:0/1109762878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c993450c60 msgr2=0x55c993445520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c993450c60 0x55c993445520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 -- 172.21.15.59:0/1109762878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c993191620 con 0x55c99344b100 2024-04-02T17:17:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.933+0000 7f7f85e00640 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f740349e0 tx=0x7f7f7403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a5b326829649b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.934+0000 7f7f6e600640 1 -- 172.21.15.59:0/1109762878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74047040 con 0x55c99344b100 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.934+0000 7f7f6e600640 1 -- 172.21.15.59:0/1109762878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7402ca10 con 0x55c99344b100 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.934+0000 7f7f6e600640 1 -- 172.21.15.59:0/1109762878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74039040 con 0x55c99344b100 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/1109762878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 msgr2=0x55c993444fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f740349e0 tx=0x7f7f7403f000 comp rx=0 tx=0).stop 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/1109762878 shutdown_connections 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c993450c60 0x55c993445520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/1109762878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c99344b100 0x55c993444fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/1109762878 >> 172.21.15.59:0/1109762878 conn(0x55c993446670 msgr2=0x55c99344cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/1109762878 shutdown_connections 2024-04-02T17:17:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/1109762878 wait complete. 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 -- start start 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.935+0000 7f7f8b5c3740 1 Processor -- start 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99345a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801acd20 con 0x55c993450c60 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ac650 con 0x55c99344b100 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801a9890 con 0x55c99344baf0 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f8b5c3740 10 client.0 _ll_register_callbacks cb 0x7f7f70010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36534/0 (socket says 172.21.15.59:36534) 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f86800640 1 -- 172.21.15.59:0/3438593520 learned_addr learned my addr 172.21.15.59:0/3438593520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:17:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f85400640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.936+0000 7f7f85e00640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99345a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 -- 172.21.15.59:0/3438593520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 msgr2=0x55c99345a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 -- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 msgr2=0x55c99345a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99345a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 -- 172.21.15.59:0/3438593520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99345bf90 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f86800640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c011510 tx=0x7f7f7c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4943447b197e0e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f63400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004da0 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7f7c004f40 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.937+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7402ca10 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c00f5c0 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 10 client.16239 fetch_fsmap learned FSMap version 33 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9934c7c60 con 0x55c993450c60 2024-04-02T17:17:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7402ca10 con 0x55c993450c60 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f7f7c034020 con 0x55c993450c60 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 10 client.16239 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 20 client.16239 populate_metadata read hostname 'smithi059' 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934c84a0 0x55c9934ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 4 client.16239 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.938+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9933be2b0 con 0x55c9934c84a0 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.939+0000 7f7f85e00640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934c84a0 0x55c9934ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:17:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.939+0000 7f7f85e00640 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934c84a0 0x55c9934ca8c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a0cb777e9cc41b server_cookie=ce2f350dc1cea262 in_seq=0 out_seq=0 2024-04-02T17:17:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.939+0000 7f7f63400640 10 client.16239 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.969+0000 7f7f63400640 1 -- 172.21.15.59:0/3438593520 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9933be2b0 con 0x55c9934c84a0 2024-04-02T17:17:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.969+0000 7f7f63400640 10 client.16239 handle_command_reply: tid=0 2024-04-02T17:17:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.969+0000 7f7f8b5c3740 1 client.16239 shutdown 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 msgr2=0x55c99345b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c011510 tx=0x7f7f7c004c30 comp rx=0 tx=0).stop 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 shutdown_connections 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55c9934c84a0 0x55c9934ca8c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c993450c60 0x55c99345b9a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c99344baf0 0x55c99345a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f63400640 0 client.16239 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 --2- 172.21.15.59:0/3438593520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c99344b100 0x55c99345a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 >> 172.21.15.59:0/3438593520 conn(0x55c993446670 msgr2=0x55c9934498d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 shutdown_connections 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 1 -- 172.21.15.59:0/3438593520 wait complete. 2024-04-02T17:17:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.970+0000 7f7f8b5c3740 20 client.16239 trim_cache size 0 max 16384 2024-04-02T17:17:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:17:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:17:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:17:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 msgr2=0x7f7f680a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 0x7f7f680a6060 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f74037040 tx=0x7f7f740058b0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 msgr2=0x7f7f8019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8019a5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000e790 tx=0x7f7f7000ec50 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 shutdown_connections 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7f680a3ba0 0x7f7f680a6060 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f8019f600 0x7f7f801a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f80105730 0x7f7f8019a5a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 --2- 172.21.15.59:0/1876955527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f80104d70 0x7f7f8019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:17:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.971+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 >> 172.21.15.59:0/1876955527 conn(0x7f7f800ff110 msgr2=0x7f7f80108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:17:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.972+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 shutdown_connections 2024-04-02T17:17:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:17:54.972+0000 7f7f87200640 1 -- 172.21.15.59:0/1876955527 wait complete. 2024-04-02T17:17:55.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100152 2.57 MB/sec execute 82 sec latency 1689.021 ms 2024-04-02T17:17:55.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:55 smithi059 ceph-mon[31664]: pgmap v533: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 494 KiB/s wr, 100 op/s 2024-04-02T17:17:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:55 smithi164 ceph-mon[36499]: pgmap v533: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 494 KiB/s wr, 100 op/s 2024-04-02T17:17:55.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:55 smithi154 ceph-mon[36479]: pgmap v533: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 494 KiB/s wr, 100 op/s 2024-04-02T17:17:56.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100152 2.54 MB/sec execute 83 sec latency 2689.215 ms 2024-04-02T17:17:57.055 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100152 2.51 MB/sec execute 84 sec latency 3689.383 ms 2024-04-02T17:17:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:57 smithi059 ceph-mon[31664]: pgmap v534: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 535 KiB/s wr, 102 op/s 2024-04-02T17:17:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:57 smithi164 ceph-mon[36499]: pgmap v534: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 535 KiB/s wr, 102 op/s 2024-04-02T17:17:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:57 smithi154 ceph-mon[36479]: pgmap v534: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 535 KiB/s wr, 102 op/s 2024-04-02T17:17:58.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 100152 2.48 MB/sec execute 85 sec latency 4689.502 ms 2024-04-02T17:17:59.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 101417 2.51 MB/sec execute 86 sec latency 4783.920 ms 2024-04-02T17:17:59.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:17:59 smithi059 ceph-mon[31664]: pgmap v535: 97 pgs: 97 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 591 KiB/s wr, 101 op/s 2024-04-02T17:17:59.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:17:59 smithi164 ceph-mon[36499]: pgmap v535: 97 pgs: 97 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 591 KiB/s wr, 101 op/s 2024-04-02T17:17:59.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:17:59 smithi154 ceph-mon[36479]: pgmap v535: 97 pgs: 97 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 591 KiB/s wr, 101 op/s 2024-04-02T17:18:00.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 102729 2.58 MB/sec execute 87 sec latency 17.602 ms 2024-04-02T17:18:01.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103299 2.61 MB/sec execute 88 sec latency 31.541 ms 2024-04-02T17:18:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:01 smithi059 ceph-mon[31664]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 673 KiB/s wr, 103 op/s 2024-04-02T17:18:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:01 smithi164 ceph-mon[36499]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 673 KiB/s wr, 103 op/s 2024-04-02T17:18:01.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:01 smithi154 ceph-mon[36479]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 673 KiB/s wr, 103 op/s 2024-04-02T17:18:02.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103576 2.58 MB/sec execute 89 sec latency 347.032 ms 2024-04-02T17:18:03.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103576 2.55 MB/sec execute 90 sec latency 1347.146 ms 2024-04-02T17:18:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:03 smithi059 ceph-mon[31664]: pgmap v537: 97 pgs: 97 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 625 KiB/s wr, 91 op/s 2024-04-02T17:18:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:03 smithi164 ceph-mon[36499]: pgmap v537: 97 pgs: 97 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 625 KiB/s wr, 91 op/s 2024-04-02T17:18:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:03 smithi154 ceph-mon[36479]: pgmap v537: 97 pgs: 97 active+clean; 42 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 625 KiB/s wr, 91 op/s 2024-04-02T17:18:04.056 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103632 2.52 MB/sec execute 91 sec latency 1445.577 ms 2024-04-02T17:18:05.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103632 2.49 MB/sec execute 92 sec latency 1808.832 ms 2024-04-02T17:18:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:05 smithi164 ceph-mon[36499]: pgmap v538: 97 pgs: 97 active+clean; 43 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 640 KiB/s wr, 100 op/s 2024-04-02T17:18:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:05 smithi154 ceph-mon[36479]: pgmap v538: 97 pgs: 97 active+clean; 43 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 640 KiB/s wr, 100 op/s 2024-04-02T17:18:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:05 smithi059 ceph-mon[31664]: pgmap v538: 97 pgs: 97 active+clean; 43 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 640 KiB/s wr, 100 op/s 2024-04-02T17:18:06.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103632 2.47 MB/sec execute 93 sec latency 2808.943 ms 2024-04-02T17:18:07.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103632 2.44 MB/sec execute 94 sec latency 3809.055 ms 2024-04-02T17:18:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:07 smithi164 ceph-mon[36499]: pgmap v539: 97 pgs: 97 active+clean; 44 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 574 KiB/s wr, 82 op/s 2024-04-02T17:18:07.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:07 smithi059 ceph-mon[31664]: pgmap v539: 97 pgs: 97 active+clean; 44 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 574 KiB/s wr, 82 op/s 2024-04-02T17:18:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:07 smithi154 ceph-mon[36479]: pgmap v539: 97 pgs: 97 active+clean; 44 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 574 KiB/s wr, 82 op/s 2024-04-02T17:18:08.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 103632 2.42 MB/sec execute 95 sec latency 4809.145 ms 2024-04-02T17:18:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 -- 172.21.15.59:0/805175150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109170 msgr2=0x7f22681095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 --2- 172.21.15.59:0/805175150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109170 0x7f22681095f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2254009900 tx=0x7f225402bc30 comp rx=0 tx=0).stop 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 -- 172.21.15.59:0/805175150 shutdown_connections 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 --2- 172.21.15.59:0/805175150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109170 0x7f22681095f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 --2- 172.21.15.59:0/805175150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268109b30 0x7f22681103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 --2- 172.21.15.59:0/805175150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268107f80 0x7f22681083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 -- 172.21.15.59:0/805175150 >> 172.21.15.59:0/805175150 conn(0x7f2268076f40 msgr2=0x7f2268079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 -- 172.21.15.59:0/805175150 shutdown_connections 2024-04-02T17:18:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.118+0000 7f226e800640 1 -- 172.21.15.59:0/805175150 wait complete. 2024-04-02T17:18:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.119+0000 7f226e800640 1 Processor -- start 2024-04-02T17:18:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.119+0000 7f226e800640 1 -- start start 2024-04-02T17:18:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268109170 0x7f226819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 0x7f22681a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2268113c10 con 0x7f2268109b30 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2268113a90 con 0x7f2268107f80 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2268113d90 con 0x7f2268109170 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 0x7f22681a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37966/0 (socket says 172.21.15.59:37966) 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 0x7f22681a57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35390/0 (socket says 172.21.15.59:35390) 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 -- 172.21.15.59:0/751505360 learned_addr learned my addr 172.21.15.59:0/751505360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226ca00640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268109170 0x7f226819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 -- 172.21.15.59:0/751505360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268109170 msgr2=0x7f226819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268109170 0x7f226819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 -- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 msgr2=0x7f22681a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 0x7f22681a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.120+0000 7f226d400640 1 -- 172.21.15.59:0/751505360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22681a5e20 con 0x7f2268107f80 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f226d400640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9c0 tx=0x7f225c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f2266000640 1 -- 172.21.15.59:0/751505360 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00cd80 con 0x7f2268107f80 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f2266000640 1 -- 172.21.15.59:0/751505360 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c0035f0 con 0x7f2268107f80 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681a6110 con 0x7f2268107f80 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f2266000640 1 -- 172.21.15.59:0/751505360 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c010690 con 0x7f2268107f80 2024-04-02T17:18:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.121+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a6630 con 0x7f2268107f80 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.124+0000 7f2266000640 1 -- 172.21.15.59:0/751505360 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f225c0037a0 con 0x7f2268107f80 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.124+0000 7f2272b76740 1 Processor -- start 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.124+0000 7f2272b76740 1 -- start start 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d758a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d758f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f3d74efd70 con 0x55f3d758aaf0 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f3d74982a0 con 0x55f3d758fc60 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f2272b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3d74ebd20 con 0x55f3d758a100 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d758f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37970/0 (socket says 172.21.15.59:37970) 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226ca00640 1 -- 172.21.15.59:0/3363074964 learned_addr learned my addr 172.21.15.59:0/3363074964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d758f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35398/0 (socket says 172.21.15.59:35398) 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.125+0000 7f226de00640 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d758a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 -- 172.21.15.59:0/3363074964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 msgr2=0x55f3d758a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d758a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 -- 172.21.15.59:0/3363074964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 msgr2=0x55f3d758f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d758f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 -- 172.21.15.59:0/3363074964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d74ac840 con 0x55f3d758fc60 2024-04-02T17:18:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f2266000640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 0x7f22500a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226ca00640 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2254009900 tx=0x7f2254009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2d13de8a339e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f2266000640 1 -- 172.21.15.59:0/751505360 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f225c0d08a0 con 0x7f2268107f80 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f226de00640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 0x7f22500a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f225a600640 1 -- 172.21.15.59:0/3363074964 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225402cb80 con 0x55f3d758fc60 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f225a600640 1 -- 172.21.15.59:0/3363074964 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254005040 con 0x55f3d758fc60 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.126+0000 7f225a600640 1 -- 172.21.15.59:0/3363074964 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254004900 con 0x55f3d758fc60 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 -- 172.21.15.59:0/3363074964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 msgr2=0x55f3d75920f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f226de00640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 0x7f22500a6060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55f3d758b350 tx=0x7f2260009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2254009900 tx=0x7f2254009560 comp rx=0 tx=0).stop 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 -- 172.21.15.59:0/3363074964 shutdown_connections 2024-04-02T17:18:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d758f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d75920f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 --2- 172.21.15.59:0/3363074964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d758a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 -- 172.21.15.59:0/3363074964 >> 172.21.15.59:0/3363074964 conn(0x55f3d7585670 msgr2=0x55f3d7585a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 -- 172.21.15.59:0/3363074964 shutdown_connections 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.127+0000 7f2272b76740 1 -- 172.21.15.59:0/3363074964 wait complete. 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 -- start start 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 Processor -- start 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f3d7592bb0 con 0x55f3d758a100 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f3d74ab660 con 0x55f3d758fc60 2024-04-02T17:18:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.128+0000 7f2272b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f3d7592d30 con 0x55f3d758aaf0 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f2272b76740 10 client.0 _ll_register_callbacks cb 0x55f3d74ab050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35408/0 (socket says 172.21.15.59:35408) 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41278/0 (socket says 172.21.15.59:41278) 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 -- 172.21.15.59:0/3017622424 learned_addr learned my addr 172.21.15.59:0/3017622424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 -- 172.21.15.59:0/3017622424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 msgr2=0x55f3d7594b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 -- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 msgr2=0x55f3d759bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 -- 172.21.15.59:0/3017622424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d74bae80 con 0x55f3d758a100 2024-04-02T17:18:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226ca00640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f226d400640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f225c0a5200 tx=0x7f225c098d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e795effeab8399 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.129+0000 7f224ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c09c420 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f225c003b30 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2254005040 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f225c09d1c0 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f2272b76740 10 client.16251 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f3d7204210 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254005040 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.130+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f225c09b980 con 0x55f3d758a100 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 10 client.16251 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 20 client.16251 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d75a0440 0x55f3d75a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 4 client.16251 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:18:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d749fcd0 con 0x55f3d75a0440 2024-04-02T17:18:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.131+0000 7f226ca00640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d75a0440 0x55f3d75a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.132+0000 7f226ca00640 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d75a0440 0x55f3d75a28d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80223154894eea7 server_cookie=f9b13060d8fd714c in_seq=0 out_seq=0 2024-04-02T17:18:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.132+0000 7f224ea00640 10 client.16251 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f224ea00640 1 -- 172.21.15.59:0/3017622424 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3d749fcd0 con 0x55f3d75a0440 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f224ea00640 10 client.16251 handle_command_reply: tid=0 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 client.16251 shutdown 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 msgr2=0x55f3d75945e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f225c0a5200 tx=0x7f225c098d60 comp rx=0 tx=0).stop 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 shutdown_connections 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d75a0440 0x55f3d75a28d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d7594b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 --2- 172.21.15.59:0/3017622424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d75945e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f224ea00640 0 client.16251 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 >> 172.21.15.59:0/3017622424 conn(0x55f3d7585670 msgr2=0x55f3d7584870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 shutdown_connections 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 1 -- 172.21.15.59:0/3017622424 wait complete. 2024-04-02T17:18:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.133+0000 7f2272b76740 20 client.16251 trim_cache size 0 max 16384 2024-04-02T17:18:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 Processor -- start 2024-04-02T17:18:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 -- start start 2024-04-02T17:18:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d7594a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d759baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22681af580 con 0x55f3d758a100 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22681b10c0 con 0x55f3d758fc60 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.156+0000 7f2272b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22681afc50 con 0x55f3d758aaf0 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37988/0 (socket says 172.21.15.59:37988) 2024-04-02T17:18:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d759baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 -- 172.21.15.59:0/128465050 learned_addr learned my addr 172.21.15.59:0/128465050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226d400640 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d7594a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 -- 172.21.15.59:0/128465050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 msgr2=0x55f3d759baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d759baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 -- 172.21.15.59:0/128465050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 msgr2=0x55f3d7594a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d7594a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226de00640 1 -- 172.21.15.59:0/128465050 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d72d0620 con 0x55f3d758fc60 2024-04-02T17:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.157+0000 7f226d400640 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d7594a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.158+0000 7f226de00640 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2260003e90 tx=0x7f226001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bfa6ad69b25014 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.158+0000 7f2259c00640 1 -- 172.21.15.59:0/128465050 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226000f5a0 con 0x55f3d758fc60 2024-04-02T17:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.158+0000 7f2259c00640 1 -- 172.21.15.59:0/128465050 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f226000fc20 con 0x55f3d758fc60 2024-04-02T17:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.158+0000 7f2259c00640 1 -- 172.21.15.59:0/128465050 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260029db0 con 0x55f3d758fc60 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- 172.21.15.59:0/128465050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 msgr2=0x55f3d7588fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2260003e90 tx=0x7f226001a000 comp rx=0 tx=0).stop 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- 172.21.15.59:0/128465050 shutdown_connections 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d7588fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758aaf0 0x55f3d759baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 --2- 172.21.15.59:0/128465050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758a100 0x55f3d7594a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- 172.21.15.59:0/128465050 >> 172.21.15.59:0/128465050 conn(0x55f3d7585670 msgr2=0x55f3d75888d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- 172.21.15.59:0/128465050 shutdown_connections 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- 172.21.15.59:0/128465050 wait complete. 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.159+0000 7f2272b76740 1 -- start start 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 Processor -- start 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d7598ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22681b09f0 con 0x55f3d758aaf0 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22681b0320 con 0x55f3d758fc60 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22681ad560 con 0x55f3d758a100 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f2272b76740 10 client.0 _ll_register_callbacks cb 0x7f225c0d6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f226ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f226d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d7598ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f226ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35424/0 (socket says 172.21.15.59:35424) 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f226de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.160+0000 7f226ca00640 1 -- 172.21.15.59:0/2142353428 learned_addr learned my addr 172.21.15.59:0/2142353428 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 -- 172.21.15.59:0/2142353428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 msgr2=0x55f3d7598ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d7598ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 -- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 msgr2=0x55f3d759a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 -- 172.21.15.59:0/2142353428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d759a9a0 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226de00640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f226ca00640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55f3d75865e0 tx=0x7f2254005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fce0e39c2d87f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.161+0000 7f224ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254004030 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f22540041d0 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226000fc20 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f22540043f0 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f2272b76740 10 client.16254 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f3d759bcc0 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.162+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f226000fc20 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f2254052160 con 0x55f3d758aaf0 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 10 client.16254 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 20 client.16254 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d7606cb0 0x55f3d76090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 4 client.16254 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3d74fd2b0 con 0x55f3d7606cb0 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.163+0000 7f226d400640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d7606cb0 0x55f3d76090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.164+0000 7f226d400640 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d7606cb0 0x55f3d76090d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c4ac3024f8ba04 server_cookie=f839770a288a994a in_seq=0 out_seq=0 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.164+0000 7f224ea00640 10 client.16254 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.188+0000 7f224ea00640 1 -- 172.21.15.59:0/2142353428 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3d74fd2b0 con 0x55f3d7606cb0 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.188+0000 7f224ea00640 10 client.16254 handle_command_reply: tid=0 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.188+0000 7f2272b76740 1 client.16254 shutdown 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 msgr2=0x55f3d7599410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55f3d75865e0 tx=0x7f2254005d50 comp rx=0 tx=0).stop 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 shutdown_connections 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f3d7606cb0 0x55f3d76090d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3d758aaf0 0x55f3d7599410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f3d758fc60 0x55f3d759a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 --2- 172.21.15.59:0/2142353428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3d758a100 0x55f3d7598ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f224ea00640 0 client.16254 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 >> 172.21.15.59:0/2142353428 conn(0x55f3d7585670 msgr2=0x55f3d7589160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 shutdown_connections 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 1 -- 172.21.15.59:0/2142353428 wait complete. 2024-04-02T17:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.189+0000 7f2272b76740 20 client.16254 trim_cache size 0 max 16384 2024-04-02T17:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 msgr2=0x7f22500a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 0x7f22500a6060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55f3d758b350 tx=0x7f2260009500 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 msgr2=0x7f226819e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e9c0 tx=0x7f225c00ee80 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 shutdown_connections 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22500a3ba0 0x7f22500a6060 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2268109b30 0x7f22681a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2268109170 0x7f226819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 --2- 172.21.15.59:0/751505360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2268107f80 0x7f226819e2a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 >> 172.21.15.59:0/751505360 conn(0x7f2268076f40 msgr2=0x7f226810c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 shutdown_connections 2024-04-02T17:18:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:08.190+0000 7f226e800640 1 -- 172.21.15.59:0/751505360 wait complete. 2024-04-02T17:18:09.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 104802 2.44 MB/sec execute 96 sec latency 4898.471 ms 2024-04-02T17:18:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:09 smithi164 ceph-mon[36499]: pgmap v540: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 578 KiB/s wr, 74 op/s 2024-04-02T17:18:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:09 smithi154 ceph-mon[36479]: pgmap v540: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 578 KiB/s wr, 74 op/s 2024-04-02T17:18:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:09 smithi059 ceph-mon[31664]: pgmap v540: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 578 KiB/s wr, 74 op/s 2024-04-02T17:18:10.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 106465 2.54 MB/sec execute 97 sec latency 15.588 ms 2024-04-02T17:18:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:10 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:18:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:10 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:18:10.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:10 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:18:11.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 106874 2.53 MB/sec execute 98 sec latency 24.685 ms 2024-04-02T17:18:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:11 smithi154 ceph-mon[36479]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 650 KiB/s wr, 79 op/s 2024-04-02T17:18:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:18:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:18:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:18:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:11 smithi059 ceph-mon[31664]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 650 KiB/s wr, 79 op/s 2024-04-02T17:18:11.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:18:11.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:18:11.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:18:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:11 smithi164 ceph-mon[36499]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 650 KiB/s wr, 79 op/s 2024-04-02T17:18:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:18:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:18:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:18:12.057 INFO:tasks.workunit.client.0.smithi059.stdout: 1 107117 2.51 MB/sec execute 99 sec latency 484.133 ms 2024-04-02T17:18:13.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 107117 2.48 MB/sec execute 100 sec latency 1484.292 ms 2024-04-02T17:18:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:13 smithi154 ceph-mon[36479]: pgmap v542: 97 pgs: 97 active+clean; 45 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 621 KiB/s wr, 79 op/s 2024-04-02T17:18:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:13 smithi059 ceph-mon[31664]: pgmap v542: 97 pgs: 97 active+clean; 45 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 621 KiB/s wr, 79 op/s 2024-04-02T17:18:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:13 smithi164 ceph-mon[36499]: pgmap v542: 97 pgs: 97 active+clean; 45 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 621 KiB/s wr, 79 op/s 2024-04-02T17:18:14.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 107875 2.51 MB/sec execute 101 sec latency 1581.797 ms 2024-04-02T17:18:15.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 109367 2.56 MB/sec execute 102 sec latency 23.881 ms 2024-04-02T17:18:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:15 smithi154 ceph-mon[36479]: pgmap v543: 97 pgs: 97 active+clean; 45 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 738 KiB/s wr, 96 op/s 2024-04-02T17:18:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:15 smithi059 ceph-mon[31664]: pgmap v543: 97 pgs: 97 active+clean; 45 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 738 KiB/s wr, 96 op/s 2024-04-02T17:18:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:15 smithi164 ceph-mon[36499]: pgmap v543: 97 pgs: 97 active+clean; 45 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 738 KiB/s wr, 96 op/s 2024-04-02T17:18:16.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 110249 2.59 MB/sec execute 103 sec latency 46.375 ms 2024-04-02T17:18:17.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 110500 2.57 MB/sec execute 104 sec latency 25.495 ms 2024-04-02T17:18:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:17 smithi154 ceph-mon[36479]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 754 KiB/s wr, 94 op/s 2024-04-02T17:18:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:17 smithi059 ceph-mon[31664]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 754 KiB/s wr, 94 op/s 2024-04-02T17:18:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:17 smithi164 ceph-mon[36499]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 754 KiB/s wr, 94 op/s 2024-04-02T17:18:18.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 110723 2.55 MB/sec execute 105 sec latency 653.959 ms 2024-04-02T17:18:19.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 111686 2.57 MB/sec execute 106 sec latency 746.438 ms 2024-04-02T17:18:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:19 smithi154 ceph-mon[36479]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 634 KiB/s wr, 94 op/s 2024-04-02T17:18:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:19 smithi059 ceph-mon[31664]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 634 KiB/s wr, 94 op/s 2024-04-02T17:18:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:19 smithi164 ceph-mon[36499]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 634 KiB/s wr, 94 op/s 2024-04-02T17:18:20.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 113167 2.62 MB/sec execute 107 sec latency 28.045 ms 2024-04-02T17:18:21.058 INFO:tasks.workunit.client.0.smithi059.stdout: 1 113851 2.65 MB/sec execute 108 sec latency 36.013 ms 2024-04-02T17:18:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.342+0000 7f0a12600640 1 -- 172.21.15.59:0/3565377318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c103d50 msgr2=0x7f0a0c104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.342+0000 7f0a12600640 1 --2- 172.21.15.59:0/3565377318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c103d50 0x7f0a0c104170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009900 tx=0x7f09fc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:18:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 -- 172.21.15.59:0/3565377318 shutdown_connections 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 --2- 172.21.15.59:0/3565377318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c104f40 0x7f0a0c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 --2- 172.21.15.59:0/3565377318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c105900 0x7f0a0c10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 --2- 172.21.15.59:0/3565377318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c103d50 0x7f0a0c104170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 -- 172.21.15.59:0/3565377318 >> 172.21.15.59:0/3565377318 conn(0x7f0a0c0ff2c0 msgr2=0x7f0a0c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 -- 172.21.15.59:0/3565377318 shutdown_connections 2024-04-02T17:18:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.343+0000 7f0a12600640 1 -- 172.21.15.59:0/3565377318 wait complete. 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.344+0000 7f0a12600640 1 Processor -- start 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.344+0000 7f0a12600640 1 -- start start 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c103d50 0x7f0a0c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 0x7f0a0c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10f9f0 con 0x7f0a0c103d50 2024-04-02T17:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10f870 con 0x7f0a0c105900 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a12600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10fb70 con 0x7f0a0c104f40 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a0be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c103d50 0x7f0a0c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a0be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57488/0 (socket says 172.21.15.59:57488) 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a0be00640 1 -- 172.21.15.59:0/2248393003 learned_addr learned my addr 172.21.15.59:0/2248393003 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.345+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 0x7f0a0c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 -- 172.21.15.59:0/2248393003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 msgr2=0x7f0a0c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 -- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c103d50 msgr2=0x7f0a0c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c103d50 0x7f0a0c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 -- 172.21.15.59:0/2248393003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c1a1bc0 con 0x7f0a0c104f40 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0be00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 0x7f0a0c19a4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e9f0 tx=0x7f09f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a03e00640 1 -- 172.21.15.59:0/2248393003 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f800ccc0 con 0x7f0a0c104f40 2024-04-02T17:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a03e00640 1 -- 172.21.15.59:0/2248393003 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f8010910 con 0x7f0a0c104f40 2024-04-02T17:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.346+0000 7f0a03e00640 1 -- 172.21.15.59:0/2248393003 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8018960 con 0x7f0a0c104f40 2024-04-02T17:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.347+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c1a1eb0 con 0x7f0a0c104f40 2024-04-02T17:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.347+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c1a2320 con 0x7f0a0c104f40 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.350+0000 7f0a15398740 1 Processor -- start 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.350+0000 7f0a15398740 1 -- start start 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb7364fd70 con 0x55eb736e9ed0 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eb735f82a0 con 0x55eb736ea8c0 2024-04-02T17:18:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a15398740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb7364bd20 con 0x55eb736efa30 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a03e00640 1 -- 172.21.15.59:0/2248393003 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09f8010ac0 con 0x7f0a0c104f40 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57492/0 (socket says 172.21.15.59:57492) 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 -- 172.21.15.59:0/524730964 learned_addr learned my addr 172.21.15.59:0/524730964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0be00640 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 -- 172.21.15.59:0/524730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 msgr2=0x55eb736f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 -- 172.21.15.59:0/524730964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 msgr2=0x55eb736ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.351+0000 7f0a0b400640 1 -- 172.21.15.59:0/524730964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7360c840 con 0x55eb736ea8c0 2024-04-02T17:18:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.352+0000 7f0a0b400640 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=151c37678a8251b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.352+0000 7f0a00c00640 1 -- 172.21.15.59:0/524730964 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc045070 con 0x55eb736ea8c0 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.352+0000 7f0a00c00640 1 -- 172.21.15.59:0/524730964 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc004040 con 0x55eb736ea8c0 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.352+0000 7f0a00c00640 1 -- 172.21.15.59:0/524730964 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004e90 con 0x55eb736ea8c0 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 -- 172.21.15.59:0/524730964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 msgr2=0x55eb736ef3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a03e00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc009560 comp rx=0 tx=0).stop 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a03e00640 1 -- 172.21.15.59:0/2248393003 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f09f80d0730 con 0x7f0a0c104f40 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 -- 172.21.15.59:0/524730964 shutdown_connections 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736ea8c0 0x55eb736ef3c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 --2- 172.21.15.59:0/524730964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a0be00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.353+0000 7f0a15398740 1 -- 172.21.15.59:0/524730964 >> 172.21.15.59:0/524730964 conn(0x55eb736e5440 msgr2=0x55eb736e5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.354+0000 7f0a15398740 1 -- 172.21.15.59:0/524730964 shutdown_connections 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.354+0000 7f0a15398740 1 -- 172.21.15.59:0/524730964 wait complete. 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.354+0000 7f0a15398740 1 -- start start 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a0be00640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 0x7f09f00a6190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55eb736eb120 tx=0x7f0a04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 Processor -- start 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736ea8c0 0x55eb736f4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb736f2980 con 0x55eb736e9ed0 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eb7360b660 con 0x55eb736efa30 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb736f2b00 con 0x55eb736ea8c0 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a15398740 10 client.0 _ll_register_callbacks cb 0x55eb7360b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.355+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38214/0 (socket says 172.21.15.59:38214) 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 -- 172.21.15.59:0/742028111 learned_addr learned my addr 172.21.15.59:0/742028111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736ea8c0 0x55eb736f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0be00640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 -- 172.21.15.59:0/742028111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736ea8c0 msgr2=0x55eb736f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736ea8c0 0x55eb736f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 -- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 msgr2=0x55eb736fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 -- 172.21.15.59:0/742028111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7361ae80 con 0x55eb736e9ed0 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f0a0b400640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f09fc036040 tx=0x7f09fc02cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64926bf1ffcbef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.356+0000 7f09f5600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc04e070 con 0x55eb736e9ed0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f09fc003a00 con 0x55eb736e9ed0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09fc004040 con 0x55eb736e9ed0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f09fc059050 con 0x55eb736e9ed0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f0a15398740 10 client.16266 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb73364210 con 0x55eb736e9ed0 2024-04-02T17:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.357+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc004040 con 0x55eb736e9ed0 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f09fc03f440 con 0x55eb736e9ed0 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 10 client.16266 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 20 client.16266 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb736ffe50 0x55eb73702240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 4 client.16266 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb735ffcd0 con 0x55eb736ffe50 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.358+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb736ffe50 0x55eb73702240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.359+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb736ffe50 0x55eb73702240 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80f228ed311a244 server_cookie=9cb3f732e386fa06 in_seq=0 out_seq=0 2024-04-02T17:18:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.359+0000 7f09f5600640 10 client.16266 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f09f5600640 1 -- 172.21.15.59:0/742028111 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb735ffcd0 con 0x55eb736ffe50 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f09f5600640 10 client.16266 handle_command_reply: tid=0 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 client.16266 shutdown 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 msgr2=0x55eb736f43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f09fc036040 tx=0x7f09fc02cbb0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 shutdown_connections 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb736ffe50 0x55eb73702240 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736ea8c0 0x55eb736f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 --2- 172.21.15.59:0/742028111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736e9ed0 0x55eb736f43e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f09f5600640 0 client.16266 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 >> 172.21.15.59:0/742028111 conn(0x55eb736e5440 msgr2=0x55eb736e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.360+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 shutdown_connections 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.361+0000 7f0a15398740 1 -- 172.21.15.59:0/742028111 wait complete. 2024-04-02T17:18:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.361+0000 7f0a15398740 20 client.16266 trim_cache size 0 max 16384 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.384+0000 7f0a15398740 1 Processor -- start 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 -- start start 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1ab2c0 con 0x55eb736ea8c0 2024-04-02T17:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1ace00 con 0x55eb736efa30 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a15398740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1ab990 con 0x55eb736e9ed0 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a0aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41766/0 (socket says 172.21.15.59:41766) 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a0b400640 1 -- 172.21.15.59:0/1692500360 learned_addr learned my addr 172.21.15.59:0/1692500360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.385+0000 7f0a0be00640 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 -- 172.21.15.59:0/1692500360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 msgr2=0x55eb736e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 -- 172.21.15.59:0/1692500360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 msgr2=0x55eb736fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 -- 172.21.15.59:0/1692500360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb73430620 con 0x55eb736e9ed0 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f0a0b400640 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c315322c3cf0ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f09f7e00640 1 -- 172.21.15.59:0/1692500360 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc04e040 con 0x55eb736e9ed0 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f09f7e00640 1 -- 172.21.15.59:0/1692500360 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc03fa00 con 0x55eb736e9ed0 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.386+0000 7f09f7e00640 1 -- 172.21.15.59:0/1692500360 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc045040 con 0x55eb736e9ed0 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 -- 172.21.15.59:0/1692500360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 msgr2=0x55eb736f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc009650 comp rx=0 tx=0).stop 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 -- 172.21.15.59:0/1692500360 shutdown_connections 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736efa30 0x55eb736e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 --2- 172.21.15.59:0/1692500360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736e9ed0 0x55eb736f46c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 -- 172.21.15.59:0/1692500360 >> 172.21.15.59:0/1692500360 conn(0x55eb736e5440 msgr2=0x55eb736e3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.387+0000 7f0a15398740 1 -- 172.21.15.59:0/1692500360 shutdown_connections 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.388+0000 7f0a15398740 1 -- 172.21.15.59:0/1692500360 wait complete. 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.388+0000 7f0a15398740 1 -- start start 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.388+0000 7f0a15398740 1 Processor -- start 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736f9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736fa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1ac730 con 0x55eb736ea8c0 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1ac060 con 0x55eb736e9ed0 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1a92a0 con 0x55eb736efa30 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a15398740 10 client.0 _ll_register_callbacks cb 0x7f09fc005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57520/0 (socket says 172.21.15.59:57520) 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 -- 172.21.15.59:0/4122521035 learned_addr learned my addr 172.21.15.59:0/4122521035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0be00640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 -- 172.21.15.59:0/4122521035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 msgr2=0x55eb736fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 -- 172.21.15.59:0/4122521035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 msgr2=0x55eb736f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.389+0000 7f0a0b400640 1 -- 172.21.15.59:0/4122521035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb736fa670 con 0x55eb736e9ed0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f0a0b400640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02c170 tx=0x7f09fc02c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9dcf5c70cd70 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f09f5600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc062060 con 0x55eb736e9ed0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09fc03fa00 con 0x55eb736e9ed0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f09fc003a00 con 0x55eb736e9ed0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f09fc04e070 con 0x55eb736e9ed0 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f0a15398740 10 client.26366 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.390+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb736fb980 con 0x55eb736e9ed0 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc03fa00 con 0x55eb736e9ed0 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f09fc066030 con 0x55eb736e9ed0 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 10 client.26366 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 20 client.26366 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb73766670 0x55eb73768a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 4 client.26366 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb7365d2b0 con 0x55eb73766670 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.391+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb73766670 0x55eb73768a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.392+0000 7f0a0aa00640 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb73766670 0x55eb73768a60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297f12f9057194d7 server_cookie=dd02817c2aff96ae in_seq=0 out_seq=0 2024-04-02T17:18:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.392+0000 7f09f5600640 10 client.26366 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:21 smithi154 ceph-mon[36479]: pgmap v546: 97 pgs: 97 active+clean; 46 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 714 KiB/s wr, 119 op/s 2024-04-02T17:18:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f09f5600640 1 -- 172.21.15.59:0/4122521035 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb7365d2b0 con 0x55eb73766670 2024-04-02T17:18:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f09f5600640 10 client.26366 handle_command_reply: tid=0 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 client.26366 shutdown 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 msgr2=0x55eb736f8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02c170 tx=0x7f09fc02c2b0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 shutdown_connections 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55eb73766670 0x55eb73768a60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb736ea8c0 0x55eb736f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb736efa30 0x55eb736fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 --2- 172.21.15.59:0/4122521035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eb736e9ed0 0x55eb736f8b30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f09f5600640 0 client.26366 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.424+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 >> 172.21.15.59:0/4122521035 conn(0x55eb736e5440 msgr2=0x55eb736e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.425+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 shutdown_connections 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.425+0000 7f0a15398740 1 -- 172.21.15.59:0/4122521035 wait complete. 2024-04-02T17:18:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.425+0000 7f0a15398740 20 client.26366 trim_cache size 0 max 16384 2024-04-02T17:18:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:18:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:18:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:18:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 msgr2=0x7f09f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 0x7f09f00a6190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55eb736eb120 tx=0x7f0a04009500 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 msgr2=0x7f0a0c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 0x7f0a0c19a4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e9f0 tx=0x7f09f800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 shutdown_connections 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a0c105900 0x7f0a0c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a0c104f40 0x7f0a0c19a4e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 --2- 172.21.15.59:0/2248393003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a0c103d50 0x7f0a0c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 >> 172.21.15.59:0/2248393003 conn(0x7f0a0c0ff2c0 msgr2=0x7f0a0c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 shutdown_connections 2024-04-02T17:18:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:21.426+0000 7f0a12600640 1 -- 172.21.15.59:0/2248393003 wait complete. 2024-04-02T17:18:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:21 smithi059 ceph-mon[31664]: pgmap v546: 97 pgs: 97 active+clean; 46 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 714 KiB/s wr, 119 op/s 2024-04-02T17:18:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:21 smithi164 ceph-mon[36499]: pgmap v546: 97 pgs: 97 active+clean; 46 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 714 KiB/s wr, 119 op/s 2024-04-02T17:18:22.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114210 2.63 MB/sec execute 109 sec latency 175.798 ms 2024-04-02T17:18:23.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114210 2.61 MB/sec execute 110 sec latency 1175.925 ms 2024-04-02T17:18:23.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:23 smithi154 ceph-mon[36479]: pgmap v547: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 704 KiB/s wr, 121 op/s 2024-04-02T17:18:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:23 smithi059 ceph-mon[31664]: pgmap v547: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 704 KiB/s wr, 121 op/s 2024-04-02T17:18:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:23 smithi164 ceph-mon[36499]: pgmap v547: 97 pgs: 97 active+clean; 46 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 704 KiB/s wr, 121 op/s 2024-04-02T17:18:24.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114334 2.59 MB/sec execute 111 sec latency 1271.921 ms 2024-04-02T17:18:25.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114334 2.56 MB/sec execute 112 sec latency 1682.029 ms 2024-04-02T17:18:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:25 smithi154 ceph-mon[36479]: pgmap v548: 97 pgs: 97 active+clean; 42 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 901 KiB/s wr, 128 op/s 2024-04-02T17:18:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:25 smithi059 ceph-mon[31664]: pgmap v548: 97 pgs: 97 active+clean; 42 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 901 KiB/s wr, 128 op/s 2024-04-02T17:18:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:25 smithi164 ceph-mon[36499]: pgmap v548: 97 pgs: 97 active+clean; 42 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 901 KiB/s wr, 128 op/s 2024-04-02T17:18:26.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114334 2.54 MB/sec execute 113 sec latency 2682.122 ms 2024-04-02T17:18:27.059 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114334 2.52 MB/sec execute 114 sec latency 3682.222 ms 2024-04-02T17:18:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:27 smithi154 ceph-mon[36479]: pgmap v549: 97 pgs: 97 active+clean; 43 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 924 KiB/s wr, 133 op/s 2024-04-02T17:18:27.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:27 smithi059 ceph-mon[31664]: pgmap v549: 97 pgs: 97 active+clean; 43 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 924 KiB/s wr, 133 op/s 2024-04-02T17:18:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:27 smithi164 ceph-mon[36499]: pgmap v549: 97 pgs: 97 active+clean; 43 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 924 KiB/s wr, 133 op/s 2024-04-02T17:18:28.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 114334 2.50 MB/sec execute 115 sec latency 4682.380 ms 2024-04-02T17:18:29.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 115436 2.52 MB/sec execute 116 sec latency 4778.931 ms 2024-04-02T17:18:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:29 smithi154 ceph-mon[36479]: pgmap v550: 97 pgs: 97 active+clean; 43 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 856 KiB/s wr, 114 op/s 2024-04-02T17:18:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:29 smithi059 ceph-mon[31664]: pgmap v550: 97 pgs: 97 active+clean; 43 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 856 KiB/s wr, 114 op/s 2024-04-02T17:18:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:29 smithi164 ceph-mon[36499]: pgmap v550: 97 pgs: 97 active+clean; 43 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 856 KiB/s wr, 114 op/s 2024-04-02T17:18:30.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 116729 2.56 MB/sec execute 117 sec latency 33.392 ms 2024-04-02T17:18:31.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 117399 2.59 MB/sec execute 118 sec latency 26.333 ms 2024-04-02T17:18:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:31 smithi154 ceph-mon[36479]: pgmap v551: 97 pgs: 97 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 888 KiB/s wr, 117 op/s 2024-04-02T17:18:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:31 smithi059 ceph-mon[31664]: pgmap v551: 97 pgs: 97 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 888 KiB/s wr, 117 op/s 2024-04-02T17:18:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:31 smithi164 ceph-mon[36499]: pgmap v551: 97 pgs: 97 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 888 KiB/s wr, 117 op/s 2024-04-02T17:18:32.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 117709 2.57 MB/sec execute 119 sec latency 29.259 ms 2024-04-02T17:18:33.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 117756 2.55 MB/sec execute 120 sec latency 961.982 ms 2024-04-02T17:18:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:33 smithi154 ceph-mon[36479]: pgmap v552: 97 pgs: 97 active+clean; 43 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 828 KiB/s wr, 94 op/s 2024-04-02T17:18:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:33 smithi059 ceph-mon[31664]: pgmap v552: 97 pgs: 97 active+clean; 43 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 828 KiB/s wr, 94 op/s 2024-04-02T17:18:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:33 smithi164 ceph-mon[36499]: pgmap v552: 97 pgs: 97 active+clean; 43 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 828 KiB/s wr, 94 op/s 2024-04-02T17:18:34.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 118578 2.57 MB/sec execute 121 sec latency 1058.803 ms 2024-04-02T17:18:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 -- 172.21.15.59:0/3121761659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 msgr2=0x7f71841053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 --2- 172.21.15.59:0/3121761659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f71841053d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f716c009900 tx=0x7f716c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 -- 172.21.15.59:0/3121761659 shutdown_connections 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 --2- 172.21.15.59:0/3121761659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f718410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 --2- 172.21.15.59:0/3121761659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f71841053d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 --2- 172.21.15.59:0/3121761659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 0x7f7184104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 -- 172.21.15.59:0/3121761659 >> 172.21.15.59:0/3121761659 conn(0x7f71840ff2f0 msgr2=0x7f7184101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 -- 172.21.15.59:0/3121761659 shutdown_connections 2024-04-02T17:18:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.591+0000 7f718b600640 1 -- 172.21.15.59:0/3121761659 wait complete. 2024-04-02T17:18:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.592+0000 7f718b600640 1 Processor -- start 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.592+0000 7f718b600640 1 -- start start 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 0x7f718419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f71841a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f718410fa10 con 0x7f7184105910 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f718410f890 con 0x7f7184103d60 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f718b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f718410fb90 con 0x7f7184104f50 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f7188c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 0x7f718419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36264/0 (socket says 172.21.15.59:36264) 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 -- 172.21.15.59:0/1880249061 learned_addr learned my addr 172.21.15.59:0/1880249061 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f7189600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f71841a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 -- 172.21.15.59:0/1880249061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 msgr2=0x7f718419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 0x7f718419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 -- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 msgr2=0x7f71841a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f71841a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f717be00640 1 -- 172.21.15.59:0/1880249061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71841a5fc0 con 0x7f7184104f50 2024-04-02T17:18:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.593+0000 7f7189600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f71841a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f717be00640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f716c005ab0 tx=0x7f716c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f7182000640 1 -- 172.21.15.59:0/1880249061 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c043020 con 0x7f7184104f50 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71841a6250 con 0x7f7184104f50 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f7182000640 1 -- 172.21.15.59:0/1880249061 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f716c0054e0 con 0x7f7184104f50 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f7182000640 1 -- 172.21.15.59:0/1880249061 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c003ce0 con 0x7f7184104f50 2024-04-02T17:18:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.594+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71841a67f0 con 0x7f7184104f50 2024-04-02T17:18:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f7182000640 1 -- 172.21.15.59:0/1880249061 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f716c046020 con 0x7f7184104f50 2024-04-02T17:18:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 Processor -- start 2024-04-02T17:18:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 -- start start 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 0x5654fa2ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654fa20ed70 con 0x5654fa2aea30 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654fa1b72a0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.598+0000 7f718e419740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654fa20ad20 con 0x5654fa2a98c0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7189600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 0x5654fa2ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7189600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52336/0 (socket says 172.21.15.59:52336) 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7189600640 1 -- 172.21.15.59:0/2147166670 learned_addr learned my addr 172.21.15.59:0/2147166670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f717be00640 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 -- 172.21.15.59:0/2147166670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 msgr2=0x5654fa2a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 -- 172.21.15.59:0/2147166670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 msgr2=0x5654fa2b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 -- 172.21.15.59:0/2147166670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654fa1cb840 con 0x5654fa2a98c0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7189600640 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.599+0000 7f7188c00640 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 0x5654fa2ae3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7174005fd0 tx=0x7f71740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd9136909b16afa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7182000640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 0x7f71680a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7179c00640 1 -- 172.21.15.59:0/2147166670 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174004d20 con 0x5654fa2a98c0 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7182000640 1 -- 172.21.15.59:0/1880249061 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f716c0f2630 con 0x7f7184104f50 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7179c00640 1 -- 172.21.15.59:0/2147166670 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174003e50 con 0x5654fa2a98c0 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7179c00640 1 -- 172.21.15.59:0/2147166670 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400f7a0 con 0x5654fa2a98c0 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.600+0000 7f7189600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 0x7f71680a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- 172.21.15.59:0/2147166670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 msgr2=0x5654fa2ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 0x5654fa2ae3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7174005fd0 tx=0x7f71740118d0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- 172.21.15.59:0/2147166670 shutdown_connections 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a98c0 0x5654fa2ae3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 --2- 172.21.15.59:0/2147166670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- 172.21.15.59:0/2147166670 >> 172.21.15.59:0/2147166670 conn(0x5654fa2a4440 msgr2=0x5654fa2a4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f7189600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 0x7f71680a6110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5654fa2aa120 tx=0x7f717c00b5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- 172.21.15.59:0/2147166670 shutdown_connections 2024-04-02T17:18:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- 172.21.15.59:0/2147166670 wait complete. 2024-04-02T17:18:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.601+0000 7f718e419740 1 -- start start 2024-04-02T17:18:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 Processor -- start 2024-04-02T17:18:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2aea30 0x5654fa2ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654fa2b19e0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654fa1ca660 con 0x5654fa2a98c0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654fa2b1b60 con 0x5654fa2aea30 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f718e419740 10 client.0 _ll_register_callbacks cb 0x5654fa1ca050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f717be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f7188c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f7189600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2aea30 0x5654fa2ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f717be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52352/0 (socket says 172.21.15.59:52352) 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.602+0000 7f717be00640 1 -- 172.21.15.59:0/4253254452 learned_addr learned my addr 172.21.15.59:0/4253254452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 -- 172.21.15.59:0/4253254452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2aea30 msgr2=0x5654fa2ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2aea30 0x5654fa2ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 -- 172.21.15.59:0/4253254452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a8ed0 msgr2=0x5654fa2b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 -- 172.21.15.59:0/4253254452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654fa1d9e80 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f717be00640 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f716c0bb220 tx=0x7f716c009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6108abf17dd9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f7172000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0be1b0 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f716c0f0180 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.603+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7174003e50 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f716c004c80 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 10 client.26378 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5654f9f23210 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174003e50 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f716c0d4030 con 0x5654fa2a98c0 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 10 client.26378 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 20 client.26378 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa2bf270 0x5654fa2c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 4 client.26378 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:18:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.604+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654fa1becd0 con 0x5654fa2bf270 2024-04-02T17:18:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.605+0000 7f7188c00640 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa2bf270 0x5654fa2c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.605+0000 7f7188c00640 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa2bf270 0x5654fa2c16a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2740c288fc45022a server_cookie=50b44dec97ffe53e in_seq=0 out_seq=0 2024-04-02T17:18:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.605+0000 7f7172000640 10 client.26378 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f7172000640 1 -- 172.21.15.59:0/4253254452 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654fa1becd0 con 0x5654fa2bf270 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f7172000640 10 client.26378 handle_command_reply: tid=0 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 client.26378 shutdown 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 msgr2=0x5654fa2b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f716c0bb220 tx=0x7f716c009620 comp rx=0 tx=0).stop 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 shutdown_connections 2024-04-02T17:18:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa2bf270 0x5654fa2c16a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2aea30 0x5654fa2ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b39b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 --2- 172.21.15.59:0/4253254452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f7172000640 0 client.26378 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 >> 172.21.15.59:0/4253254452 conn(0x5654fa2a4440 msgr2=0x5654fa2a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 shutdown_connections 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.607+0000 7f718e419740 1 -- 172.21.15.59:0/4253254452 wait complete. 2024-04-02T17:18:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.608+0000 7f718e419740 20 client.26378 trim_cache size 0 max 16384 2024-04-02T17:18:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.627+0000 7f718e419740 1 Processor -- start 2024-04-02T17:18:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.627+0000 7f718e419740 1 -- start start 2024-04-02T17:18:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.627+0000 7f718e419740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.627+0000 7f718e419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a98c0 0x5654fa2a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f718e419740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2aea30 0x5654fa2a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f718e419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71841af610 con 0x5654fa2a98c0 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f718e419740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71841b1150 con 0x5654fa2aea30 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f718e419740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71841afce0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f717be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a98c0 0x5654fa2a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36294/0 (socket says 172.21.15.59:36294) 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 -- 172.21.15.59:0/2450089433 learned_addr learned my addr 172.21.15.59:0/2450089433 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7189600640 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2aea30 0x5654fa2a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 -- 172.21.15.59:0/2450089433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2aea30 msgr2=0x5654fa2a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2aea30 0x5654fa2a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 -- 172.21.15.59:0/2450089433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a98c0 msgr2=0x5654fa2a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a98c0 0x5654fa2a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.628+0000 7f7188c00640 1 -- 172.21.15.59:0/2450089433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f9fef620 con 0x5654fa2a8ed0 2024-04-02T17:18:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.629+0000 7f7188c00640 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7174011dd0 tx=0x7f7174004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752f9e89bde63257 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.629+0000 7f7179200640 1 -- 172.21.15.59:0/2450089433 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717401d040 con 0x5654fa2a8ed0 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.629+0000 7f7179200640 1 -- 172.21.15.59:0/2450089433 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174004730 con 0x5654fa2a8ed0 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.629+0000 7f7179200640 1 -- 172.21.15.59:0/2450089433 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174005050 con 0x5654fa2a8ed0 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 -- 172.21.15.59:0/2450089433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 msgr2=0x5654fa2a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7174011dd0 tx=0x7f7174004340 comp rx=0 tx=0).stop 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 -- 172.21.15.59:0/2450089433 shutdown_connections 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2a98c0 0x5654fa2a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2aea30 0x5654fa2a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 --2- 172.21.15.59:0/2450089433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2a2db0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 -- 172.21.15.59:0/2450089433 >> 172.21.15.59:0/2450089433 conn(0x5654fa2a4440 msgr2=0x5654fa2b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 -- 172.21.15.59:0/2450089433 shutdown_connections 2024-04-02T17:18:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.630+0000 7f718e419740 1 -- 172.21.15.59:0/2450089433 wait complete. 2024-04-02T17:18:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 -- start start 2024-04-02T17:18:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 Processor -- start 2024-04-02T17:18:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71841b0a80 con 0x5654fa2aea30 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71841b03b0 con 0x5654fa2a98c0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71841ad5f0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f718e419740 10 client.0 _ll_register_callbacks cb 0x7f716c009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.631+0000 7f7188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36302/0 (socket says 172.21.15.59:36302) 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 -- 172.21.15.59:0/4069437155 learned_addr learned my addr 172.21.15.59:0/4069437155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f717be00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7189600640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 -- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 msgr2=0x5654fa2b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 -- 172.21.15.59:0/4069437155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 msgr2=0x5654fa2b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 -- 172.21.15.59:0/4069437155 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654fa2b90a0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7189600640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f717be00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7188c00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7174006010 tx=0x7f7174009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd75096647db19c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7172000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.632+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717403a060 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7174004730 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f7174003970 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f71740049e0 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 10 client.26401 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5654fa325820 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174004730 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f717403e050 con 0x5654fa2a8ed0 2024-04-02T17:18:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 10 client.26401 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 20 client.26401 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.633+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa325ed0 0x5654fa3282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.634+0000 7f718e419740 4 client.26401 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.634+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654fa21c2b0 con 0x5654fa325ed0 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.634+0000 7f717be00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa325ed0 0x5654fa3282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.634+0000 7f7172000640 10 client.26401 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.634+0000 7f717be00640 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa325ed0 0x5654fa3282f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5b9f65930fc4e7 server_cookie=efaad659331548b3 in_seq=0 out_seq=0 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.666+0000 7f7172000640 1 -- 172.21.15.59:0/4069437155 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654fa21c2b0 con 0x5654fa325ed0 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.666+0000 7f7172000640 10 client.26401 handle_command_reply: tid=0 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.666+0000 7f718e419740 1 client.26401 shutdown 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 msgr2=0x5654fa2b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7174006010 tx=0x7f7174009740 comp rx=0 tx=0).stop 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 shutdown_connections 2024-04-02T17:18:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654fa325ed0 0x5654fa3282f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654fa2aea30 0x5654fa2b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654fa2a98c0 0x5654fa2b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 --2- 172.21.15.59:0/4069437155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654fa2a8ed0 0x5654fa2b7560 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f7172000640 0 client.26401 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 >> 172.21.15.59:0/4069437155 conn(0x5654fa2a4440 msgr2=0x5654fa2a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 shutdown_connections 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 1 -- 172.21.15.59:0/4069437155 wait complete. 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.667+0000 7f718e419740 20 client.26401 trim_cache size 0 max 16384 2024-04-02T17:18:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:18:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:18:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:18:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:18:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 msgr2=0x7f71680a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 0x7f71680a6110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5654fa2aa120 tx=0x7f717c00b5c0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 msgr2=0x7f718419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f716c005ab0 tx=0x7f716c02c6c0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 shutdown_connections 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f71680a3c50 0x7f71680a6110 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7184105910 0x7f71841a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7184104f50 0x7f718419e8e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 --2- 172.21.15.59:0/1880249061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7184103d60 0x7f718419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 >> 172.21.15.59:0/1880249061 conn(0x7f71840ff2f0 msgr2=0x7f7184077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 shutdown_connections 2024-04-02T17:18:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:34.668+0000 7f718b600640 1 -- 172.21.15.59:0/1880249061 wait complete. 2024-04-02T17:18:35.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 120085 2.61 MB/sec execute 122 sec latency 23.306 ms 2024-04-02T17:18:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:35 smithi154 ceph-mon[36479]: pgmap v553: 97 pgs: 97 active+clean; 44 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 818 KiB/s wr, 91 op/s 2024-04-02T17:18:35.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:35 smithi059 ceph-mon[31664]: pgmap v553: 97 pgs: 97 active+clean; 44 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 818 KiB/s wr, 91 op/s 2024-04-02T17:18:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:35 smithi164 ceph-mon[36499]: pgmap v553: 97 pgs: 97 active+clean; 44 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 818 KiB/s wr, 91 op/s 2024-04-02T17:18:36.060 INFO:tasks.workunit.client.0.smithi059.stdout: 1 120917 2.64 MB/sec execute 123 sec latency 30.117 ms 2024-04-02T17:18:37.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121212 2.62 MB/sec execute 124 sec latency 25.441 ms 2024-04-02T17:18:37.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:37 smithi154 ceph-mon[36479]: pgmap v554: 97 pgs: 97 active+clean; 44 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 662 KiB/s wr, 107 op/s 2024-04-02T17:18:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:37 smithi059 ceph-mon[31664]: pgmap v554: 97 pgs: 97 active+clean; 44 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 662 KiB/s wr, 107 op/s 2024-04-02T17:18:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:37 smithi164 ceph-mon[36499]: pgmap v554: 97 pgs: 97 active+clean; 44 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 662 KiB/s wr, 107 op/s 2024-04-02T17:18:38.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121300 2.60 MB/sec execute 125 sec latency 900.147 ms 2024-04-02T17:18:39.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121356 2.58 MB/sec execute 126 sec latency 994.792 ms 2024-04-02T17:18:39.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:39 smithi154 ceph-mon[36479]: pgmap v555: 97 pgs: 97 active+clean; 45 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 547 KiB/s wr, 87 op/s 2024-04-02T17:18:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:39 smithi059 ceph-mon[31664]: pgmap v555: 97 pgs: 97 active+clean; 45 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 547 KiB/s wr, 87 op/s 2024-04-02T17:18:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:39 smithi164 ceph-mon[36499]: pgmap v555: 97 pgs: 97 active+clean; 45 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 547 KiB/s wr, 87 op/s 2024-04-02T17:18:40.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121356 2.56 MB/sec execute 127 sec latency 1802.683 ms 2024-04-02T17:18:41.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121356 2.54 MB/sec execute 128 sec latency 2802.825 ms 2024-04-02T17:18:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:41 smithi059 ceph-mon[31664]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 635 KiB/s wr, 110 op/s 2024-04-02T17:18:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:41 smithi164 ceph-mon[36499]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 635 KiB/s wr, 110 op/s 2024-04-02T17:18:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:41 smithi154 ceph-mon[36479]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 635 KiB/s wr, 110 op/s 2024-04-02T17:18:42.061 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121356 2.52 MB/sec execute 129 sec latency 3802.981 ms 2024-04-02T17:18:43.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 121356 2.50 MB/sec execute 130 sec latency 4803.090 ms 2024-04-02T17:18:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:43 smithi059 ceph-mon[31664]: pgmap v557: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 580 KiB/s wr, 103 op/s 2024-04-02T17:18:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:43 smithi164 ceph-mon[36499]: pgmap v557: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 580 KiB/s wr, 103 op/s 2024-04-02T17:18:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:43 smithi154 ceph-mon[36479]: pgmap v557: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 580 KiB/s wr, 103 op/s 2024-04-02T17:18:44.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 122163 2.52 MB/sec execute 131 sec latency 4901.219 ms 2024-04-02T17:18:45.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 123612 2.56 MB/sec execute 132 sec latency 19.391 ms 2024-04-02T17:18:45.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:45 smithi059 ceph-mon[31664]: pgmap v558: 97 pgs: 97 active+clean; 46 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 619 KiB/s wr, 104 op/s 2024-04-02T17:18:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:45 smithi164 ceph-mon[36499]: pgmap v558: 97 pgs: 97 active+clean; 46 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 619 KiB/s wr, 104 op/s 2024-04-02T17:18:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:45 smithi154 ceph-mon[36479]: pgmap v558: 97 pgs: 97 active+clean; 46 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 619 KiB/s wr, 104 op/s 2024-04-02T17:18:46.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124453 2.58 MB/sec execute 133 sec latency 37.391 ms 2024-04-02T17:18:47.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124736 2.57 MB/sec execute 134 sec latency 27.061 ms 2024-04-02T17:18:47.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:47 smithi059 ceph-mon[31664]: pgmap v559: 97 pgs: 97 active+clean; 47 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 608 KiB/s wr, 108 op/s 2024-04-02T17:18:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:47 smithi164 ceph-mon[36499]: pgmap v559: 97 pgs: 97 active+clean; 47 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 608 KiB/s wr, 108 op/s 2024-04-02T17:18:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:47 smithi154 ceph-mon[36479]: pgmap v559: 97 pgs: 97 active+clean; 47 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 608 KiB/s wr, 108 op/s 2024-04-02T17:18:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 -- 172.21.15.59:0/814459845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109170 msgr2=0x7fb1941095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 --2- 172.21.15.59:0/814459845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109170 0x7fb1941095f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002bc30 comp rx=0 tx=0).stop 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 -- 172.21.15.59:0/814459845 shutdown_connections 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 --2- 172.21.15.59:0/814459845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194109b30 0x7fb1941103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 --2- 172.21.15.59:0/814459845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109170 0x7fb1941095f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 --2- 172.21.15.59:0/814459845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194107f80 0x7fb1941083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 -- 172.21.15.59:0/814459845 >> 172.21.15.59:0/814459845 conn(0x7fb194076f40 msgr2=0x7fb194079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 -- 172.21.15.59:0/814459845 shutdown_connections 2024-04-02T17:18:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.817+0000 7fb19b000640 1 -- 172.21.15.59:0/814459845 wait complete. 2024-04-02T17:18:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.818+0000 7fb19b000640 1 Processor -- start 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.818+0000 7fb19b000640 1 -- start start 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194107f80 0x7fb19419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109b30 0x7fb1941a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb194113c60 con 0x7fb194107f80 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb194113ae0 con 0x7fb194109b30 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19b000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb194113de0 con 0x7fb194109170 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47682/0 (socket says 172.21.15.59:47682) 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 -- 172.21.15.59:0/182388329 learned_addr learned my addr 172.21.15.59:0/182388329 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb19a600640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109b30 0x7fb1941a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199c00640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194107f80 0x7fb19419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 -- 172.21.15.59:0/182388329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109b30 msgr2=0x7fb1941a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109b30 0x7fb1941a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 -- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194107f80 msgr2=0x7fb19419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194107f80 0x7fb19419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.819+0000 7fb199200640 1 -- 172.21.15.59:0/182388329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1941a5eb0 con 0x7fb194109170 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb199200640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb180005ab0 tx=0x7fb18002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb192000640 1 -- 172.21.15.59:0/182388329 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180043020 con 0x7fb194109170 2024-04-02T17:18:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb192000640 1 -- 172.21.15.59:0/182388329 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1800054e0 con 0x7fb194109170 2024-04-02T17:18:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1941a6140 con 0x7fb194109170 2024-04-02T17:18:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb192000640 1 -- 172.21.15.59:0/182388329 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180003ce0 con 0x7fb194109170 2024-04-02T17:18:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.820+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1941a6690 con 0x7fb194109170 2024-04-02T17:18:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 Processor -- start 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 -- start start 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb192000640 1 -- 172.21.15.59:0/182388329 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb180046020 con 0x7fb194109170 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1af0 0x556cbf8d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556cbf836d70 con 0x556cbf8d1100 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556cbf7df2a0 con 0x556cbf8d1af0 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.825+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556cbf832d20 con 0x556cbf8d6c60 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57116/0 (socket says 172.21.15.59:57116) 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb199200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 -- 172.21.15.59:0/1439387803 learned_addr learned my addr 172.21.15.59:0/1439387803 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb199c00640 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1af0 0x556cbf8d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 -- 172.21.15.59:0/1439387803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 msgr2=0x556cbf8d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 -- 172.21.15.59:0/1439387803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1af0 msgr2=0x556cbf8d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1af0 0x556cbf8d65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 -- 172.21.15.59:0/1439387803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbf7f3840 con 0x556cbf8d1100 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.826+0000 7fb19a600640 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0039f0 tx=0x7fb18c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbfa06ca87e93dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb192000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 0x7fb17c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb186a00640 1 -- 172.21.15.59:0/1439387803 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c01a470 con 0x556cbf8d1100 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb192000640 1 -- 172.21.15.59:0/182388329 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb1800f2520 con 0x7fb194109170 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb186a00640 1 -- 172.21.15.59:0/1439387803 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c015070 con 0x556cbf8d1100 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb186a00640 1 -- 172.21.15.59:0/1439387803 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c010900 con 0x556cbf8d1100 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb199c00640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 0x7fb17c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb19f3ac740 1 -- 172.21.15.59:0/1439387803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 msgr2=0x556cbf8d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.827+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0039f0 tx=0x7fb18c003eb0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb199c00640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 0x7fb17c0a6060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x556cbf8d1ef0 tx=0x7fb188005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 -- 172.21.15.59:0/1439387803 shutdown_connections 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1af0 0x556cbf8d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/1439387803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1100 0x556cbf8d1520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 -- 172.21.15.59:0/1439387803 >> 172.21.15.59:0/1439387803 conn(0x556cbf8cc670 msgr2=0x556cbf8cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 -- 172.21.15.59:0/1439387803 shutdown_connections 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 -- 172.21.15.59:0/1439387803 wait complete. 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 -- start start 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.828+0000 7fb19f3ac740 1 Processor -- start 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556cbf8d9c10 con 0x556cbf8d1af0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556cbf7f2660 con 0x556cbf8d1100 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556cbf8d9d90 con 0x556cbf8d6c60 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19f3ac740 10 client.0 _ll_register_callbacks cb 0x556cbf7f2050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb199200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb199200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57124/0 (socket says 172.21.15.59:57124) 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb199200640 1 -- 172.21.15.59:0/2347978723 learned_addr learned my addr 172.21.15.59:0/2347978723 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb199c00640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.829+0000 7fb19a600640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 -- 172.21.15.59:0/2347978723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 msgr2=0x556cbf8e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 -- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 msgr2=0x556cbf8db680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8db680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 -- 172.21.15.59:0/2347978723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbf801e80 con 0x556cbf8d1af0 2024-04-02T17:18:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb19a600640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199c00640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb199200640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb180006010 tx=0x7fb1800ba440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7003e3f85ef45c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb17b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1800be570 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb1800bcaa0 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18c015070 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb1800be100 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb19f3ac740 10 client.16302 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.830+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556cbf54b210 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c015070 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fb1800c6f80 con 0x556cbf8d1af0 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 10 client.16302 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 20 client.16302 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf8e7550 0x556cbf8e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 4 client.16302 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:18:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cbf7e6cd0 con 0x556cbf8e7550 2024-04-02T17:18:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.831+0000 7fb199c00640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf8e7550 0x556cbf8e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.832+0000 7fb199c00640 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf8e7550 0x556cbf8e9940 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adb7f8d35b8b504 server_cookie=527cbec2adae06ee in_seq=0 out_seq=0 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.832+0000 7fb17b400640 10 client.16302 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb17b400640 1 -- 172.21.15.59:0/2347978723 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cbf7e6cd0 con 0x556cbf8e7550 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb17b400640 10 client.16302 handle_command_reply: tid=0 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 client.16302 shutdown 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 msgr2=0x556cbf8dbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb180006010 tx=0x7fb1800ba440 comp rx=0 tx=0).stop 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 shutdown_connections 2024-04-02T17:18:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf8e7550 0x556cbf8e9940 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8dbbc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2347978723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 >> 172.21.15.59:0/2347978723 conn(0x556cbf8cc670 msgr2=0x556cbf8cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb17b400640 0 client.16302 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.833+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 shutdown_connections 2024-04-02T17:18:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.834+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2347978723 wait complete. 2024-04-02T17:18:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.834+0000 7fb19f3ac740 20 client.16302 trim_cache size 0 max 16384 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 Processor -- start 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 -- start start 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941af5a0 con 0x556cbf8d1af0 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.850+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941b10e0 con 0x556cbf8d1100 2024-04-02T17:18:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941afc70 con 0x556cbf8d6c60 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb19a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37574/0 (socket says 172.21.15.59:37574) 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 -- 172.21.15.59:0/2074180885 learned_addr learned my addr 172.21.15.59:0/2074180885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb19a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47702/0 (socket says 172.21.15.59:47702) 2024-04-02T17:18:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57128/0 (socket says 172.21.15.59:57128) 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 -- 172.21.15.59:0/2074180885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 msgr2=0x556cbf8d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 -- 172.21.15.59:0/2074180885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 msgr2=0x556cbf8e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199c00640 1 -- 172.21.15.59:0/2074180885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbf617620 con 0x556cbf8d1100 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.851+0000 7fb199200640 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.852+0000 7fb199c00640 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x556cbf8d2560 tx=0x7fb18801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d0e25ca8398ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.852+0000 7fb186000640 1 -- 172.21.15.59:0/2074180885 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18802b420 con 0x556cbf8d1100 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.852+0000 7fb186000640 1 -- 172.21.15.59:0/2074180885 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188005020 con 0x556cbf8d1100 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.852+0000 7fb186000640 1 -- 172.21.15.59:0/2074180885 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188004890 con 0x556cbf8d1100 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2074180885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 msgr2=0x556cbf8d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x556cbf8d2560 tx=0x7fb18801a000 comp rx=0 tx=0).stop 2024-04-02T17:18:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2074180885 shutdown_connections 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d6c60 0x556cbf8d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2074180885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d1100 0x556cbf8d3030 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2074180885 >> 172.21.15.59:0/2074180885 conn(0x556cbf8cc670 msgr2=0x556cbf8cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2074180885 shutdown_connections 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2074180885 wait complete. 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 -- start start 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.853+0000 7fb19f3ac740 1 Processor -- start 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 0x556cbf94bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941b0a10 con 0x556cbf8d1af0 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941b0340 con 0x556cbf8d6c60 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941ad580 con 0x556cbf8d1100 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19f3ac740 10 client.0 _ll_register_callbacks cb 0x7fb1800f5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19a600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 0x556cbf94bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47712/0 (socket says 172.21.15.59:47712) 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 -- 172.21.15.59:0/2984988362 learned_addr learned my addr 172.21.15.59:0/2984988362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199200640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 -- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 msgr2=0x556cbf94bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 0x556cbf94bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 -- 172.21.15.59:0/2984988362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 msgr2=0x556cbf8e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199c00640 1 -- 172.21.15.59:0/2984988362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbf94bff0 con 0x556cbf8d1100 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb199200640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.854+0000 7fb19a600640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 0x556cbf94bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb199c00640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb188009fa0 tx=0x7fb188009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5486426522f1bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb17b400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188039030 con 0x556cbf8d1100 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb188005020 con 0x556cbf8d1100 2024-04-02T17:18:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb188030070 con 0x556cbf8d1100 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb188005310 con 0x556cbf8d1100 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb19f3ac740 10 client.26419 fetch_fsmap learned FSMap version 33 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.855+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556cbf94d300 con 0x556cbf8d1100 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188005020 con 0x556cbf8d1100 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fb188004c80 con 0x556cbf8d1100 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 10 client.26419 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:18:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 20 client.26419 populate_metadata read hostname 'smithi059' 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf94dbd0 0x556cbf94fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 4 client.26419 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cbf8442b0 con 0x556cbf94dbd0 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.856+0000 7fb199200640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf94dbd0 0x556cbf94fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.857+0000 7fb199200640 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf94dbd0 0x556cbf94fff0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e591695bc4f60490 server_cookie=f146269da40d8ba6 in_seq=0 out_seq=0 2024-04-02T17:18:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.857+0000 7fb17b400640 10 client.26419 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb17b400640 1 -- 172.21.15.59:0/2984988362 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cbf8442b0 con 0x556cbf94dbd0 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb17b400640 10 client.26419 handle_command_reply: tid=0 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 client.26419 shutdown 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 msgr2=0x556cbf8dfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb188009fa0 tx=0x7fb188009e30 comp rx=0 tx=0).stop 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 shutdown_connections 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cbf94dbd0 0x556cbf94fff0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cbf8d1af0 0x556cbf8e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cbf8d6c60 0x556cbf94bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 --2- 172.21.15.59:0/2984988362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cbf8d1100 0x556cbf8dfba0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb17b400640 0 client.26419 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:18:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 >> 172.21.15.59:0/2984988362 conn(0x556cbf8cc670 msgr2=0x556cbf8db620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 shutdown_connections 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 1 -- 172.21.15.59:0/2984988362 wait complete. 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.886+0000 7fb19f3ac740 20 client.26419 trim_cache size 0 max 16384 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:18:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:18:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:18:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 msgr2=0x7fb17c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 0x7fb17c0a6060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x556cbf8d1ef0 tx=0x7fb188005f90 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 msgr2=0x7fb19419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb180005ab0 tx=0x7fb18002c6c0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 shutdown_connections 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb194109b30 0x7fb1941a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb194109170 0x7fb19419e820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb17c0a3ba0 0x7fb17c0a6060 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 --2- 172.21.15.59:0/182388329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb194107f80 0x7fb19419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:18:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 >> 172.21.15.59:0/182388329 conn(0x7fb194076f40 msgr2=0x7fb19410c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:18:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 shutdown_connections 2024-04-02T17:18:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:18:47.887+0000 7fb19b000640 1 -- 172.21.15.59:0/182388329 wait complete. 2024-04-02T17:18:48.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124905 2.55 MB/sec execute 135 sec latency 711.820 ms 2024-04-02T17:18:49.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124973 2.53 MB/sec execute 136 sec latency 804.381 ms 2024-04-02T17:18:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:49 smithi059 ceph-mon[31664]: pgmap v560: 97 pgs: 97 active+clean; 47 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-04-02T17:18:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:49 smithi164 ceph-mon[36499]: pgmap v560: 97 pgs: 97 active+clean; 47 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-04-02T17:18:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:49 smithi154 ceph-mon[36479]: pgmap v560: 97 pgs: 97 active+clean; 47 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-04-02T17:18:50.062 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124973 2.51 MB/sec execute 137 sec latency 1778.170 ms 2024-04-02T17:18:51.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124973 2.50 MB/sec execute 138 sec latency 2778.275 ms 2024-04-02T17:18:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:51 smithi059 ceph-mon[31664]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 734 KiB/s wr, 105 op/s 2024-04-02T17:18:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:51 smithi164 ceph-mon[36499]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 734 KiB/s wr, 105 op/s 2024-04-02T17:18:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:51 smithi154 ceph-mon[36479]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 734 KiB/s wr, 105 op/s 2024-04-02T17:18:52.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124973 2.48 MB/sec execute 139 sec latency 3778.458 ms 2024-04-02T17:18:53.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 124973 2.46 MB/sec execute 140 sec latency 4778.660 ms 2024-04-02T17:18:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:53 smithi059 ceph-mon[31664]: pgmap v562: 97 pgs: 97 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-04-02T17:18:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:53 smithi164 ceph-mon[36499]: pgmap v562: 97 pgs: 97 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-04-02T17:18:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:53 smithi154 ceph-mon[36479]: pgmap v562: 97 pgs: 97 active+clean; 47 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-04-02T17:18:54.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 125845 2.48 MB/sec execute 141 sec latency 4876.376 ms 2024-04-02T17:18:55.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 127167 2.51 MB/sec execute 142 sec latency 32.488 ms 2024-04-02T17:18:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:55 smithi059 ceph-mon[31664]: pgmap v563: 97 pgs: 97 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 856 KiB/s wr, 89 op/s 2024-04-02T17:18:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:55 smithi164 ceph-mon[36499]: pgmap v563: 97 pgs: 97 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 856 KiB/s wr, 89 op/s 2024-04-02T17:18:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:55 smithi154 ceph-mon[36479]: pgmap v563: 97 pgs: 97 active+clean; 48 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 856 KiB/s wr, 89 op/s 2024-04-02T17:18:56.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 127995 2.54 MB/sec execute 143 sec latency 29.440 ms 2024-04-02T17:18:57.063 INFO:tasks.workunit.client.0.smithi059.stdout: 1 128254 2.52 MB/sec execute 144 sec latency 23.674 ms 2024-04-02T17:18:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:57 smithi059 ceph-mon[31664]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 890 KiB/s wr, 97 op/s 2024-04-02T17:18:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:57 smithi164 ceph-mon[36499]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 890 KiB/s wr, 97 op/s 2024-04-02T17:18:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:57 smithi154 ceph-mon[36479]: pgmap v564: 97 pgs: 97 active+clean; 48 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 890 KiB/s wr, 97 op/s 2024-04-02T17:18:58.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 128518 2.51 MB/sec execute 145 sec latency 565.887 ms 2024-04-02T17:18:59.064 INFO:tasks.workunit.client.0.smithi059.stdout: 1 129502 2.52 MB/sec execute 146 sec latency 655.862 ms 2024-04-02T17:18:59.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:18:59 smithi154 ceph-mon[36479]: pgmap v565: 97 pgs: 97 active+clean; 47 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 866 KiB/s wr, 99 op/s 2024-04-02T17:18:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:18:59 smithi059 ceph-mon[31664]: pgmap v565: 97 pgs: 97 active+clean; 47 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 866 KiB/s wr, 99 op/s 2024-04-02T17:18:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:18:59 smithi164 ceph-mon[36499]: pgmap v565: 97 pgs: 97 active+clean; 47 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 866 KiB/s wr, 99 op/s 2024-04-02T17:19:00.064 INFO:tasks.workunit.client.0.smithi059.stdout: 1 130750 2.56 MB/sec execute 147 sec latency 19.760 ms 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 -- 172.21.15.59:0/1614813652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 msgr2=0x7fe86410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 --2- 172.21.15.59:0/1614813652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe86410c1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009ca0 tx=0x7fe85c02c100 comp rx=0 tx=0).stop 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 -- 172.21.15.59:0/1614813652 shutdown_connections 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 --2- 172.21.15.59:0/1614813652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 0x7fe8641053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 --2- 172.21.15.59:0/1614813652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe86410c1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 --2- 172.21.15.59:0/1614813652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.052+0000 7fe86b600640 1 -- 172.21.15.59:0/1614813652 >> 172.21.15.59:0/1614813652 conn(0x7fe8640ff2f0 msgr2=0x7fe864101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.053+0000 7fe86b600640 1 -- 172.21.15.59:0/1614813652 shutdown_connections 2024-04-02T17:19:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.053+0000 7fe86b600640 1 -- 172.21.15.59:0/1614813652 wait complete. 2024-04-02T17:19:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.054+0000 7fe86b600640 1 Processor -- start 2024-04-02T17:19:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.054+0000 7fe86b600640 1 -- start start 2024-04-02T17:19:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.054+0000 7fe86b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe86b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 0x7fe86419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe86b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe8641a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe86b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410f9e0 con 0x7fe864104f50 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe86b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410f860 con 0x7fe864103d60 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe86b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410fb60 con 0x7fe864105910 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40720/0 (socket says 172.21.15.59:40720) 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 -- 172.21.15.59:0/3301932839 learned_addr learned my addr 172.21.15.59:0/3301932839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe863e00640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 0x7fe86419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe869600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe8641a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 -- 172.21.15.59:0/3301932839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 msgr2=0x7fe8641a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe8641a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 -- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 msgr2=0x7fe86419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 0x7fe86419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.055+0000 7fe868c00640 1 -- 172.21.15.59:0/3301932839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641a1b60 con 0x7fe864103d60 2024-04-02T17:19:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe868c00640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe85800e9c0 tx=0x7fe85800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe861600640 1 -- 172.21.15.59:0/3301932839 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85800cd80 con 0x7fe864103d60 2024-04-02T17:19:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe861600640 1 -- 172.21.15.59:0/3301932839 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8580035f0 con 0x7fe864103d60 2024-04-02T17:19:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe861600640 1 -- 172.21.15.59:0/3301932839 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858010690 con 0x7fe864103d60 2024-04-02T17:19:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8641a1e50 con 0x7fe864103d60 2024-04-02T17:19:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.056+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a22b0 con 0x7fe864103d60 2024-04-02T17:19:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.060+0000 7fe86e437740 1 Processor -- start 2024-04-02T17:19:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.060+0000 7fe86e437740 1 -- start start 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13c8c0 0x556a1b1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a1b0a1d70 con 0x556a1b13c8c0 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a1b04a2a0 con 0x556a1b13bed0 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe86e437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a1b09dd20 con 0x556a1b141a30 2024-04-02T17:19:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe861600640 1 -- 172.21.15.59:0/3301932839 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe8580037a0 con 0x7fe864103d60 2024-04-02T17:19:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe869600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe869600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40736/0 (socket says 172.21.15.59:40736) 2024-04-02T17:19:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe869600640 1 -- 172.21.15.59:0/79650855 learned_addr learned my addr 172.21.15.59:0/79650855 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe863e00640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.061+0000 7fe868c00640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13c8c0 0x556a1b1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 -- 172.21.15.59:0/79650855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 msgr2=0x556a1b143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 -- 172.21.15.59:0/79650855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13c8c0 msgr2=0x556a1b1413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13c8c0 0x556a1b1413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 -- 172.21.15.59:0/79650855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1b05e840 con 0x556a1b13bed0 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe863e00640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.062+0000 7fe869600640 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe85c02c710 tx=0x7fe85c0323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8197c87921782e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.063+0000 7fe855c00640 1 -- 172.21.15.59:0/79650855 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c043070 con 0x556a1b13bed0 2024-04-02T17:19:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.063+0000 7fe855c00640 1 -- 172.21.15.59:0/79650855 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c032c90 con 0x556a1b13bed0 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.063+0000 7fe855c00640 1 -- 172.21.15.59:0/79650855 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c039040 con 0x556a1b13bed0 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 -- 172.21.15.59:0/79650855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 msgr2=0x556a1b13c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe85c02c710 tx=0x7fe85c0323d0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 -- 172.21.15.59:0/79650855 shutdown_connections 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13c8c0 0x556a1b1413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 --2- 172.21.15.59:0/79650855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13bed0 0x556a1b13c2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 -- 172.21.15.59:0/79650855 >> 172.21.15.59:0/79650855 conn(0x556a1b137440 msgr2=0x556a1b137840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe861600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 0x7fe84c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe861600640 1 -- 172.21.15.59:0/3301932839 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe8580d0740 con 0x7fe864103d60 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 -- 172.21.15.59:0/79650855 shutdown_connections 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe869600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 0x7fe84c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.064+0000 7fe86e437740 1 -- 172.21.15.59:0/79650855 wait complete. 2024-04-02T17:19:01.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 131569 2.58 MB/sec execute 148 sec latency 31.460 ms 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.065+0000 7fe86e437740 1 -- start start 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.065+0000 7fe869600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 0x7fe84c0a6190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe85c02c640 tx=0x7fe85c02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.065+0000 7fe86e437740 1 Processor -- start 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b146380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556a1b144980 con 0x556a1b13bed0 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a1b05d660 con 0x556a1b141a30 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556a1b144b00 con 0x556a1b13c8c0 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe86e437740 10 client.0 _ll_register_callbacks cb 0x556a1b05d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe868c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe869600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe868c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35444/0 (socket says 172.21.15.59:35444) 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe868c00640 1 -- 172.21.15.59:0/3005041848 learned_addr learned my addr 172.21.15.59:0/3005041848 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.066+0000 7fe863e00640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 -- 172.21.15.59:0/3005041848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 msgr2=0x556a1b1468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b1468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 -- 172.21.15.59:0/3005041848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 msgr2=0x556a1b146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 -- 172.21.15.59:0/3005041848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1b06ce80 con 0x556a1b141a30 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe869600640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14d900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006010 tx=0x7fe85c003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c99cabc9733c109 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe84a000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c058070 con 0x556a1b141a30 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.067+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe85c004780 con 0x556a1b141a30 2024-04-02T17:19:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85c032c90 con 0x556a1b141a30 2024-04-02T17:19:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe85c064050 con 0x556a1b141a30 2024-04-02T17:19:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe86e437740 10 client.26414 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a1adb6210 con 0x556a1b141a30 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c032c90 con 0x556a1b141a30 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.068+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe85c053070 con 0x556a1b141a30 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 10 client.26414 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 20 client.26414 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b151df0 0x556a1b1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 4 client.26414 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a1b051cd0 con 0x556a1b151df0 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.069+0000 7fe868c00640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b151df0 0x556a1b1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe868c00640 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b151df0 0x556a1b1541e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9bd86868ca271ae server_cookie=9b40193c2d73abc8 in_seq=0 out_seq=0 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe84a000640 10 client.26414 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe84a000640 1 -- 172.21.15.59:0/3005041848 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a1b051cd0 con 0x556a1b151df0 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe84a000640 10 client.26414 handle_command_reply: tid=0 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe86e437740 1 client.26414 shutdown 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 msgr2=0x556a1b14d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.070+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14d900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006010 tx=0x7fe85c003f90 comp rx=0 tx=0).stop 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 shutdown_connections 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b151df0 0x556a1b1541e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14d900 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b1468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 --2- 172.21.15.59:0/3005041848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe84a000640 0 client.26414 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 >> 172.21.15.59:0/3005041848 conn(0x556a1b137440 msgr2=0x556a1b136640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 shutdown_connections 2024-04-02T17:19:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 1 -- 172.21.15.59:0/3005041848 wait complete. 2024-04-02T17:19:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.071+0000 7fe86e437740 20 client.26414 trim_cache size 0 max 16384 2024-04-02T17:19:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.094+0000 7fe86e437740 1 Processor -- start 2024-04-02T17:19:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 -- start start 2024-04-02T17:19:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b13ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b13e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ab330 con 0x556a1b13bed0 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ace70 con 0x556a1b13c8c0 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe86e437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641aba00 con 0x556a1b141a30 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe863e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe868c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b13ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe863e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40766/0 (socket says 172.21.15.59:40766) 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe863e00640 1 -- 172.21.15.59:0/2601772750 learned_addr learned my addr 172.21.15.59:0/2601772750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:01.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.095+0000 7fe869600640 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b13e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 -- 172.21.15.59:0/2601772750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 msgr2=0x556a1b13e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b13e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 -- 172.21.15.59:0/2601772750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 msgr2=0x556a1b13ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b13ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 -- 172.21.15.59:0/2601772750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1ae82620 con 0x556a1b13c8c0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe863e00640 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe850009c80 tx=0x7fe850005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d81667d9edc3578 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe855200640 1 -- 172.21.15.59:0/2601772750 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500120d0 con 0x556a1b13c8c0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe855200640 1 -- 172.21.15.59:0/2601772750 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe850021070 con 0x556a1b13c8c0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.096+0000 7fe855200640 1 -- 172.21.15.59:0/2601772750 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85001c660 con 0x556a1b13c8c0 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 -- 172.21.15.59:0/2601772750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 msgr2=0x556a1b14d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe850009c80 tx=0x7fe850005f90 comp rx=0 tx=0).stop 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 -- 172.21.15.59:0/2601772750 shutdown_connections 2024-04-02T17:19:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b141a30 0x556a1b13e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b13c8c0 0x556a1b14d8c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 --2- 172.21.15.59:0/2601772750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b13ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.097+0000 7fe86e437740 1 -- 172.21.15.59:0/2601772750 >> 172.21.15.59:0/2601772750 conn(0x556a1b137440 msgr2=0x556a1b13d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.098+0000 7fe86e437740 1 -- 172.21.15.59:0/2601772750 shutdown_connections 2024-04-02T17:19:01.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.098+0000 7fe86e437740 1 -- 172.21.15.59:0/2601772750 wait complete. 2024-04-02T17:19:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.098+0000 7fe86e437740 1 -- start start 2024-04-02T17:19:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.098+0000 7fe86e437740 1 Processor -- start 2024-04-02T17:19:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ac7a0 con 0x556a1b13bed0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641ac0d0 con 0x556a1b141a30 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641a9310 con 0x556a1b13c8c0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe86e437740 10 client.0 _ll_register_callbacks cb 0x7fe858005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe863e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe868c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe863e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35062/0 (socket says 172.21.15.59:35062) 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe863e00640 1 -- 172.21.15.59:0/3715896389 learned_addr learned my addr 172.21.15.59:0/3715896389 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.099+0000 7fe869600640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 -- 172.21.15.59:0/3715896389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 msgr2=0x556a1b14b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b14b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 -- 172.21.15.59:0/3715896389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 msgr2=0x556a1b14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b14ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 -- 172.21.15.59:0/3715896389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1b14c650 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe869600640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14c060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0324c0 tx=0x7fe85c003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a56abddf707916 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe84a000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c06d060 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe85c004680 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.100+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe850021070 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe85c058040 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 10 client.26420 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a1b14d960 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe850021070 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe85c071030 con 0x556a1b141a30 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 10 client.26420 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 20 client.26420 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b1b8750 0x556a1b1bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 4 client.26420 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.101+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a1b0af2b0 con 0x556a1b1b8750 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.102+0000 7fe868c00640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b1b8750 0x556a1b1bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.102+0000 7fe868c00640 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b1b8750 0x556a1b1bab70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e812f69849aed01f server_cookie=ac5b417504e288ce in_seq=0 out_seq=0 2024-04-02T17:19:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.102+0000 7fe84a000640 10 client.26420 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.133+0000 7fe84a000640 1 -- 172.21.15.59:0/3715896389 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a1b0af2b0 con 0x556a1b1b8750 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe84a000640 10 client.26420 handle_command_reply: tid=0 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 client.26420 shutdown 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 msgr2=0x556a1b14c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14c060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0324c0 tx=0x7fe85c003ef0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 shutdown_connections 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556a1b1b8750 0x556a1b1bab70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a1b141a30 0x556a1b14c060 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a1b13c8c0 0x556a1b14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 --2- 172.21.15.59:0/3715896389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556a1b13bed0 0x556a1b14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe84a000640 0 client.26420 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 >> 172.21.15.59:0/3715896389 conn(0x556a1b137440 msgr2=0x556a1b13ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 shutdown_connections 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 1 -- 172.21.15.59:0/3715896389 wait complete. 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.134+0000 7fe86e437740 20 client.26420 trim_cache size 0 max 16384 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:19:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:19:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 msgr2=0x7fe84c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 0x7fe84c0a6190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe85c02c640 tx=0x7fe85c02c100 comp rx=0 tx=0).stop 2024-04-02T17:19:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 msgr2=0x7fe864199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe85800e9c0 tx=0x7fe85800ee80 comp rx=0 tx=0).stop 2024-04-02T17:19:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 shutdown_connections 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe84c0a3cd0 0x7fe84c0a6190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe864104f50 0x7fe86419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe864105910 0x7fe8641a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 --2- 172.21.15.59:0/3301932839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe864103d60 0x7fe864199f40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.135+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 >> 172.21.15.59:0/3301932839 conn(0x7fe8640ff2f0 msgr2=0x7fe8641081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.136+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 shutdown_connections 2024-04-02T17:19:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:01.136+0000 7fe86b600640 1 -- 172.21.15.59:0/3301932839 wait complete. 2024-04-02T17:19:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:01 smithi059 ceph-mon[31664]: pgmap v566: 97 pgs: 97 active+clean; 48 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 789 KiB/s wr, 107 op/s 2024-04-02T17:19:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:01 smithi164 ceph-mon[36499]: pgmap v566: 97 pgs: 97 active+clean; 48 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 789 KiB/s wr, 107 op/s 2024-04-02T17:19:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:01 smithi154 ceph-mon[36479]: pgmap v566: 97 pgs: 97 active+clean; 48 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 789 KiB/s wr, 107 op/s 2024-04-02T17:19:02.064 INFO:tasks.workunit.client.0.smithi059.stdout: 1 131943 2.57 MB/sec execute 149 sec latency 121.016 ms 2024-04-02T17:19:03.064 INFO:tasks.workunit.client.0.smithi059.stdout: 1 131943 2.55 MB/sec execute 150 sec latency 1121.158 ms 2024-04-02T17:19:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:03 smithi059 ceph-mon[31664]: pgmap v567: 97 pgs: 97 active+clean; 48 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 668 KiB/s wr, 82 op/s 2024-04-02T17:19:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:03 smithi164 ceph-mon[36499]: pgmap v567: 97 pgs: 97 active+clean; 48 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 668 KiB/s wr, 82 op/s 2024-04-02T17:19:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:03 smithi154 ceph-mon[36479]: pgmap v567: 97 pgs: 97 active+clean; 48 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 668 KiB/s wr, 82 op/s 2024-04-02T17:19:04.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 132528 2.55 MB/sec execute 151 sec latency 1217.118 ms 2024-04-02T17:19:05.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 134256 2.60 MB/sec execute 152 sec latency 13.984 ms 2024-04-02T17:19:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:05 smithi059 ceph-mon[31664]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 702 KiB/s wr, 101 op/s 2024-04-02T17:19:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:05 smithi164 ceph-mon[36499]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 702 KiB/s wr, 101 op/s 2024-04-02T17:19:05.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:05 smithi154 ceph-mon[36479]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 702 KiB/s wr, 101 op/s 2024-04-02T17:19:06.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135142 2.62 MB/sec execute 153 sec latency 41.097 ms 2024-04-02T17:19:07.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135487 2.61 MB/sec execute 154 sec latency 97.494 ms 2024-04-02T17:19:07.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:07 smithi059 ceph-mon[31664]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 789 KiB/s wr, 123 op/s 2024-04-02T17:19:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:07 smithi164 ceph-mon[36499]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 789 KiB/s wr, 123 op/s 2024-04-02T17:19:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:07 smithi154 ceph-mon[36479]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 789 KiB/s wr, 123 op/s 2024-04-02T17:19:08.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135487 2.59 MB/sec execute 155 sec latency 1097.650 ms 2024-04-02T17:19:09.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135611 2.58 MB/sec execute 156 sec latency 1182.499 ms 2024-04-02T17:19:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:09 smithi059 ceph-mon[31664]: pgmap v570: 97 pgs: 97 active+clean; 50 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 797 KiB/s wr, 137 op/s 2024-04-02T17:19:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:09 smithi164 ceph-mon[36499]: pgmap v570: 97 pgs: 97 active+clean; 50 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 797 KiB/s wr, 137 op/s 2024-04-02T17:19:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:09 smithi154 ceph-mon[36479]: pgmap v570: 97 pgs: 97 active+clean; 50 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 797 KiB/s wr, 137 op/s 2024-04-02T17:19:10.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135611 2.56 MB/sec execute 157 sec latency 1693.487 ms 2024-04-02T17:19:11.065 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135611 2.54 MB/sec execute 158 sec latency 2693.585 ms 2024-04-02T17:19:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:11 smithi059 ceph-mon[31664]: pgmap v571: 97 pgs: 97 active+clean; 51 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 862 KiB/s wr, 136 op/s 2024-04-02T17:19:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:11 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:19:11.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:11 smithi164 ceph-mon[36499]: pgmap v571: 97 pgs: 97 active+clean; 51 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 862 KiB/s wr, 136 op/s 2024-04-02T17:19:11.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:11 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:19:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:11 smithi154 ceph-mon[36479]: pgmap v571: 97 pgs: 97 active+clean; 51 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 862 KiB/s wr, 136 op/s 2024-04-02T17:19:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:11 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:19:12.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135611 2.53 MB/sec execute 159 sec latency 3693.691 ms 2024-04-02T17:19:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:12.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:19:13.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 135611 2.51 MB/sec execute 160 sec latency 4693.831 ms 2024-04-02T17:19:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 811 KiB/s wr, 112 op/s 2024-04-02T17:19:13.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:19:13.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:19:13.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:19:13.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:19:13.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 811 KiB/s wr, 112 op/s 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:19:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:19:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 811 KiB/s wr, 112 op/s 2024-04-02T17:19:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:19:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:19:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:19:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:19:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:19:14.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 137156 2.54 MB/sec execute 161 sec latency 4780.452 ms 2024-04-02T17:19:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 -- 172.21.15.59:0/966218750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 msgr2=0x7fd4e41053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 --2- 172.21.15.59:0/966218750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e41053d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009900 tx=0x7fd4cc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:19:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 -- 172.21.15.59:0/966218750 shutdown_connections 2024-04-02T17:19:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 --2- 172.21.15.59:0/966218750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4105910 0x7fd4e410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 --2- 172.21.15.59:0/966218750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e41053d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 --2- 172.21.15.59:0/966218750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4103d60 0x7fd4e4104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 -- 172.21.15.59:0/966218750 >> 172.21.15.59:0/966218750 conn(0x7fd4e40ff2f0 msgr2=0x7fd4e4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 -- 172.21.15.59:0/966218750 shutdown_connections 2024-04-02T17:19:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.291+0000 7fd4ea400640 1 -- 172.21.15.59:0/966218750 wait complete. 2024-04-02T17:19:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.292+0000 7fd4ea400640 1 Processor -- start 2024-04-02T17:19:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.292+0000 7fd4ea400640 1 -- start start 2024-04-02T17:19:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.292+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4103d60 0x7fd4e4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4ea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410fa10 con 0x7fd4e4103d60 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4ea400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410f890 con 0x7fd4e4104f50 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4ea400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410fb90 con 0x7fd4e4105910 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34640/0 (socket says 172.21.15.59:34640) 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4e3e00640 1 -- 172.21.15.59:0/320032923 learned_addr learned my addr 172.21.15.59:0/320032923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4e3400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4103d60 0x7fd4e4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 -- 172.21.15.59:0/320032923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 msgr2=0x7fd4e41a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 -- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4103d60 msgr2=0x7fd4e4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4103d60 0x7fd4e4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4daa00640 1 -- 172.21.15.59:0/320032923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e41a1c10 con 0x7fd4e4104f50 2024-04-02T17:19:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.293+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4daa00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e419a530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc036040 tx=0x7fd4cc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4e0c00640 1 -- 172.21.15.59:0/320032923 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc043020 con 0x7fd4e4104f50 2024-04-02T17:19:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4e0c00640 1 -- 172.21.15.59:0/320032923 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4cc0054e0 con 0x7fd4e4104f50 2024-04-02T17:19:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4e0c00640 1 -- 172.21.15.59:0/320032923 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003c30 con 0x7fd4e4104f50 2024-04-02T17:19:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e41a1ea0 con 0x7fd4e4104f50 2024-04-02T17:19:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.294+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e41a23b0 con 0x7fd4e4104f50 2024-04-02T17:19:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.297+0000 7fd4ecffd740 1 Processor -- start 2024-04-02T17:19:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.297+0000 7fd4e0c00640 1 -- 172.21.15.59:0/320032923 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd4cc046020 con 0x7fd4e4104f50 2024-04-02T17:19:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 -- start start 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f3dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8e9bd70 con 0x55f2b8f35ed0 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8e442a0 con 0x55f2b8f3ba30 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8e97d20 con 0x55f2b8f368c0 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f3dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.298+0000 7fd4e3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34644/0 (socket says 172.21.15.59:34644) 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 -- 172.21.15.59:0/835899899 learned_addr learned my addr 172.21.15.59:0/835899899 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 -- 172.21.15.59:0/835899899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 msgr2=0x55f2b8f3dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f3dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 -- 172.21.15.59:0/835899899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 msgr2=0x55f2b8f362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 -- 172.21.15.59:0/835899899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8e58840 con 0x55f2b8f368c0 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4daa00640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f3dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.299+0000 7fd4e3400640 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009ca0 tx=0x7fd4d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1e01b0478cecaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4e0c00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 0x7fd4c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4d8c00640 1 -- 172.21.15.59:0/835899899 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4004220 con 0x55f2b8f368c0 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4d8c00640 1 -- 172.21.15.59:0/835899899 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d40043c0 con 0x55f2b8f368c0 2024-04-02T17:19:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4d8c00640 1 -- 172.21.15.59:0/835899899 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40198d0 con 0x55f2b8f368c0 2024-04-02T17:19:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4e0c00640 1 -- 172.21.15.59:0/320032923 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fd4cc0f23d0 con 0x7fd4e4104f50 2024-04-02T17:19:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.300+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 0x7fd4c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- 172.21.15.59:0/835899899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 msgr2=0x55f2b8f3b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009ca0 tx=0x7fd4d4011810 comp rx=0 tx=0).stop 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- 172.21.15.59:0/835899899 shutdown_connections 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f3dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f3b3c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/835899899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- 172.21.15.59:0/835899899 >> 172.21.15.59:0/835899899 conn(0x55f2b8f31440 msgr2=0x55f2b8f31840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 0x7fd4c80a6190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f2b8f37120 tx=0x7fd4dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- 172.21.15.59:0/835899899 shutdown_connections 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- 172.21.15.59:0/835899899 wait complete. 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.301+0000 7fd4ecffd740 1 -- start start 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 Processor -- start 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 0x55f2b8f40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8f3e9e0 con 0x55f2b8f368c0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8e57660 con 0x55f2b8f35ed0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8f3eb60 con 0x55f2b8f3ba30 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4ecffd740 10 client.0 _ll_register_callbacks cb 0x55f2b8e57050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4e3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4e3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59214/0 (socket says 172.21.15.59:59214) 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.302+0000 7fd4e3400640 1 -- 172.21.15.59:0/2041478512 learned_addr learned my addr 172.21.15.59:0/2041478512 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4daa00640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 0x55f2b8f40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 -- 172.21.15.59:0/2041478512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f3ba30 msgr2=0x55f2b8f47940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f47940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 -- 172.21.15.59:0/2041478512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 msgr2=0x55f2b8f40900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 0x55f2b8f40900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 -- 172.21.15.59:0/2041478512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b8e66e80 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4daa00640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 0x55f2b8f40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4e3400640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4011d10 tx=0x7fd4d40038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f764da9f9f0aa081 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4d1600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.303+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4028070 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fd4d4005040 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d40043c0 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd4d4034050 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 10 client.26432 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2b8bb0210 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d40043c0 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fd4d4004b80 con 0x55f2b8f35ed0 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 10 client.26432 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 10 client.26432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 20 client.26432 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8f4c070 0x55f2b8f4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 4 client.26432 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.304+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b8e4bcd0 con 0x55f2b8f4c070 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.305+0000 7fd4daa00640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8f4c070 0x55f2b8f4e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.305+0000 7fd4daa00640 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8f4c070 0x55f2b8f4e460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc16f0535ba58ad server_cookie=59128fe03efeda5c in_seq=0 out_seq=0 2024-04-02T17:19:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.305+0000 7fd4d1600640 10 client.26432 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4d1600640 1 -- 172.21.15.59:0/2041478512 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2b8e4bcd0 con 0x55f2b8f4c070 2024-04-02T17:19:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4d1600640 10 client.26432 handle_command_reply: tid=0 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 client.26432 shutdown 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 msgr2=0x55f2b8f403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4011d10 tx=0x7fd4d40038a0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 shutdown_connections 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8f4c070 0x55f2b8f4e460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f368c0 0x55f2b8f40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f47940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/2041478512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f403c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 >> 172.21.15.59:0/2041478512 conn(0x55f2b8f31440 msgr2=0x55f2b8f30640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.306+0000 7fd4d1600640 0 client.26432 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.307+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 shutdown_connections 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.307+0000 7fd4ecffd740 1 -- 172.21.15.59:0/2041478512 wait complete. 2024-04-02T17:19:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.307+0000 7fd4ecffd740 20 client.26432 trim_cache size 0 max 16384 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.326+0000 7fd4ecffd740 1 Processor -- start 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.326+0000 7fd4ecffd740 1 -- start start 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41ab310 con 0x55f2b8f3ba30 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41ace50 con 0x55f2b8f368c0 2024-04-02T17:19:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41ab9e0 con 0x55f2b8f35ed0 2024-04-02T17:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59222/0 (socket says 172.21.15.59:59222) 2024-04-02T17:19:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4daa00640 1 -- 172.21.15.59:0/3170210365 learned_addr learned my addr 172.21.15.59:0/3170210365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4e3400640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.327+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 -- 172.21.15.59:0/3170210365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 msgr2=0x55f2b8f2fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f2fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 -- 172.21.15.59:0/3170210365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f3ba30 msgr2=0x55f2b8f30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 -- 172.21.15.59:0/3170210365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8c7c620 con 0x55f2b8f368c0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4e3400640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4daa00640 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0baa50 tx=0x7fd4cc0f5e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d29c3960d5acd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4d3e00640 1 -- 172.21.15.59:0/3170210365 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0ba890 con 0x55f2b8f368c0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4d3e00640 1 -- 172.21.15.59:0/3170210365 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4cc005690 con 0x55f2b8f368c0 2024-04-02T17:19:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.328+0000 7fd4d3e00640 1 -- 172.21.15.59:0/3170210365 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0bdf50 con 0x55f2b8f368c0 2024-04-02T17:19:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 -- 172.21.15.59:0/3170210365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 msgr2=0x55f2b8f302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0baa50 tx=0x7fd4cc0f5e10 comp rx=0 tx=0).stop 2024-04-02T17:19:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 -- 172.21.15.59:0/3170210365 shutdown_connections 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f368c0 0x55f2b8f302f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/3170210365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 -- 172.21.15.59:0/3170210365 >> 172.21.15.59:0/3170210365 conn(0x55f2b8f31440 msgr2=0x55f2b8f31820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 -- 172.21.15.59:0/3170210365 shutdown_connections 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.329+0000 7fd4ecffd740 1 -- 172.21.15.59:0/3170210365 wait complete. 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 -- start start 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 Processor -- start 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f44270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41ac780 con 0x55f2b8f35ed0 2024-04-02T17:19:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41ac0b0 con 0x55f2b8f3ba30 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e41a92f0 con 0x55f2b8f368c0 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.330+0000 7fd4ecffd740 10 client.0 _ll_register_callbacks cb 0x7fd4d4004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f44270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59230/0 (socket says 172.21.15.59:59230) 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34668/0 (socket says 172.21.15.59:34668) 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4e3e00640 1 -- 172.21.15.59:0/10829593 learned_addr learned my addr 172.21.15.59:0/10829593 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 -- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 msgr2=0x55f2b8f457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 -- 172.21.15.59:0/10829593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 msgr2=0x55f2b8f44270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f44270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 -- 172.21.15.59:0/10829593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b8f45db0 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4e3e00640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4daa00640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f2b8f37840 tx=0x7fd4cc003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31eb6b3065e78406 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4d1600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.331+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0be130 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2b8ea92b0 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fd4cc004410 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd4cc0be3c0 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 10 client.26455 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2b8f47110 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f2b8ea92b0 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fd4cc0bac60 con 0x55f2b8f368c0 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 10 client.26455 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 20 client.26455 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8fb2c80 0x55f2b8fb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 4 client.26455 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.332+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2b8ea3600 con 0x55f2b8fb2c80 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.333+0000 7fd4e3400640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8fb2c80 0x55f2b8fb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.333+0000 7fd4e3400640 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8fb2c80 0x55f2b8fb50a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62661e6eb36eac server_cookie=79f61ba4314b73c in_seq=0 out_seq=0 2024-04-02T17:19:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.333+0000 7fd4d1600640 10 client.26455 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4d1600640 1 -- 172.21.15.59:0/10829593 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2b8ea3600 con 0x55f2b8fb2c80 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4d1600640 10 client.26455 handle_command_reply: tid=0 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4ecffd740 1 client.26455 shutdown 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 msgr2=0x55f2b8f447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55f2b8f37840 tx=0x7fd4cc003760 comp rx=0 tx=0).stop 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.359+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 shutdown_connections 2024-04-02T17:19:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f2b8fb2c80 0x55f2b8fb50a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f2b8f3ba30 0x55f2b8f457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f2b8f368c0 0x55f2b8f447b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 --2- 172.21.15.59:0/10829593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2b8f35ed0 0x55f2b8f44270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4d1600640 0 client.26455 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 >> 172.21.15.59:0/10829593 conn(0x55f2b8f31440 msgr2=0x55f2b8f346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 shutdown_connections 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 1 -- 172.21.15.59:0/10829593 wait complete. 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.360+0000 7fd4ecffd740 20 client.26455 trim_cache size 0 max 16384 2024-04-02T17:19:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:19:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:19:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:19:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:19:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 msgr2=0x7fd4c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 0x7fd4c80a6190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f2b8f37120 tx=0x7fd4dc009500 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 msgr2=0x7fd4e419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e419a530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc036040 tx=0x7fd4cc02c650 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 shutdown_connections 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4e4105910 0x7fd4e41a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e4104f50 0x7fd4e419a530 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd4c80a3cd0 0x7fd4c80a6190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 --2- 172.21.15.59:0/320032923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4e4103d60 0x7fd4e4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.361+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 >> 172.21.15.59:0/320032923 conn(0x7fd4e40ff2f0 msgr2=0x7fd4e41081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.362+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 shutdown_connections 2024-04-02T17:19:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:14.362+0000 7fd4ea400640 1 -- 172.21.15.59:0/320032923 wait complete. 2024-04-02T17:19:15.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 138190 2.57 MB/sec execute 162 sec latency 44.148 ms 2024-04-02T17:19:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:15 smithi059 ceph-mon[31664]: pgmap v573: 97 pgs: 97 active+clean; 50 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 802 KiB/s wr, 117 op/s 2024-04-02T17:19:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:15 smithi164 ceph-mon[36499]: pgmap v573: 97 pgs: 97 active+clean; 50 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 802 KiB/s wr, 117 op/s 2024-04-02T17:19:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:15 smithi154 ceph-mon[36479]: pgmap v573: 97 pgs: 97 active+clean; 50 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 802 KiB/s wr, 117 op/s 2024-04-02T17:19:16.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 138757 2.58 MB/sec execute 163 sec latency 37.236 ms 2024-04-02T17:19:17.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 139158 2.57 MB/sec execute 164 sec latency 165.164 ms 2024-04-02T17:19:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:17 smithi059 ceph-mon[31664]: pgmap v574: 97 pgs: 97 active+clean; 46 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 741 KiB/s wr, 100 op/s 2024-04-02T17:19:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:17 smithi164 ceph-mon[36499]: pgmap v574: 97 pgs: 97 active+clean; 46 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 741 KiB/s wr, 100 op/s 2024-04-02T17:19:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:17 smithi154 ceph-mon[36479]: pgmap v574: 97 pgs: 97 active+clean; 46 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 741 KiB/s wr, 100 op/s 2024-04-02T17:19:18.066 INFO:tasks.workunit.client.0.smithi059.stdout: 1 139158 2.55 MB/sec execute 165 sec latency 1165.294 ms 2024-04-02T17:19:19.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 140703 2.57 MB/sec execute 166 sec latency 1251.895 ms 2024-04-02T17:19:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:19 smithi059 ceph-mon[31664]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 691 KiB/s wr, 105 op/s 2024-04-02T17:19:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:19 smithi164 ceph-mon[36499]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 691 KiB/s wr, 105 op/s 2024-04-02T17:19:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:19 smithi154 ceph-mon[36479]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 691 KiB/s wr, 105 op/s 2024-04-02T17:19:20.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 141912 2.62 MB/sec execute 167 sec latency 18.112 ms 2024-04-02T17:19:21.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142336 2.62 MB/sec execute 168 sec latency 22.834 ms 2024-04-02T17:19:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:21 smithi059 ceph-mon[31664]: pgmap v576: 97 pgs: 97 active+clean; 49 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 638 KiB/s wr, 94 op/s 2024-04-02T17:19:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:21 smithi164 ceph-mon[36499]: pgmap v576: 97 pgs: 97 active+clean; 49 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 638 KiB/s wr, 94 op/s 2024-04-02T17:19:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:21 smithi154 ceph-mon[36479]: pgmap v576: 97 pgs: 97 active+clean; 49 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 638 KiB/s wr, 94 op/s 2024-04-02T17:19:22.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142577 2.60 MB/sec execute 169 sec latency 560.602 ms 2024-04-02T17:19:23.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142577 2.59 MB/sec execute 170 sec latency 1560.803 ms 2024-04-02T17:19:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:23 smithi059 ceph-mon[31664]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 536 KiB/s wr, 82 op/s 2024-04-02T17:19:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:23 smithi164 ceph-mon[36499]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 536 KiB/s wr, 82 op/s 2024-04-02T17:19:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:23 smithi154 ceph-mon[36479]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 536 KiB/s wr, 82 op/s 2024-04-02T17:19:24.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142633 2.57 MB/sec execute 171 sec latency 1650.450 ms 2024-04-02T17:19:25.067 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142633 2.56 MB/sec execute 172 sec latency 1812.901 ms 2024-04-02T17:19:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:25 smithi059 ceph-mon[31664]: pgmap v578: 97 pgs: 97 active+clean; 49 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 616 KiB/s wr, 109 op/s 2024-04-02T17:19:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:25 smithi164 ceph-mon[36499]: pgmap v578: 97 pgs: 97 active+clean; 49 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 616 KiB/s wr, 109 op/s 2024-04-02T17:19:25.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:25 smithi154 ceph-mon[36479]: pgmap v578: 97 pgs: 97 active+clean; 49 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 616 KiB/s wr, 109 op/s 2024-04-02T17:19:26.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142633 2.54 MB/sec execute 173 sec latency 2813.050 ms 2024-04-02T17:19:27.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142633 2.53 MB/sec execute 174 sec latency 3813.183 ms 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 -- 172.21.15.59:0/582136909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790107f80 msgr2=0x7fa7901083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 --2- 172.21.15.59:0/582136909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790107f80 0x7fa7901083a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa780009ca0 tx=0x7fa78002c0f0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 -- 172.21.15.59:0/582136909 shutdown_connections 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 --2- 172.21.15.59:0/582136909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 0x7fa7901095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 --2- 172.21.15.59:0/582136909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790109b30 0x7fa7901103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 --2- 172.21.15.59:0/582136909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790107f80 0x7fa7901083a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.507+0000 7fa78fe00640 1 -- 172.21.15.59:0/582136909 >> 172.21.15.59:0/582136909 conn(0x7fa790076f40 msgr2=0x7fa790079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.508+0000 7fa78fe00640 1 -- 172.21.15.59:0/582136909 shutdown_connections 2024-04-02T17:19:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.508+0000 7fa78fe00640 1 -- 172.21.15.59:0/582136909 wait complete. 2024-04-02T17:19:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.508+0000 7fa78fe00640 1 Processor -- start 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 -- start start 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790107f80 0x7fa79019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 0x7fa79019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa790113c10 con 0x7fa790109170 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa790113a90 con 0x7fa790107f80 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa790113d90 con 0x7fa790109b30 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790107f80 0x7fa79019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:38542/0 (socket says 172.21.15.59:38542) 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78f400640 1 -- 172.21.15.59:0/4002257878 learned_addr learned my addr 172.21.15.59:0/4002257878 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.509+0000 7fa78e000640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 0x7fa79019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 -- 172.21.15.59:0/4002257878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790107f80 msgr2=0x7fa79019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790107f80 0x7fa79019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 -- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 msgr2=0x7fa79019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 0x7fa79019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 -- 172.21.15.59:0/4002257878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7901a5e70 con 0x7fa790109b30 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78f400640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e970 tx=0x7fa78800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa787400640 1 -- 172.21.15.59:0/4002257878 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78800ccc0 con 0x7fa790109b30 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa787400640 1 -- 172.21.15.59:0/4002257878 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78800ce60 con 0x7fa790109b30 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7901a6160 con 0x7fa790109b30 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.510+0000 7fa787400640 1 -- 172.21.15.59:0/4002257878 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788010490 con 0x7fa790109b30 2024-04-02T17:19:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.511+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901a66b0 con 0x7fa790109b30 2024-04-02T17:19:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 Processor -- start 2024-04-02T17:19:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa787400640 1 -- 172.21.15.59:0/4002257878 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa788005020 con 0x7fa790109b30 2024-04-02T17:19:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 -- start start 2024-04-02T17:19:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a4f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5591069b4d70 con 0x559106a4f100 2024-04-02T17:19:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55910695d2a0 con 0x559106a54c60 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.514+0000 7fa7985f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5591069b0d20 con 0x559106a4faf0 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:38550/0 (socket says 172.21.15.59:38550) 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 -- 172.21.15.59:0/1938708759 learned_addr learned my addr 172.21.15.59:0/1938708759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78e000640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a4f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78e000640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 -- 172.21.15.59:0/1938708759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 msgr2=0x559106a570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 -- 172.21.15.59:0/1938708759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 msgr2=0x559106a4f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a4f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78ea00640 1 -- 172.21.15.59:0/1938708759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559106971840 con 0x559106a4faf0 2024-04-02T17:19:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa78e000640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.515+0000 7fa787400640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 0x7fa7740a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa790108fd0 tx=0x7fa7800323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4daf01dc828bb585 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa787400640 1 -- 172.21.15.59:0/4002257878 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7880d08b0 con 0x7fa790109b30 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa78e000640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 0x7fa7740a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa77fe00640 1 -- 172.21.15.59:0/1938708759 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780043070 con 0x559106a4faf0 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa77fe00640 1 -- 172.21.15.59:0/1938708759 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa780032c90 con 0x559106a4faf0 2024-04-02T17:19:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa77fe00640 1 -- 172.21.15.59:0/1938708759 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780039040 con 0x559106a4faf0 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.516+0000 7fa7985f0740 1 -- 172.21.15.59:0/1938708759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 msgr2=0x559106a545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa790108fd0 tx=0x7fa7800323d0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa78e000640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 0x7fa7740a60e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559106a50350 tx=0x7fa778005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 -- 172.21.15.59:0/1938708759 shutdown_connections 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4faf0 0x559106a545f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1938708759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a4f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 -- 172.21.15.59:0/1938708759 >> 172.21.15.59:0/1938708759 conn(0x559106a4a670 msgr2=0x559106a4aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 -- 172.21.15.59:0/1938708759 shutdown_connections 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 -- 172.21.15.59:0/1938708759 wait complete. 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 -- start start 2024-04-02T17:19:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.517+0000 7fa7985f0740 1 Processor -- start 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 0x559106a595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106a60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559106a57bb0 con 0x559106a4faf0 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559106970660 con 0x559106a4f100 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559106a57d30 con 0x559106a54c60 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa7985f0740 10 client.0 _ll_register_callbacks cb 0x559106970050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa78e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 0x559106a595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa78e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46928/0 (socket says 172.21.15.59:46928) 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa78e000640 1 -- 172.21.15.59:0/1950030077 learned_addr learned my addr 172.21.15.59:0/1950030077 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.518+0000 7fa78f400640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106a60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 -- 172.21.15.59:0/1950030077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 msgr2=0x559106a60b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106a60b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 -- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 msgr2=0x559106a595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 0x559106a595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 -- 172.21.15.59:0/1950030077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910697fe80 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78f400640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106a60b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 0x559106a595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa78e000640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7780045f0 tx=0x7fa778004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db90c3d68c2bb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa77cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778020700 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7780208a0 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.519+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa780032c90 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa778020aa0 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 10 client.16356 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591066c9210 con 0x559106a4faf0 2024-04-02T17:19:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa780032c90 con 0x559106a4faf0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa778034410 con 0x559106a4faf0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 10 client.16356 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 20 client.16356 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106a653f0 0x559106a67860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 4 client.16356 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.520+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559106964cd0 con 0x559106a653f0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.521+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106a653f0 0x559106a67860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.521+0000 7fa78ea00640 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106a653f0 0x559106a67860 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8a60311b3ca3e1 server_cookie=9dd71e8634d013d4 in_seq=0 out_seq=0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.521+0000 7fa77cc00640 10 client.16356 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa77cc00640 1 -- 172.21.15.59:0/1950030077 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559106964cd0 con 0x559106a653f0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa77cc00640 10 client.16356 handle_command_reply: tid=0 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 client.16356 shutdown 2024-04-02T17:19:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 msgr2=0x559106a59b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7780045f0 tx=0x7fa778004730 comp rx=0 tx=0).stop 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 shutdown_connections 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106a653f0 0x559106a67860 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a59b00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106a60b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa77cc00640 0 client.16356 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 --2- 172.21.15.59:0/1950030077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4f100 0x559106a595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.522+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 >> 172.21.15.59:0/1950030077 conn(0x559106a4a670 msgr2=0x559106a49870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.523+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 shutdown_connections 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.523+0000 7fa7985f0740 1 -- 172.21.15.59:0/1950030077 wait complete. 2024-04-02T17:19:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.523+0000 7fa7985f0740 20 client.16356 trim_cache size 0 max 16384 2024-04-02T17:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.539+0000 7fa7985f0740 1 Processor -- start 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.539+0000 7fa7985f0740 1 -- start start 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4f100 0x559106a4aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901af620 con 0x559106a4faf0 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b1160 con 0x559106a54c60 2024-04-02T17:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa7985f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901afcf0 con 0x559106a4f100 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46932/0 (socket says 172.21.15.59:46932) 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 -- 172.21.15.59:0/568136060 learned_addr learned my addr 172.21.15.59:0/568136060 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78ea00640 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4f100 0x559106a4aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78f400640 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 -- 172.21.15.59:0/568136060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4f100 msgr2=0x559106a4aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4f100 0x559106a4aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 -- 172.21.15.59:0/568136060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 msgr2=0x559106a48fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a48fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.540+0000 7fa78e000640 1 -- 172.21.15.59:0/568136060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559106795620 con 0x559106a4faf0 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa78e000640 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa77801a040 tx=0x7fa778005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e866ce23c94d8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa77f400640 1 -- 172.21.15.59:0/568136060 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778020900 con 0x559106a4faf0 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa77f400640 1 -- 172.21.15.59:0/568136060 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778003e50 con 0x559106a4faf0 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa77f400640 1 -- 172.21.15.59:0/568136060 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77802abc0 con 0x559106a4faf0 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa7985f0740 1 -- 172.21.15.59:0/568136060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 msgr2=0x559106a60b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.541+0000 7fa7985f0740 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa77801a040 tx=0x7fa778005fd0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 -- 172.21.15.59:0/568136060 shutdown_connections 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4faf0 0x559106a60b00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a54c60 0x559106a48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 --2- 172.21.15.59:0/568136060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a4f100 0x559106a4aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 -- 172.21.15.59:0/568136060 >> 172.21.15.59:0/568136060 conn(0x559106a4a670 msgr2=0x559106a45360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 -- 172.21.15.59:0/568136060 shutdown_connections 2024-04-02T17:19:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 -- 172.21.15.59:0/568136060 wait complete. 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 -- start start 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.542+0000 7fa7985f0740 1 Processor -- start 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 0x559106a5dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106ac99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b0a90 con 0x559106a4f100 2024-04-02T17:19:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b03c0 con 0x559106a4faf0 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901ad600 con 0x559106a54c60 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa7985f0740 10 client.0 _ll_register_callbacks cb 0x7fa7780044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46934/0 (socket says 172.21.15.59:46934) 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa78e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 0x559106a5dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa78f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106ac99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.543+0000 7fa78ea00640 1 -- 172.21.15.59:0/2910394805 learned_addr learned my addr 172.21.15.59:0/2910394805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 -- 172.21.15.59:0/2910394805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 msgr2=0x559106ac99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106ac99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 -- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 msgr2=0x559106a5dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 0x559106a5dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 -- 172.21.15.59:0/2910394805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559106ac9f90 con 0x559106a4f100 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78e000640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 0x559106a5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa78ea00640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7800324c0 tx=0x7fa780032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bef41b2e1c8eb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa77cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780039040 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa778003e50 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa780009800 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa780043040 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa7985f0740 10 client.16362 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.544+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559106acb310 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778003e50 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa780004000 con 0x559106a4f100 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 10 client.16362 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 20 client.16362 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106acb8a0 0x559106acdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 4 client.16362 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:19:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591069c22b0 con 0x559106acb8a0 2024-04-02T17:19:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.545+0000 7fa78e000640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106acb8a0 0x559106acdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.546+0000 7fa78e000640 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106acb8a0 0x559106acdc90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a2a97da5c1daf7 server_cookie=5fc77091be42a90c in_seq=0 out_seq=0 2024-04-02T17:19:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.546+0000 7fa77cc00640 10 client.16362 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:27 smithi059 ceph-mon[31664]: pgmap v579: 97 pgs: 97 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 630 KiB/s wr, 110 op/s 2024-04-02T17:19:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:27 smithi164 ceph-mon[36499]: pgmap v579: 97 pgs: 97 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 630 KiB/s wr, 110 op/s 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.575+0000 7fa77cc00640 1 -- 172.21.15.59:0/2910394805 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591069c22b0 con 0x559106acb8a0 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.575+0000 7fa77cc00640 10 client.16362 handle_command_reply: tid=0 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.575+0000 7fa7985f0740 1 client.16362 shutdown 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 msgr2=0x559106a5daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa7800324c0 tx=0x7fa780032590 comp rx=0 tx=0).stop 2024-04-02T17:19:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 shutdown_connections 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559106acb8a0 0x559106acdc90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559106a54c60 0x559106ac99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559106a4faf0 0x559106a5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 --2- 172.21.15.59:0/2910394805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559106a4f100 0x559106a5daa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa77cc00640 0 client.16362 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 >> 172.21.15.59:0/2910394805 conn(0x559106a4a670 msgr2=0x559106a4aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 shutdown_connections 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 1 -- 172.21.15.59:0/2910394805 wait complete. 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.576+0000 7fa7985f0740 20 client.16362 trim_cache size 0 max 16384 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:19:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:19:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:19:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:19:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 msgr2=0x7fa7740a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 0x7fa7740a60e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x559106a50350 tx=0x7fa778005850 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 msgr2=0x7fa7901a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e970 tx=0x7fa78800ed70 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 shutdown_connections 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7740a3c20 0x7fa7740a60e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa790109170 0x7fa79019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa790109b30 0x7fa7901a5800 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 --2- 172.21.15.59:0/4002257878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa790107f80 0x7fa79019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 >> 172.21.15.59:0/4002257878 conn(0x7fa790076f40 msgr2=0x7fa79010c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 shutdown_connections 2024-04-02T17:19:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:27.577+0000 7fa78fe00640 1 -- 172.21.15.59:0/4002257878 wait complete. 2024-04-02T17:19:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:27 smithi154 ceph-mon[36479]: pgmap v579: 97 pgs: 97 active+clean; 49 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 630 KiB/s wr, 110 op/s 2024-04-02T17:19:28.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 142633 2.51 MB/sec execute 175 sec latency 4813.349 ms 2024-04-02T17:19:29.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 143737 2.53 MB/sec execute 176 sec latency 4897.537 ms 2024-04-02T17:19:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:29 smithi059 ceph-mon[31664]: pgmap v580: 97 pgs: 97 active+clean; 50 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 740 KiB/s wr, 109 op/s 2024-04-02T17:19:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:29 smithi164 ceph-mon[36499]: pgmap v580: 97 pgs: 97 active+clean; 50 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 740 KiB/s wr, 109 op/s 2024-04-02T17:19:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:29 smithi154 ceph-mon[36479]: pgmap v580: 97 pgs: 97 active+clean; 50 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 740 KiB/s wr, 109 op/s 2024-04-02T17:19:30.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 145259 2.56 MB/sec execute 177 sec latency 16.531 ms 2024-04-02T17:19:31.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 145837 2.58 MB/sec execute 178 sec latency 27.416 ms 2024-04-02T17:19:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:31 smithi059 ceph-mon[31664]: pgmap v581: 97 pgs: 97 active+clean; 50 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 590 KiB/s wr, 81 op/s 2024-04-02T17:19:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:31 smithi164 ceph-mon[36499]: pgmap v581: 97 pgs: 97 active+clean; 50 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 590 KiB/s wr, 81 op/s 2024-04-02T17:19:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:31 smithi154 ceph-mon[36479]: pgmap v581: 97 pgs: 97 active+clean; 50 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 590 KiB/s wr, 81 op/s 2024-04-02T17:19:32.068 INFO:tasks.workunit.client.0.smithi059.stdout: 1 146126 2.56 MB/sec execute 179 sec latency 380.233 ms 2024-04-02T17:19:33.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 146126 2.55 MB/sec execute 180 sec latency 1380.398 ms 2024-04-02T17:19:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:33 smithi059 ceph-mon[31664]: pgmap v582: 97 pgs: 97 active+clean; 50 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 580 KiB/s wr, 71 op/s 2024-04-02T17:19:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:33 smithi164 ceph-mon[36499]: pgmap v582: 97 pgs: 97 active+clean; 50 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 580 KiB/s wr, 71 op/s 2024-04-02T17:19:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:33 smithi154 ceph-mon[36479]: pgmap v582: 97 pgs: 97 active+clean; 50 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 580 KiB/s wr, 71 op/s 2024-04-02T17:19:34.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 146989 2.56 MB/sec execute 181 sec latency 1469.386 ms 2024-04-02T17:19:35.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 148646 2.59 MB/sec execute 182 sec latency 17.204 ms 2024-04-02T17:19:35.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:35 smithi059 ceph-mon[31664]: pgmap v583: 97 pgs: 97 active+clean; 50 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 767 KiB/s wr, 105 op/s 2024-04-02T17:19:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:35 smithi164 ceph-mon[36499]: pgmap v583: 97 pgs: 97 active+clean; 50 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 767 KiB/s wr, 105 op/s 2024-04-02T17:19:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:35 smithi154 ceph-mon[36479]: pgmap v583: 97 pgs: 97 active+clean; 50 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 767 KiB/s wr, 105 op/s 2024-04-02T17:19:36.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 149368 2.61 MB/sec execute 183 sec latency 22.517 ms 2024-04-02T17:19:37.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 149795 2.60 MB/sec execute 184 sec latency 29.702 ms 2024-04-02T17:19:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:37 smithi059 ceph-mon[31664]: pgmap v584: 97 pgs: 97 active+clean; 50 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 755 KiB/s wr, 85 op/s 2024-04-02T17:19:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:37 smithi164 ceph-mon[36499]: pgmap v584: 97 pgs: 97 active+clean; 50 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 755 KiB/s wr, 85 op/s 2024-04-02T17:19:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:37 smithi154 ceph-mon[36479]: pgmap v584: 97 pgs: 97 active+clean; 50 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 755 KiB/s wr, 85 op/s 2024-04-02T17:19:38.069 INFO:tasks.workunit.client.0.smithi059.stdout: 1 151425 2.62 MB/sec execute 185 sec latency 28.133 ms 2024-04-02T17:19:39.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 152665 2.66 MB/sec execute 186 sec latency 18.686 ms 2024-04-02T17:19:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:39 smithi164 ceph-mon[36499]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-02T17:19:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:39 smithi154 ceph-mon[36479]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-02T17:19:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:39 smithi059 ceph-mon[31664]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-04-02T17:19:40.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 152986 2.66 MB/sec execute 187 sec latency 25.053 ms 2024-04-02T17:19:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 -- 172.21.15.59:0/3962561986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4105730 msgr2=0x7f1ce410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 --2- 172.21.15.59:0/3962561986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4105730 0x7f1ce410bfc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ca0 tx=0x7f1cdc02c100 comp rx=0 tx=0).stop 2024-04-02T17:19:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 -- 172.21.15.59:0/3962561986 shutdown_connections 2024-04-02T17:19:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 --2- 172.21.15.59:0/3962561986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4105730 0x7f1ce410bfc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 --2- 172.21.15.59:0/3962561986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4104d70 0x7f1ce41051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 --2- 172.21.15.59:0/3962561986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4103b80 0x7f1ce4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 -- 172.21.15.59:0/3962561986 >> 172.21.15.59:0/3962561986 conn(0x7f1ce40ff110 msgr2=0x7f1ce4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 -- 172.21.15.59:0/3962561986 shutdown_connections 2024-04-02T17:19:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.735+0000 7f1ceae00640 1 -- 172.21.15.59:0/3962561986 wait complete. 2024-04-02T17:19:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.736+0000 7f1ceae00640 1 Processor -- start 2024-04-02T17:19:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.736+0000 7f1ceae00640 1 -- start start 2024-04-02T17:19:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4103b80 0x7f1ce4199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410f7e0 con 0x7f1ce4103b80 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410f660 con 0x7f1ce4104d70 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ceae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410f960 con 0x7f1ce4105730 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44778/0 (socket says 172.21.15.59:44778) 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60344/0 (socket says 172.21.15.59:60344) 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ce9000640 1 -- 172.21.15.59:0/1895164584 learned_addr learned my addr 172.21.15.59:0/1895164584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4103b80 0x7f1ce4199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 -- 172.21.15.59:0/1895164584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 msgr2=0x7f1ce419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 -- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4103b80 msgr2=0x7f1ce4199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.737+0000 7f1cea400640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4103b80 0x7f1ce4199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1cea400640 1 -- 172.21.15.59:0/1895164584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce41a1a40 con 0x7f1ce4105730 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1ce9000640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1cea400640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc037040 tx=0x7f1cdc035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1ce2000640 1 -- 172.21.15.59:0/1895164584 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc039040 con 0x7f1ce4105730 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1ce2000640 1 -- 172.21.15.59:0/1895164584 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cdc044070 con 0x7f1ce4105730 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce41a1cd0 con 0x7f1ce4105730 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.738+0000 7f1ce2000640 1 -- 172.21.15.59:0/1895164584 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc004350 con 0x7f1ce4105730 2024-04-02T17:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.739+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce41a2190 con 0x7f1ce4105730 2024-04-02T17:19:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.743+0000 7f1ce2000640 1 -- 172.21.15.59:0/1895164584 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1cdc0044f0 con 0x7f1ce4105730 2024-04-02T17:19:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.743+0000 7f1ceeffe740 1 Processor -- start 2024-04-02T17:19:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.743+0000 7f1ceeffe740 1 -- start start 2024-04-02T17:19:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.743+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8edcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 0x55e7e8ee1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ee4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8e39d70 con 0x55e7e8edc910 2024-04-02T17:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8de22a0 con 0x55e7e8ee2470 2024-04-02T17:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8e35d20 con 0x55e7e8edd300 2024-04-02T17:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8edcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8edcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59558/0 (socket says 172.21.15.59:59558) 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ce9000640 1 -- 172.21.15.59:0/652328308 learned_addr learned my addr 172.21.15.59:0/652328308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 0x55e7e8ee1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.744+0000 7f1ce9000640 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ee4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 -- 172.21.15.59:0/652328308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 msgr2=0x55e7e8ee4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ee4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 -- 172.21.15.59:0/652328308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 msgr2=0x55e7e8edcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8edcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 -- 172.21.15.59:0/652328308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e8df6840 con 0x55e7e8edd300 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 0x55e7e8ee1e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009bf0 tx=0x7f1cd8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958a015a58981898 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.745+0000 7f1ce2000640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 0x7f1ccc0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.746+0000 7f1cd6a00640 1 -- 172.21.15.59:0/652328308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd801e070 con 0x55e7e8edd300 2024-04-02T17:19:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.746+0000 7f1cd6a00640 1 -- 172.21.15.59:0/652328308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd8009da0 con 0x55e7e8edd300 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.746+0000 7f1cd6a00640 1 -- 172.21.15.59:0/652328308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8019450 con 0x55e7e8edd300 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.746+0000 7f1ce2000640 1 -- 172.21.15.59:0/1895164584 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1cdc0f2990 con 0x7f1ce4105730 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.746+0000 7f1ce9000640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 0x7f1ccc0a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 -- 172.21.15.59:0/652328308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 msgr2=0x55e7e8ee1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ce9000640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 0x7f1ccc0a6160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e7e8eddb60 tx=0x7f1cd0009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 0x55e7e8ee1e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009bf0 tx=0x7f1cd8011810 comp rx=0 tx=0).stop 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 -- 172.21.15.59:0/652328308 shutdown_connections 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ee4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edd300 0x55e7e8ee1e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/652328308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8edcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 -- 172.21.15.59:0/652328308 >> 172.21.15.59:0/652328308 conn(0x55e7e8ed7e80 msgr2=0x55e7e8ed8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 -- 172.21.15.59:0/652328308 shutdown_connections 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.747+0000 7f1ceeffe740 1 -- 172.21.15.59:0/652328308 wait complete. 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.748+0000 7f1ceeffe740 1 -- start start 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.748+0000 7f1ceeffe740 1 Processor -- start 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8ee6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8ee5420 con 0x55e7e8edc910 2024-04-02T17:19:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8df5660 con 0x55e7e8edd300 2024-04-02T17:19:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7e8ee55a0 con 0x55e7e8ee2470 2024-04-02T17:19:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ceeffe740 10 client.0 _ll_register_callbacks cb 0x55e7e8df5050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44802/0 (socket says 172.21.15.59:44802) 2024-04-02T17:19:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ce9000640 1 -- 172.21.15.59:0/3313208587 learned_addr learned my addr 172.21.15.59:0/3313208587 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8ee6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.749+0000 7f1cea400640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 -- 172.21.15.59:0/3313208587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 msgr2=0x55e7e8eee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 -- 172.21.15.59:0/3313208587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 msgr2=0x55e7e8ee6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8ee6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 -- 172.21.15.59:0/3313208587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e8e04e80 con 0x55e7e8edd300 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ce9000640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009710 tx=0x7f1cd0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b6a9c3fdafbd07 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.750+0000 7f1ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd002f060 con 0x55e7e8edd300 2024-04-02T17:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1cd001c030 con 0x55e7e8edd300 2024-04-02T17:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd8009da0 con 0x55e7e8edd300 2024-04-02T17:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1cd0020650 con 0x55e7e8edd300 2024-04-02T17:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ceeffe740 10 client.26465 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.751+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7e8b4e210 con 0x55e7e8edd300 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd8009da0 con 0x55e7e8edd300 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd002a900 con 0x55e7e8edd300 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 10 client.26465 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 20 client.26465 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8ef2c50 0x55e7e8ef50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 4 client.26465 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e8de9cd0 con 0x55e7e8ef2c50 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.752+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8ef2c50 0x55e7e8ef50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.753+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8ef2c50 0x55e7e8ef50e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61666ecf927bf10 server_cookie=31dce25b71aef765 in_seq=0 out_seq=0 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.753+0000 7f1ccb400640 10 client.26465 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ccb400640 1 -- 172.21.15.59:0/3313208587 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7e8de9cd0 con 0x55e7e8ef2c50 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ccb400640 10 client.26465 handle_command_reply: tid=0 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 client.26465 shutdown 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 msgr2=0x55e7e8ee7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009710 tx=0x7f1cd0005f90 comp rx=0 tx=0).stop 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 shutdown_connections 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8ef2c50 0x55e7e8ef50e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8ee7370 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3313208587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8ee6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 >> 172.21.15.59:0/3313208587 conn(0x55e7e8ed7e80 msgr2=0x55e7e8ece870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.754+0000 7f1ccb400640 0 client.26465 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.755+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 shutdown_connections 2024-04-02T17:19:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.755+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3313208587 wait complete. 2024-04-02T17:19:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.755+0000 7f1ceeffe740 20 client.26465 trim_cache size 0 max 16384 2024-04-02T17:19:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.774+0000 7f1ceeffe740 1 Processor -- start 2024-04-02T17:19:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 -- start start 2024-04-02T17:19:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edc910 0x55e7e8ef4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 0x55e7e8eee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ef4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a6ce0 con 0x55e7e8edd300 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a8820 con 0x55e7e8ee2470 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a73b0 con 0x55e7e8edc910 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 0x55e7e8eee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1cea400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ef4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 0x55e7e8eee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59580/0 (socket says 172.21.15.59:59580) 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edc910 0x55e7e8ef4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.775+0000 7f1ce9000640 1 -- 172.21.15.59:0/3366883350 learned_addr learned my addr 172.21.15.59:0/3366883350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 -- 172.21.15.59:0/3366883350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edc910 msgr2=0x55e7e8ef4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edc910 0x55e7e8ef4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 -- 172.21.15.59:0/3366883350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 msgr2=0x55e7e8eee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 0x55e7e8eee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 -- 172.21.15.59:0/3366883350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e8c1a620 con 0x55e7e8ee2470 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cea400640 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ef4f70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005e30 tx=0x7f1cdc02c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=730ef6e86aae16f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cd6000640 1 -- 172.21.15.59:0/3366883350 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0bf2b0 con 0x55e7e8ee2470 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cd6000640 1 -- 172.21.15.59:0/3366883350 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cdc0098b0 con 0x55e7e8ee2470 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.776+0000 7f1cd6000640 1 -- 172.21.15.59:0/3366883350 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0df0b0 con 0x55e7e8ee2470 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3366883350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 msgr2=0x55e7e8ef4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ef4f70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005e30 tx=0x7f1cdc02c100 comp rx=0 tx=0).stop 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3366883350 shutdown_connections 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edd300 0x55e7e8eee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8ee2470 0x55e7e8ef4f70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/3366883350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8edc910 0x55e7e8ef4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.777+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3366883350 >> 172.21.15.59:0/3366883350 conn(0x55e7e8ed7e80 msgr2=0x55e7e8ede7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.778+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3366883350 shutdown_connections 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.778+0000 7f1ceeffe740 1 -- 172.21.15.59:0/3366883350 wait complete. 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.778+0000 7f1ceeffe740 1 -- start start 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.778+0000 7f1ceeffe740 1 Processor -- start 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8eee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a8150 con 0x55e7e8edc910 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a7a80 con 0x55e7e8edd300 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41a4ce0 con 0x55e7e8ee2470 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ceeffe740 10 client.0 _ll_register_callbacks cb 0x7f1cd0013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1cea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8eee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44826/0 (socket says 172.21.15.59:44826) 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 -- 172.21.15.59:0/4157981236 learned_addr learned my addr 172.21.15.59:0/4157981236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1cea400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60368/0 (socket says 172.21.15.59:60368) 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 -- 172.21.15.59:0/4157981236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 msgr2=0x55e7e8eeb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 -- 172.21.15.59:0/4157981236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 msgr2=0x55e7e8eee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.779+0000 7f1ce9000640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8eee100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ce9000640 1 -- 172.21.15.59:0/4157981236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e8f57250 con 0x55e7e8edd300 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1cea400640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8eee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ce9000640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009710 tx=0x7f1cd0013570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d65912d78d9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd002f040 con 0x55e7e8edd300 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1cd0009040 con 0x55e7e8edd300 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cdc0098b0 con 0x55e7e8edd300 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1cd0039080 con 0x55e7e8edd300 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.780+0000 7f1ceeffe740 10 client.26471 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7e8f58430 con 0x55e7e8edd300 2024-04-02T17:19:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cdc0098b0 con 0x55e7e8edd300 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd0004730 con 0x55e7e8edd300 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 10 client.26471 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 20 client.26471 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8f58e10 0x55e7e8f5b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 4 client.26471 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:19:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.781+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7e8e472b0 con 0x55e7e8f58e10 2024-04-02T17:19:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.782+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8f58e10 0x55e7e8f5b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.782+0000 7f1ce9a00640 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8f58e10 0x55e7e8f5b230 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7258c914fc919c9b server_cookie=121ebd4e77c2386e in_seq=0 out_seq=0 2024-04-02T17:19:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.782+0000 7f1ccb400640 10 client.26471 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ccb400640 1 -- 172.21.15.59:0/4157981236 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7e8e472b0 con 0x55e7e8f58e10 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ccb400640 10 client.26471 handle_command_reply: tid=0 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 client.26471 shutdown 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 msgr2=0x55e7e8eeabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009710 tx=0x7f1cd0013570 comp rx=0 tx=0).stop 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 shutdown_connections 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e7e8f58e10 0x55e7e8f5b230 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e8ee2470 0x55e7e8eeb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ccb400640 0 client.26471 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7e8edd300 0x55e7e8eeabf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 --2- 172.21.15.59:0/4157981236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7e8edc910 0x55e7e8eee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 >> 172.21.15.59:0/4157981236 conn(0x55e7e8ed7e80 msgr2=0x55e7e8ef4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 shutdown_connections 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.814+0000 7f1ceeffe740 1 -- 172.21.15.59:0/4157981236 wait complete. 2024-04-02T17:19:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceeffe740 20 client.26471 trim_cache size 0 max 16384 2024-04-02T17:19:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:19:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:19:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:19:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:19:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 msgr2=0x7f1ccc0a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 0x7f1ccc0a6160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55e7e8eddb60 tx=0x7f1cd0009550 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 msgr2=0x7f1ce41a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc037040 tx=0x7f1cdc035d10 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 shutdown_connections 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ce4105730 0x7f1ce41a13d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce4104d70 0x7f1ce419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1ccc0a3ca0 0x7f1ccc0a6160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 --2- 172.21.15.59:0/1895164584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce4103b80 0x7f1ce4199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.815+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 >> 172.21.15.59:0/1895164584 conn(0x7f1ce40ff110 msgr2=0x7f1ce4108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.816+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 shutdown_connections 2024-04-02T17:19:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:40.816+0000 7f1ceae00640 1 -- 172.21.15.59:0/1895164584 wait complete. 2024-04-02T17:19:41.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 153705 2.65 MB/sec execute 188 sec latency 29.714 ms 2024-04-02T17:19:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:41 smithi164 ceph-mon[36499]: pgmap v586: 97 pgs: 97 active+clean; 53 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-04-02T17:19:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:41 smithi154 ceph-mon[36479]: pgmap v586: 97 pgs: 97 active+clean; 53 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-04-02T17:19:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:41 smithi059 ceph-mon[31664]: pgmap v586: 97 pgs: 97 active+clean; 53 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-04-02T17:19:42.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 155427 2.69 MB/sec execute 189 sec latency 18.363 ms 2024-04-02T17:19:43.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 156354 2.71 MB/sec execute 190 sec latency 33.796 ms 2024-04-02T17:19:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:43 smithi164 ceph-mon[36499]: pgmap v587: 97 pgs: 97 active+clean; 53 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-04-02T17:19:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:43 smithi154 ceph-mon[36479]: pgmap v587: 97 pgs: 97 active+clean; 53 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-04-02T17:19:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:43 smithi059 ceph-mon[31664]: pgmap v587: 97 pgs: 97 active+clean; 53 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-04-02T17:19:44.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 156684 2.70 MB/sec execute 191 sec latency 19.630 ms 2024-04-02T17:19:45.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 157623 2.71 MB/sec execute 192 sec latency 27.181 ms 2024-04-02T17:19:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:45 smithi164 ceph-mon[36499]: pgmap v588: 97 pgs: 97 active+clean; 54 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-02T17:19:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:45 smithi154 ceph-mon[36479]: pgmap v588: 97 pgs: 97 active+clean; 54 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-02T17:19:45.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:45 smithi059 ceph-mon[31664]: pgmap v588: 97 pgs: 97 active+clean; 54 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-04-02T17:19:46.070 INFO:tasks.workunit.client.0.smithi059.stdout: 1 159279 2.74 MB/sec execute 193 sec latency 17.548 ms 2024-04-02T17:19:47.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 160001 2.76 MB/sec execute 194 sec latency 33.320 ms 2024-04-02T17:19:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:47 smithi154 ceph-mon[36479]: pgmap v589: 97 pgs: 97 active+clean; 54 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 931 KiB/s wr, 147 op/s 2024-04-02T17:19:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:47 smithi059 ceph-mon[31664]: pgmap v589: 97 pgs: 97 active+clean; 54 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 931 KiB/s wr, 147 op/s 2024-04-02T17:19:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:47 smithi164 ceph-mon[36499]: pgmap v589: 97 pgs: 97 active+clean; 54 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 931 KiB/s wr, 147 op/s 2024-04-02T17:19:48.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 160523 2.75 MB/sec execute 195 sec latency 24.928 ms 2024-04-02T17:19:49.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 162069 2.77 MB/sec execute 196 sec latency 23.320 ms 2024-04-02T17:19:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:49 smithi154 ceph-mon[36479]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 960 KiB/s wr, 176 op/s 2024-04-02T17:19:49.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:49 smithi059 ceph-mon[31664]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 960 KiB/s wr, 176 op/s 2024-04-02T17:19:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:49 smithi164 ceph-mon[36499]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 960 KiB/s wr, 176 op/s 2024-04-02T17:19:50.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 163292 2.81 MB/sec execute 197 sec latency 16.738 ms 2024-04-02T17:19:51.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 163618 2.80 MB/sec execute 198 sec latency 25.171 ms 2024-04-02T17:19:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:51 smithi154 ceph-mon[36479]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 788 KiB/s wr, 149 op/s 2024-04-02T17:19:51.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:51 smithi059 ceph-mon[31664]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 788 KiB/s wr, 149 op/s 2024-04-02T17:19:51.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:51 smithi164 ceph-mon[36499]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 788 KiB/s wr, 149 op/s 2024-04-02T17:19:52.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 164186 2.79 MB/sec execute 199 sec latency 35.055 ms 2024-04-02T17:19:53.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 165789 2.81 MB/sec execute 200 sec latency 19.909 ms 2024-04-02T17:19:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:53 smithi154 ceph-mon[36479]: pgmap v592: 97 pgs: 97 active+clean; 57 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 795 KiB/s wr, 137 op/s 2024-04-02T17:19:53.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:53 smithi059 ceph-mon[31664]: pgmap v592: 97 pgs: 97 active+clean; 57 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 795 KiB/s wr, 137 op/s 2024-04-02T17:19:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:53 smithi164 ceph-mon[36499]: pgmap v592: 97 pgs: 97 active+clean; 57 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 795 KiB/s wr, 137 op/s 2024-04-02T17:19:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.973+0000 7f2003200640 1 -- 172.21.15.59:0/2012212541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc103b80 msgr2=0x7f1ffc103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.973+0000 7f2003200640 1 --2- 172.21.15.59:0/2012212541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc103b80 0x7f1ffc103fa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009900 tx=0x7f1ff002bc30 comp rx=0 tx=0).stop 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 -- 172.21.15.59:0/2012212541 shutdown_connections 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 --2- 172.21.15.59:0/2012212541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc105730 0x7f1ffc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 --2- 172.21.15.59:0/2012212541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 --2- 172.21.15.59:0/2012212541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc103b80 0x7f1ffc103fa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 -- 172.21.15.59:0/2012212541 >> 172.21.15.59:0/2012212541 conn(0x7f1ffc0ff110 msgr2=0x7f1ffc101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 -- 172.21.15.59:0/2012212541 shutdown_connections 2024-04-02T17:19:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.974+0000 7f2003200640 1 -- 172.21.15.59:0/2012212541 wait complete. 2024-04-02T17:19:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.975+0000 7f2003200640 1 Processor -- start 2024-04-02T17:19:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.975+0000 7f2003200640 1 -- start start 2024-04-02T17:19:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc105730 0x7f1ffc1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc10f7f0 con 0x7f1ffc103b80 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc10f670 con 0x7f1ffc104d70 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2003200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc10f970 con 0x7f1ffc105730 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49542/0 (socket says 172.21.15.59:49542) 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001e00640 1 -- 172.21.15.59:0/215554838 learned_addr learned my addr 172.21.15.59:0/215554838 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2002800640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc105730 0x7f1ffc1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 -- 172.21.15.59:0/215554838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc105730 msgr2=0x7f1ffc1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc105730 0x7f1ffc1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 -- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 msgr2=0x7f1ffc199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001400640 1 -- 172.21.15.59:0/215554838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc1a1ae0 con 0x7f1ffc104d70 2024-04-02T17:19:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.976+0000 7f2001e00640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f2001400640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc19a400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f1ff6a00640 1 -- 172.21.15.59:0/215554838 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00ccc0 con 0x7f1ffc104d70 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f1ff6a00640 1 -- 172.21.15.59:0/215554838 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec005020 con 0x7f1ffc104d70 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc1a1dd0 con 0x7f1ffc104d70 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f1ff6a00640 1 -- 172.21.15.59:0/215554838 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0049e0 con 0x7f1ffc104d70 2024-04-02T17:19:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.977+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1a2240 con 0x7f1ffc104d70 2024-04-02T17:19:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.981+0000 7f1ff6a00640 1 -- 172.21.15.59:0/215554838 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1fec005560 con 0x7f1ffc104d70 2024-04-02T17:19:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.981+0000 7f2007437740 1 Processor -- start 2024-04-02T17:19:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.981+0000 7f2007437740 1 -- start start 2024-04-02T17:19:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560713838d70 con 0x5607138d3100 2024-04-02T17:19:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5607137e12a0 con 0x5607138d3af0 2024-04-02T17:19:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2007437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560713834d20 con 0x5607138d8c60 2024-04-02T17:19:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2002800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2001e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2002800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49558/0 (socket says 172.21.15.59:49558) 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2001e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48572/0 (socket says 172.21.15.59:48572) 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2002800640 1 -- 172.21.15.59:0/799453649 learned_addr learned my addr 172.21.15.59:0/799453649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.982+0000 7f2001400640 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 -- 172.21.15.59:0/799453649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 msgr2=0x5607138db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 -- 172.21.15.59:0/799453649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 msgr2=0x5607138d85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 -- 172.21.15.59:0/799453649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607137f5840 con 0x5607138d3100 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2001e00640 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f1ff6a00640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 0x7f1fe40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f2002800640 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80039f0 tx=0x7f1ff8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac6a64304820e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.983+0000 7f1ff6a00640 1 -- 172.21.15.59:0/215554838 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1fec0d03f0 con 0x7f1ffc104d70 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.984+0000 7f2001e00640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 0x7f1fe40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.984+0000 7f1feb000640 1 -- 172.21.15.59:0/799453649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff801a470 con 0x5607138d3100 2024-04-02T17:19:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.984+0000 7f1feb000640 1 -- 172.21.15.59:0/799453649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8015070 con 0x5607138d3100 2024-04-02T17:19:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.984+0000 7f1feb000640 1 -- 172.21.15.59:0/799453649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8010900 con 0x5607138d3100 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 -- 172.21.15.59:0/799453649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 msgr2=0x5607138d3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80039f0 tx=0x7f1ff8003eb0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2001e00640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 0x7f1fe40a6060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002c8e0 tx=0x7f1ff002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 -- 172.21.15.59:0/799453649 shutdown_connections 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138d85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 --2- 172.21.15.59:0/799453649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138d3520 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 -- 172.21.15.59:0/799453649 >> 172.21.15.59:0/799453649 conn(0x5607138ce670 msgr2=0x5607138cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 -- 172.21.15.59:0/799453649 shutdown_connections 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.985+0000 7f2007437740 1 -- 172.21.15.59:0/799453649 wait complete. 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 -- start start 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 Processor -- start 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5607138dbc10 con 0x5607138d3100 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5607137f4660 con 0x5607138d3af0 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5607138dbd90 con 0x5607138d8c60 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.986+0000 7f2007437740 10 client.0 _ll_register_callbacks cb 0x5607137f4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2002800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48586/0 (socket says 172.21.15.59:48586) 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2002800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36942/0 (socket says 172.21.15.59:36942) 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 -- 172.21.15.59:0/1596445247 learned_addr learned my addr 172.21.15.59:0/1596445247 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 -- 172.21.15.59:0/1596445247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 msgr2=0x5607138e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 -- 172.21.15.59:0/1596445247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 msgr2=0x5607138dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 -- 172.21.15.59:0/1596445247 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560713803e80 con 0x5607138d3af0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2002800640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f2001400640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0a4d50 tx=0x7f1fec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2813db19c23242 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.987+0000 7f1fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec01c070 con 0x5607138d3af0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1fec0a4df0 con 0x5607138d3af0 2024-04-02T17:19:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff8015070 con 0x5607138d3af0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1fec09bf70 con 0x5607138d3af0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f2007437740 10 client.26483 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56071354d210 con 0x5607138d3af0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8015070 con 0x5607138d3af0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.988+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1fec0cf070 con 0x5607138d3af0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 10 client.26483 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 20 client.26483 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5607138e9530 0x5607138eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 4 client.26483 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607137e8cd0 con 0x5607138e9530 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.989+0000 7f2001e00640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5607138e9530 0x5607138eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.990+0000 7f2001e00640 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5607138e9530 0x5607138eb920 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b889f4837002c2fe server_cookie=eeb5f9679f3dd569 in_seq=0 out_seq=0 2024-04-02T17:19:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.990+0000 7f1fdf400640 10 client.26483 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.990+0000 7f1fdf400640 1 -- 172.21.15.59:0/1596445247 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607137e8cd0 con 0x5607138e9530 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.990+0000 7f1fdf400640 10 client.26483 handle_command_reply: tid=0 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.990+0000 7f2007437740 1 client.26483 shutdown 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 msgr2=0x5607138ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0a4d50 tx=0x7f1fec004730 comp rx=0 tx=0).stop 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 shutdown_connections 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5607138e9530 0x5607138eb920 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f1fdf400640 0 client.26483 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138ddc50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 --2- 172.21.15.59:0/1596445247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 >> 172.21.15.59:0/1596445247 conn(0x5607138ce670 msgr2=0x5607138cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 shutdown_connections 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 1 -- 172.21.15.59:0/1596445247 wait complete. 2024-04-02T17:19:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:53.991+0000 7f2007437740 20 client.26483 trim_cache size 0 max 16384 2024-04-02T17:19:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.010+0000 7f2007437740 1 Processor -- start 2024-04-02T17:19:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 -- start start 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ab230 con 0x5607138d3100 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1acd70 con 0x5607138d3af0 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2007437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ab900 con 0x5607138d8c60 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2001400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49578/0 (socket says 172.21.15.59:49578) 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2001e00640 1 -- 172.21.15.59:0/2060185093 learned_addr learned my addr 172.21.15.59:0/2060185093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.011+0000 7f2002800640 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 -- 172.21.15.59:0/2060185093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 msgr2=0x5607138cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 -- 172.21.15.59:0/2060185093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 msgr2=0x5607138e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 -- 172.21.15.59:0/2060185093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560713619620 con 0x5607138d3100 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f2001e00640 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002e010 tx=0x7f1ff0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb850b23c92c788a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f1fea600640 1 -- 172.21.15.59:0/2060185093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0055660 con 0x5607138d3100 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f1fea600640 1 -- 172.21.15.59:0/2060185093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff0055800 con 0x5607138d3100 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.012+0000 7f1fea600640 1 -- 172.21.15.59:0/2060185093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00055f0 con 0x5607138d3100 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.013+0000 7f2007437740 1 -- 172.21.15.59:0/2060185093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 msgr2=0x5607138ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.013+0000 7f2007437740 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002e010 tx=0x7f1ff0009260 comp rx=0 tx=0).stop 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 -- 172.21.15.59:0/2060185093 shutdown_connections 2024-04-02T17:19:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3af0 0x5607138e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 --2- 172.21.15.59:0/2060185093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3100 0x5607138ccfe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 -- 172.21.15.59:0/2060185093 >> 172.21.15.59:0/2060185093 conn(0x5607138ce670 msgr2=0x5607138d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 -- 172.21.15.59:0/2060185093 shutdown_connections 2024-04-02T17:19:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 -- 172.21.15.59:0/2060185093 wait complete. 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.014+0000 7f2007437740 1 -- start start 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 Processor -- start 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 0x5607138e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3af0 0x5607138e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ac6a0 con 0x5607138d3af0 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1abfd0 con 0x5607138d3100 2024-04-02T17:19:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1a9210 con 0x5607138d8c60 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.015+0000 7f2007437740 10 client.0 _ll_register_callbacks cb 0x7f1fec0d3b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2002800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3af0 0x5607138e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2002800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36948/0 (socket says 172.21.15.59:36948) 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2002800640 1 -- 172.21.15.59:0/3529581926 learned_addr learned my addr 172.21.15.59:0/3529581926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 0x5607138e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 -- 172.21.15.59:0/3529581926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 msgr2=0x5607138e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 -- 172.21.15.59:0/3529581926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3af0 msgr2=0x5607138e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3af0 0x5607138e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 -- 172.21.15.59:0/3529581926 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607138e38e0 con 0x5607138d3100 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2002800640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f2001e00640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 0x5607138e1df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5607138e5b60 tx=0x7f1ff0055a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8927f5ad1a8c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.016+0000 7f1fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:19:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0064070 con 0x5607138d3100 2024-04-02T17:19:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f1ff0005370 con 0x5607138d3100 2024-04-02T17:19:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff0055800 con 0x5607138d3100 2024-04-02T17:19:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ff0070050 con 0x5607138d3100 2024-04-02T17:19:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f2007437740 10 client.26489 fetch_fsmap learned FSMap version 33 2024-04-02T17:19:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5607138e4c60 con 0x5607138d3100 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.017+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff0055800 con 0x5607138d3100 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1ff00037b0 con 0x5607138d3100 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 10 client.26489 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 20 client.26489 populate_metadata read hostname 'smithi059' 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56071394fe70 0x560713952290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 4 client.26489 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607138462b0 con 0x56071394fe70 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2001400640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56071394fe70 0x560713952290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:19:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f2001400640 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56071394fe70 0x560713952290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cded2e95e61988 server_cookie=bd1faef661b385f0 in_seq=0 out_seq=0 2024-04-02T17:19:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.018+0000 7f1fdf400640 10 client.26489 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f1fdf400640 1 -- 172.21.15.59:0/3529581926 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607138462b0 con 0x56071394fe70 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f1fdf400640 10 client.26489 handle_command_reply: tid=0 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 client.26489 shutdown 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 msgr2=0x5607138e1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 0x5607138e1df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5607138e5b60 tx=0x7f1ff0055a30 comp rx=0 tx=0).stop 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 shutdown_connections 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56071394fe70 0x560713952290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5607138d3af0 0x5607138e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607138d8c60 0x5607138e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 --2- 172.21.15.59:0/3529581926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607138d3100 0x5607138e1df0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f1fdf400640 0 client.26489 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 >> 172.21.15.59:0/3529581926 conn(0x5607138ce670 msgr2=0x5607138d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 shutdown_connections 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.053+0000 7f2007437740 1 -- 172.21.15.59:0/3529581926 wait complete. 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.054+0000 7f2007437740 20 client.26489 trim_cache size 0 max 16384 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:19:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.054+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 msgr2=0x7f1fe40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.054+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 0x7f1fe40a6060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ff002c8e0 tx=0x7f1ff002e040 comp rx=0 tx=0).stop 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.054+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 msgr2=0x7f1ffc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.054+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc19a400 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 shutdown_connections 2024-04-02T17:19:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ffc105730 0x7f1ffc1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc104d70 0x7f1ffc19a400 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f1fe40a3ba0 0x7f1fe40a6060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 --2- 172.21.15.59:0/215554838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ffc103b80 0x7f1ffc199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 >> 172.21.15.59:0/215554838 conn(0x7f1ffc0ff110 msgr2=0x7f1ffc108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 shutdown_connections 2024-04-02T17:19:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:19:54.055+0000 7f2003200640 1 -- 172.21.15.59:0/215554838 wait complete. 2024-04-02T17:19:54.071 INFO:tasks.workunit.client.0.smithi059.stdout: 1 166972 2.85 MB/sec execute 201 sec latency 33.740 ms 2024-04-02T17:19:55.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 167208 2.84 MB/sec execute 202 sec latency 44.806 ms 2024-04-02T17:19:55.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:55 smithi154 ceph-mon[36479]: pgmap v593: 97 pgs: 97 active+clean; 58 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-04-02T17:19:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:55 smithi059 ceph-mon[31664]: pgmap v593: 97 pgs: 97 active+clean; 58 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-04-02T17:19:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:55 smithi164 ceph-mon[36499]: pgmap v593: 97 pgs: 97 active+clean; 58 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-04-02T17:19:56.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 167953 2.83 MB/sec execute 203 sec latency 24.359 ms 2024-04-02T17:19:57.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 169651 2.87 MB/sec execute 204 sec latency 24.351 ms 2024-04-02T17:19:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:57 smithi154 ceph-mon[36479]: pgmap v594: 97 pgs: 97 active+clean; 58 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-04-02T17:19:57.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:57 smithi059 ceph-mon[31664]: pgmap v594: 97 pgs: 97 active+clean; 58 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-04-02T17:19:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:57 smithi164 ceph-mon[36499]: pgmap v594: 97 pgs: 97 active+clean; 58 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2024-04-02T17:19:58.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 170545 2.89 MB/sec execute 205 sec latency 35.287 ms 2024-04-02T17:19:59.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 170875 2.88 MB/sec execute 206 sec latency 21.757 ms 2024-04-02T17:19:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:19:59 smithi154 ceph-mon[36479]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:19:59.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:19:59 smithi059 ceph-mon[31664]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:19:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:19:59 smithi164 ceph-mon[36499]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:20:00.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 172037 2.89 MB/sec execute 207 sec latency 27.416 ms 2024-04-02T17:20:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:19:59.999+0000 7fe5e1c00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:20:00.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:19:59.999+0000 7fe5e1c00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-04-02T17:20:00.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:19:59.999+0000 7fe5e1c00700 -1 log_channel(cluster) log [ERR] : mds.g(mds.1): Metadata damage detected 2024-04-02T17:20:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:00 smithi154 ceph-mon[36479]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:20:00.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:00 smithi154 ceph-mon[36479]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-04-02T17:20:00.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:00 smithi154 ceph-mon[36479]: mds.g(mds.1): Metadata damage detected 2024-04-02T17:20:00.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-mon[31664]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:20:00.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-mon[31664]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-04-02T17:20:00.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:00 smithi059 ceph-mon[31664]: mds.g(mds.1): Metadata damage detected 2024-04-02T17:20:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:00 smithi164 ceph-mon[36499]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:20:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:00 smithi164 ceph-mon[36499]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-04-02T17:20:00.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:00 smithi164 ceph-mon[36499]: mds.g(mds.1): Metadata damage detected 2024-04-02T17:20:01.072 INFO:tasks.workunit.client.0.smithi059.stdout: 1 173577 2.92 MB/sec execute 208 sec latency 15.773 ms 2024-04-02T17:20:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:01 smithi154 ceph-mon[36479]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-04-02T17:20:01.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:01 smithi059 ceph-mon[31664]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-04-02T17:20:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:01 smithi164 ceph-mon[36499]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.0 MiB/s wr, 159 op/s 2024-04-02T17:20:02.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 174183 2.93 MB/sec execute 209 sec latency 24.210 ms 2024-04-02T17:20:03.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 174766 2.92 MB/sec execute 210 sec latency 25.601 ms 2024-04-02T17:20:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:03 smithi154 ceph-mon[36479]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 901 KiB/s wr, 146 op/s 2024-04-02T17:20:03.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:03 smithi059 ceph-mon[31664]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 901 KiB/s wr, 146 op/s 2024-04-02T17:20:03.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:03 smithi164 ceph-mon[36499]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 901 KiB/s wr, 146 op/s 2024-04-02T17:20:04.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 176320 2.94 MB/sec execute 211 sec latency 34.259 ms 2024-04-02T17:20:05.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 177567 2.97 MB/sec execute 212 sec latency 19.697 ms 2024-04-02T17:20:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:05 smithi154 ceph-mon[36479]: pgmap v598: 97 pgs: 97 active+clean; 58 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-02T17:20:05.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:05 smithi059 ceph-mon[31664]: pgmap v598: 97 pgs: 97 active+clean; 58 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-02T17:20:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:05 smithi164 ceph-mon[36499]: pgmap v598: 97 pgs: 97 active+clean; 58 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-02T17:20:06.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 177806 2.96 MB/sec execute 213 sec latency 25.932 ms 2024-04-02T17:20:07.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 178403 2.96 MB/sec execute 214 sec latency 22.647 ms 2024-04-02T17:20:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.226+0000 7f184a600640 1 -- 172.21.15.59:0/3460748204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844073860 msgr2=0x7f1844108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.226+0000 7f184a600640 1 --2- 172.21.15.59:0/3460748204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844073860 0x7f1844108b00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802bc30 comp rx=0 tx=0).stop 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.227+0000 7f184a600640 1 -- 172.21.15.59:0/3460748204 shutdown_connections 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.227+0000 7f184a600640 1 --2- 172.21.15.59:0/3460748204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f184410fe90 0x7f1844112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.227+0000 7f184a600640 1 --2- 172.21.15.59:0/3460748204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844109040 0x7f18441094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.227+0000 7f184a600640 1 --2- 172.21.15.59:0/3460748204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844073860 0x7f1844108b00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.227+0000 7f184a600640 1 -- 172.21.15.59:0/3460748204 >> 172.21.15.59:0/3460748204 conn(0x7f184406e970 msgr2=0x7f1844070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.228+0000 7f184a600640 1 -- 172.21.15.59:0/3460748204 shutdown_connections 2024-04-02T17:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.228+0000 7f184a600640 1 -- 172.21.15.59:0/3460748204 wait complete. 2024-04-02T17:20:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.230+0000 7f184a600640 1 Processor -- start 2024-04-02T17:20:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 -- start start 2024-04-02T17:20:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 0x7f184410b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 0x7f1844109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 0x7f184410a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1844115e30 con 0x7f184410fe90 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1844115cb0 con 0x7f1844073860 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.231+0000 7f184a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1844115fb0 con 0x7f1844109040 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1849200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 0x7f184410b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1849c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 0x7f184410a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1849200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 0x7f184410b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35650/0 (socket says 172.21.15.59:35650) 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1849200640 1 -- 172.21.15.59:0/3784012953 learned_addr learned my addr 172.21.15.59:0/3784012953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 0x7f1844109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 -- 172.21.15.59:0/3784012953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 msgr2=0x7f184410b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 0x7f184410b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 -- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 msgr2=0x7f184410a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 0x7f184410a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1843e00640 1 -- 172.21.15.59:0/3784012953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184410a870 con 0x7f1844109040 2024-04-02T17:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.232+0000 7f1849c00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 0x7f184410a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.233+0000 7f1843e00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 0x7f1844109df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f183000e9f0 tx=0x7f183000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.233+0000 7f1841600640 1 -- 172.21.15.59:0/3784012953 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000ccc0 con 0x7f1844109040 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.234+0000 7f1841600640 1 -- 172.21.15.59:0/3784012953 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830010910 con 0x7f1844109040 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.234+0000 7f1841600640 1 -- 172.21.15.59:0/3784012953 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830018800 con 0x7f1844109040 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.234+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184410ab60 con 0x7f1844109040 2024-04-02T17:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.234+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441aeef0 con 0x7f1844109040 2024-04-02T17:20:07.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.238+0000 7f1841600640 1 -- 172.21.15.59:0/3784012953 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1830010ac0 con 0x7f1844109040 2024-04-02T17:20:07.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.238+0000 7f184e86f740 1 Processor -- start 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.238+0000 7f184e86f740 1 -- start start 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafd3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd8c60 0x55ccdafdb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccdaf38d70 con 0x55ccdafd3100 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ccdaee12a0 con 0x55ccdafd3af0 2024-04-02T17:20:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f184e86f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ccdaf34d20 con 0x55ccdafd8c60 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f1849200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f1849c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafd3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f1849200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35666/0 (socket says 172.21.15.59:35666) 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f1849200640 1 -- 172.21.15.59:0/2861207444 learned_addr learned my addr 172.21.15.59:0/2861207444 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.239+0000 7f1843e00640 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd8c60 0x55ccdafdb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 -- 172.21.15.59:0/2861207444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd8c60 msgr2=0x55ccdafdb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd8c60 0x55ccdafdb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 -- 172.21.15.59:0/2861207444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 msgr2=0x55ccdafd3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafd3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 -- 172.21.15.59:0/2861207444 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdaef5840 con 0x55ccdafd3af0 2024-04-02T17:20:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849200640 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f183802c8e0 tx=0x7f1838009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44b71c33908b324 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1841600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 0x7f182c0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1841600640 1 -- 172.21.15.59:0/3784012953 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f18300d06f0 con 0x7f1844109040 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.240+0000 7f1849c00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 0x7f182c0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f1836000640 1 -- 172.21.15.59:0/2861207444 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838044020 con 0x55ccdafd3af0 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f1836000640 1 -- 172.21.15.59:0/2861207444 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183802cb80 con 0x55ccdafd3af0 2024-04-02T17:20:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f1836000640 1 -- 172.21.15.59:0/2861207444 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004920 con 0x55ccdafd3af0 2024-04-02T17:20:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f184e86f740 1 -- 172.21.15.59:0/2861207444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 msgr2=0x55ccdafd85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f184e86f740 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f183802c8e0 tx=0x7f1838009560 comp rx=0 tx=0).stop 2024-04-02T17:20:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.241+0000 7f1849c00640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 0x7f182c0a61b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55ccdafd4350 tx=0x7f183c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 -- 172.21.15.59:0/2861207444 shutdown_connections 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd8c60 0x55ccdafdb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafd85f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 --2- 172.21.15.59:0/2861207444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafd3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 -- 172.21.15.59:0/2861207444 >> 172.21.15.59:0/2861207444 conn(0x55ccdafce670 msgr2=0x55ccdafcea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 -- 172.21.15.59:0/2861207444 shutdown_connections 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 -- 172.21.15.59:0/2861207444 wait complete. 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.242+0000 7f184e86f740 1 -- start start 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 Processor -- start 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafe4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccdafdbbb0 con 0x55ccdafd8c60 2024-04-02T17:20:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ccdaef4660 con 0x55ccdafd3af0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ccdafdbd30 con 0x55ccdafd3100 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f184e86f740 10 client.0 _ll_register_callbacks cb 0x55ccdaef4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1849200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1843e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1849200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34578/0 (socket says 172.21.15.59:34578) 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1843e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35678/0 (socket says 172.21.15.59:35678) 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1849200640 1 -- 172.21.15.59:0/3367310632 learned_addr learned my addr 172.21.15.59:0/3367310632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.243+0000 7f1849c00640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafe4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 -- 172.21.15.59:0/3367310632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 msgr2=0x55ccdafdd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 -- 172.21.15.59:0/3367310632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 msgr2=0x55ccdafe4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafe4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 -- 172.21.15.59:0/3367310632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdaf03e80 con 0x55ccdafd3af0 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1849200640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1849c00640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafe4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f1843e00640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f18300058c0 tx=0x7f183009c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7408f1209dfb1429 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f182aa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300d2db0 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f183009bd60 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.244+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183802cb80 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18300a0cc0 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 10 client.26501 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ccdac4d210 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183802cb80 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1830098660 con 0x55ccdafd3af0 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 10 client.26501 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 10 client.26501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 20 client.26501 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.245+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdafe9440 0x55ccdafeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.246+0000 7f184e86f740 4 client.26501 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.246+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccdaee8cd0 con 0x55ccdafe9440 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.246+0000 7f1849200640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdafe9440 0x55ccdafeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.246+0000 7f1849200640 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdafe9440 0x55ccdafeb830 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b5b4e9d01d7037 server_cookie=5ce0b6ec56914bbb in_seq=0 out_seq=0 2024-04-02T17:20:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.246+0000 7f182aa00640 10 client.26501 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.247+0000 7f182aa00640 1 -- 172.21.15.59:0/3367310632 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccdaee8cd0 con 0x55ccdafe9440 2024-04-02T17:20:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.247+0000 7f182aa00640 10 client.26501 handle_command_reply: tid=0 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.247+0000 7f184e86f740 1 client.26501 shutdown 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 msgr2=0x55ccdafddb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f18300058c0 tx=0x7f183009c270 comp rx=0 tx=0).stop 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 shutdown_connections 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdafe9440 0x55ccdafeb830 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafe4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafddb60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 --2- 172.21.15.59:0/3367310632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafdd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 >> 172.21.15.59:0/3367310632 conn(0x55ccdafce670 msgr2=0x55ccdafcd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f182aa00640 0 client.26501 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 shutdown_connections 2024-04-02T17:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 1 -- 172.21.15.59:0/3367310632 wait complete. 2024-04-02T17:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.248+0000 7f184e86f740 20 client.26501 trim_cache size 0 max 16384 2024-04-02T17:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 Processor -- start 2024-04-02T17:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 -- start start 2024-04-02T17:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafe4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafd55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b7ef0 con 0x55ccdafd8c60 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b9a30 con 0x55ccdafd3af0 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.265+0000 7f184e86f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b85c0 con 0x55ccdafd3100 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34582/0 (socket says 172.21.15.59:34582) 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 -- 172.21.15.59:0/2100836846 learned_addr learned my addr 172.21.15.59:0/2100836846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849c00640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafd55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1843e00640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafe4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 -- 172.21.15.59:0/2100836846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 msgr2=0x55ccdafe4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafe4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 -- 172.21.15.59:0/2100836846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 msgr2=0x55ccdafd55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafd55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 -- 172.21.15.59:0/2100836846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdad19620 con 0x55ccdafd3100 2024-04-02T17:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1843e00640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafe4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.266+0000 7f1849200640 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f183802c8e0 tx=0x7f1838005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d955cd85bafc6a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.267+0000 7f1835600640 1 -- 172.21.15.59:0/2100836846 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838044040 con 0x55ccdafd3100 2024-04-02T17:20:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.267+0000 7f1835600640 1 -- 172.21.15.59:0/2100836846 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183804e070 con 0x55ccdafd3100 2024-04-02T17:20:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.267+0000 7f1835600640 1 -- 172.21.15.59:0/2100836846 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004920 con 0x55ccdafd3100 2024-04-02T17:20:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- 172.21.15.59:0/2100836846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 msgr2=0x55ccdafd5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f183802c8e0 tx=0x7f1838005fd0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- 172.21.15.59:0/2100836846 shutdown_connections 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd8c60 0x55ccdafd55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd3af0 0x55ccdafe4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 --2- 172.21.15.59:0/2100836846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3100 0x55ccdafd5080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- 172.21.15.59:0/2100836846 >> 172.21.15.59:0/2100836846 conn(0x55ccdafce670 msgr2=0x55ccdafdd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- 172.21.15.59:0/2100836846 shutdown_connections 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- 172.21.15.59:0/2100836846 wait complete. 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 -- start start 2024-04-02T17:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.268+0000 7f184e86f740 1 Processor -- start 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b9360 con 0x55ccdafd3100 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b8c90 con 0x55ccdafd8c60 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b5ed0 con 0x55ccdafd3af0 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f184e86f740 10 client.0 _ll_register_callbacks cb 0x7f183009c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1849200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1849c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1843e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1849200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56328/0 (socket says 172.21.15.59:56328) 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1849200640 1 -- 172.21.15.59:0/2888310064 learned_addr learned my addr 172.21.15.59:0/2888310064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1849c00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:35696/0 (socket says 172.21.15.59:35696) 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.269+0000 7f1843e00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34594/0 (socket says 172.21.15.59:34594) 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 -- 172.21.15.59:0/2888310064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 msgr2=0x55ccdafe22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 -- 172.21.15.59:0/2888310064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 msgr2=0x55ccdafe1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 -- 172.21.15.59:0/2888310064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdafe3890 con 0x55ccdafd8c60 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1843e00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849200640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f1849c00640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f183c013010 tx=0x7f183c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fcdc71f3e7c1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f182aa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c02f060 con 0x55ccdafd8c60 2024-04-02T17:20:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f183c01f420 con 0x55ccdafd8c60 2024-04-02T17:20:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.270+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183804e070 con 0x55ccdafd8c60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f183c011e60 con 0x55ccdafd8c60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 10 client.26507 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ccdafe4c10 con 0x55ccdafd8c60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183804e070 con 0x55ccdafd8c60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f183c033030 con 0x55ccdafd8c60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 10 client.26507 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 20 client.26507 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdb04fe60 0x55ccdb052280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 4 client.26507 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.271+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccdaf462b0 con 0x55ccdb04fe60 2024-04-02T17:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.272+0000 7f1849200640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdb04fe60 0x55ccdb052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.272+0000 7f1849200640 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdb04fe60 0x55ccdb052280 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e95b074d0c8d73 server_cookie=5c0356f6ce78ed61 in_seq=0 out_seq=0 2024-04-02T17:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.272+0000 7f182aa00640 10 client.26507 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f182aa00640 1 -- 172.21.15.59:0/2888310064 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccdaf462b0 con 0x55ccdb04fe60 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f182aa00640 10 client.26507 handle_command_reply: tid=0 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 client.26507 shutdown 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 msgr2=0x55ccdafe32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f183c013010 tx=0x7f183c004ab0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 shutdown_connections 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ccdb04fe60 0x55ccdb052280 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ccdafd8c60 0x55ccdafe32a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccdafd3af0 0x55ccdafe22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 --2- 172.21.15.59:0/2888310064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccdafd3100 0x55ccdafe1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f182aa00640 0 client.26507 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 >> 172.21.15.59:0/2888310064 conn(0x55ccdafce670 msgr2=0x55ccdafdd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 shutdown_connections 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.303+0000 7f184e86f740 1 -- 172.21.15.59:0/2888310064 wait complete. 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.304+0000 7f184e86f740 20 client.26507 trim_cache size 0 max 16384 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:20:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.304+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 msgr2=0x7f182c0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.304+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 0x7f182c0a61b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55ccdafd4350 tx=0x7f183c009500 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.304+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 msgr2=0x7f1844109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.304+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 0x7f1844109df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f183000e9f0 tx=0x7f183000edf0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 shutdown_connections 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f182c0a3cf0 0x7f182c0a61b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184410fe90 0x7f184410a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1844109040 0x7f1844109df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 --2- 172.21.15.59:0/3784012953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1844073860 0x7f184410b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 >> 172.21.15.59:0/3784012953 conn(0x7f184406e970 msgr2=0x7f184410be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 shutdown_connections 2024-04-02T17:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:07.305+0000 7f184a600640 1 -- 172.21.15.59:0/3784012953 wait complete. 2024-04-02T17:20:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:07 smithi154 ceph-mon[36479]: pgmap v599: 97 pgs: 97 active+clean; 59 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 961 KiB/s wr, 156 op/s 2024-04-02T17:20:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:07 smithi059 ceph-mon[31664]: pgmap v599: 97 pgs: 97 active+clean; 59 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 961 KiB/s wr, 156 op/s 2024-04-02T17:20:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:07 smithi164 ceph-mon[36499]: pgmap v599: 97 pgs: 97 active+clean; 59 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 961 KiB/s wr, 156 op/s 2024-04-02T17:20:08.074 INFO:tasks.workunit.client.0.smithi059.stdout: 1 179973 2.97 MB/sec execute 215 sec latency 39.061 ms 2024-04-02T17:20:09.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 181139 3.01 MB/sec execute 216 sec latency 22.839 ms 2024-04-02T17:20:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:09 smithi154 ceph-mon[36479]: pgmap v600: 97 pgs: 97 active+clean; 60 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 985 KiB/s wr, 171 op/s 2024-04-02T17:20:09.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:09 smithi059 ceph-mon[31664]: pgmap v600: 97 pgs: 97 active+clean; 60 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 985 KiB/s wr, 171 op/s 2024-04-02T17:20:09.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:09 smithi164 ceph-mon[36499]: pgmap v600: 97 pgs: 97 active+clean; 60 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 985 KiB/s wr, 171 op/s 2024-04-02T17:20:10.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 181377 2.99 MB/sec execute 217 sec latency 27.101 ms 2024-04-02T17:20:11.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 182054 2.99 MB/sec execute 218 sec latency 26.167 ms 2024-04-02T17:20:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:11 smithi154 ceph-mon[36479]: pgmap v601: 97 pgs: 97 active+clean; 62 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-02T17:20:11.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:11 smithi059 ceph-mon[31664]: pgmap v601: 97 pgs: 97 active+clean; 62 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-02T17:20:11.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:11 smithi164 ceph-mon[36499]: pgmap v601: 97 pgs: 97 active+clean; 62 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-02T17:20:12.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 183787 3.02 MB/sec execute 219 sec latency 18.713 ms 2024-04-02T17:20:13.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 184735 3.04 MB/sec execute 220 sec latency 36.366 ms 2024-04-02T17:20:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: pgmap v602: 97 pgs: 97 active+clean; 62 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-04-02T17:20:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:13 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: pgmap v602: 97 pgs: 97 active+clean; 62 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:13 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: pgmap v602: 97 pgs: 97 active+clean; 62 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-04-02T17:20:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:20:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:13.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:13 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:20:14.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 185323 3.03 MB/sec execute 221 sec latency 21.845 ms 2024-04-02T17:20:14.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:20:14.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:20:14.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:20:14.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:20:14.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:20:15.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 186886 3.05 MB/sec execute 222 sec latency 22.604 ms 2024-04-02T17:20:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:15 smithi059 ceph-mon[31664]: pgmap v603: 97 pgs: 97 active+clean; 64 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-02T17:20:15.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:15 smithi164 ceph-mon[36499]: pgmap v603: 97 pgs: 97 active+clean; 64 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-02T17:20:15.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:15 smithi154 ceph-mon[36479]: pgmap v603: 97 pgs: 97 active+clean; 64 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.4 MiB/s wr, 184 op/s 2024-04-02T17:20:16.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 188109 3.08 MB/sec execute 223 sec latency 16.669 ms 2024-04-02T17:20:17.075 INFO:tasks.workunit.client.0.smithi059.stdout: 1 188437 3.07 MB/sec execute 224 sec latency 28.690 ms 2024-04-02T17:20:17.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:17 smithi059 ceph-mon[31664]: pgmap v604: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:17 smithi164 ceph-mon[36499]: pgmap v604: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:17.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:17 smithi154 ceph-mon[36479]: pgmap v604: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:18.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 188985 3.06 MB/sec execute 225 sec latency 46.108 ms 2024-04-02T17:20:19.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 190577 3.08 MB/sec execute 226 sec latency 15.191 ms 2024-04-02T17:20:19.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:19 smithi059 ceph-mon[31664]: pgmap v605: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:19.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:19 smithi164 ceph-mon[36499]: pgmap v605: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:19.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:19 smithi154 ceph-mon[36479]: pgmap v605: 97 pgs: 97 active+clean; 64 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:20.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 191783 3.11 MB/sec execute 227 sec latency 29.180 ms 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 -- 172.21.15.59:0/2194071189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370103b80 msgr2=0x7f1370103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 --2- 172.21.15.59:0/2194071189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370103b80 0x7f1370103fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1364009860 tx=0x7f136402bb70 comp rx=0 tx=0).stop 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 -- 172.21.15.59:0/2194071189 shutdown_connections 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 --2- 172.21.15.59:0/2194071189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370104d70 0x7f13701051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 --2- 172.21.15.59:0/2194071189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f137010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 --2- 172.21.15.59:0/2194071189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370103b80 0x7f1370103fa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.461+0000 7f1376800640 1 -- 172.21.15.59:0/2194071189 >> 172.21.15.59:0/2194071189 conn(0x7f13700ff110 msgr2=0x7f1370101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.462+0000 7f1376800640 1 -- 172.21.15.59:0/2194071189 shutdown_connections 2024-04-02T17:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.462+0000 7f1376800640 1 -- 172.21.15.59:0/2194071189 wait complete. 2024-04-02T17:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.463+0000 7f1376800640 1 Processor -- start 2024-04-02T17:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.463+0000 7f1376800640 1 -- start start 2024-04-02T17:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.463+0000 7f1376800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370103b80 0x7f13700798b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.464+0000 7f1376800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 0x7f1370079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.464+0000 7f1376800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.464+0000 7f1376800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f800 con 0x7f1370103b80 2024-04-02T17:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.464+0000 7f1376800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f680 con 0x7f1370104d70 2024-04-02T17:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.464+0000 7f1376800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f980 con 0x7f1370105730 2024-04-02T17:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1375e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 0x7f1370079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1375e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42046/0 (socket says 172.21.15.59:42046) 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1375e00640 1 -- 172.21.15.59:0/2420496308 learned_addr learned my addr 172.21.15.59:0/2420496308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1375400640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370103b80 0x7f13700798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 -- 172.21.15.59:0/2420496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 msgr2=0x7f1370076420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 -- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370103b80 msgr2=0x7f13700798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370103b80 0x7f13700798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.465+0000 7f1374a00640 1 -- 172.21.15.59:0/2420496308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1370076ce0 con 0x7f1370104d70 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f1375e00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f1374a00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 0x7f1370079df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f135c005fd0 tx=0x7f135c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f136e000640 1 -- 172.21.15.59:0/2420496308 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c00cd00 con 0x7f1370104d70 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f136e000640 1 -- 172.21.15.59:0/2420496308 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135c0035f0 con 0x7f1370104d70 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701a62c0 con 0x7f1370104d70 2024-04-02T17:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f136e000640 1 -- 172.21.15.59:0/2420496308 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c010670 con 0x7f1370104d70 2024-04-02T17:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.466+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a66d0 con 0x7f1370104d70 2024-04-02T17:20:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.470+0000 7f136e000640 1 -- 172.21.15.59:0/2420496308 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f135c0037a0 con 0x7f1370104d70 2024-04-02T17:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 Processor -- start 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 -- start start 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d29b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d29e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55820d1fbd70 con 0x55820d296100 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55820d1a42a0 con 0x55820d29bc60 2024-04-02T17:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.471+0000 7f137abba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55820d1f7d20 con 0x55820d296af0 2024-04-02T17:20:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33702/0 (socket says 172.21.15.59:33702) 2024-04-02T17:20:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 -- 172.21.15.59:0/3726184903 learned_addr learned my addr 172.21.15.59:0/3726184903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375400640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d29b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f136e000640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 0x7f13580a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1374a00640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d29e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 -- 172.21.15.59:0/3726184903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 msgr2=0x55820d29b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d29b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 -- 172.21.15.59:0/3726184903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 msgr2=0x55820d29e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.472+0000 7f1375e00640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d29e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.473+0000 7f1375e00640 1 -- 172.21.15.59:0/3726184903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820d1b8840 con 0x55820d296100 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.473+0000 7f1375400640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d29b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.473+0000 7f136e000640 1 -- 172.21.15.59:0/2420496308 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f135c0d0640 con 0x7f1370104d70 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.473+0000 7f1375e00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 0x7f13580a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.473+0000 7f1375e00640 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1368011a30 tx=0x7f1368011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea015da25752fab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.474+0000 7f1362600640 1 -- 172.21.15.59:0/3726184903 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368017070 con 0x55820d296100 2024-04-02T17:20:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.474+0000 7f1375e00640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 0x7f13580a6060 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1368023000 tx=0x7f1368024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.474+0000 7f1362600640 1 -- 172.21.15.59:0/3726184903 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1368010910 con 0x55820d296100 2024-04-02T17:20:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.474+0000 7f1362600640 1 -- 172.21.15.59:0/3726184903 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136800e7b0 con 0x55820d296100 2024-04-02T17:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.475+0000 7f137abba740 1 -- 172.21.15.59:0/3726184903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 msgr2=0x55820d296520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.475+0000 7f137abba740 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1368011a30 tx=0x7f1368011ef0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.475+0000 7f137abba740 1 -- 172.21.15.59:0/3726184903 shutdown_connections 2024-04-02T17:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d29e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d29b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 --2- 172.21.15.59:0/3726184903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d296520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 -- 172.21.15.59:0/3726184903 >> 172.21.15.59:0/3726184903 conn(0x55820d291670 msgr2=0x55820d291a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 -- 172.21.15.59:0/3726184903 shutdown_connections 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 -- 172.21.15.59:0/3726184903 wait complete. 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 -- start start 2024-04-02T17:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.476+0000 7f137abba740 1 Processor -- start 2024-04-02T17:20:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d2a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d2a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55820d29ec10 con 0x55820d296100 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55820d1b7660 con 0x55820d29bc60 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55820d29ed90 con 0x55820d296af0 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f1374a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f1375e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d2a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f1374a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42070/0 (socket says 172.21.15.59:42070) 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f1374a00640 1 -- 172.21.15.59:0/3556129206 learned_addr learned my addr 172.21.15.59:0/3556129206 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f1375400640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d2a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.477+0000 7f137abba740 10 client.0 _ll_register_callbacks cb 0x55820d1b7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 -- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 msgr2=0x55820d2a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d2a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 -- 172.21.15.59:0/3556129206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 msgr2=0x55820d2a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d2a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 -- 172.21.15.59:0/3556129206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820d1c6e80 con 0x55820d296af0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1375e00640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d2a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1374a00640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ce080 tx=0x7f135c009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98941d7f65eb2d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1356a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0a0c10 con 0x55820d296af0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f135c003af0 con 0x55820d296af0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.478+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1368010910 con 0x55820d296af0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f135c0a54d0 con 0x55820d296af0 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 10 client.26539 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820cf10210 con 0x55820d296af0 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1368010910 con 0x55820d296af0 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f135c09bcb0 con 0x55820d296af0 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 10 client.26539 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 10 client.26539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 20 client.26539 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d2ac300 0x55820d2ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 4 client.26539 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.479+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55820d1abcd0 con 0x55820d2ac300 2024-04-02T17:20:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.480+0000 7f1375400640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d2ac300 0x55820d2ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.480+0000 7f1375400640 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d2ac300 0x55820d2ae770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f747448cf2bac14 server_cookie=cec98f4c6a9af637 in_seq=0 out_seq=0 2024-04-02T17:20:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.480+0000 7f1356a00640 10 client.26539 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f1356a00640 1 -- 172.21.15.59:0/3556129206 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55820d1abcd0 con 0x55820d2ac300 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f1356a00640 10 client.26539 handle_command_reply: tid=0 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 client.26539 shutdown 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 msgr2=0x55820d2a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ce080 tx=0x7f135c009530 comp rx=0 tx=0).stop 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 shutdown_connections 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d2ac300 0x55820d2ae770 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d2a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296af0 0x55820d2a0a70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f137abba740 1 --2- 172.21.15.59:0/3556129206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296100 0x55820d2a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.482+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 >> 172.21.15.59:0/3556129206 conn(0x55820d291670 msgr2=0x55820d290870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.481+0000 7f1356a00640 0 client.26539 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.482+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 shutdown_connections 2024-04-02T17:20:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.482+0000 7f137abba740 1 -- 172.21.15.59:0/3556129206 wait complete. 2024-04-02T17:20:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.482+0000 7f137abba740 20 client.26539 trim_cache size 0 max 16384 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.506+0000 7f137abba740 1 Processor -- start 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.506+0000 7f137abba740 1 -- start start 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d298490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d28ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13701af6f0 con 0x55820d296af0 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b1230 con 0x55820d29bc60 2024-04-02T17:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f137abba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13701afdc0 con 0x55820d296100 2024-04-02T17:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42076/0 (socket says 172.21.15.59:42076) 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d28ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 -- 172.21.15.59:0/216705187 learned_addr learned my addr 172.21.15.59:0/216705187 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1374a00640 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d298490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 -- 172.21.15.59:0/216705187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 msgr2=0x55820d28ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d28ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 -- 172.21.15.59:0/216705187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 msgr2=0x55820d298490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.507+0000 7f1375400640 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d298490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.508+0000 7f1375400640 1 -- 172.21.15.59:0/216705187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820cfdc620 con 0x55820d296100 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.508+0000 7f1375400640 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f136402c180 tx=0x7f1364034ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a21e1cd6459d78c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.508+0000 7f1361c00640 1 -- 172.21.15.59:0/216705187 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364039040 con 0x55820d296100 2024-04-02T17:20:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.508+0000 7f1361c00640 1 -- 172.21.15.59:0/216705187 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364034640 con 0x55820d296100 2024-04-02T17:20:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.508+0000 7f1361c00640 1 -- 172.21.15.59:0/216705187 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364032640 con 0x55820d296100 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 -- 172.21.15.59:0/216705187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 msgr2=0x55820d297f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f136402c180 tx=0x7f1364034ab0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 -- 172.21.15.59:0/216705187 shutdown_connections 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d298490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d29bc60 0x55820d28ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 --2- 172.21.15.59:0/216705187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d296100 0x55820d297f50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 -- 172.21.15.59:0/216705187 >> 172.21.15.59:0/216705187 conn(0x55820d291670 msgr2=0x55820d291a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 -- 172.21.15.59:0/216705187 shutdown_connections 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.509+0000 7f137abba740 1 -- 172.21.15.59:0/216705187 wait complete. 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 -- start start 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 Processor -- start 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d2a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 0x55820d2a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0b60 con 0x55820d296af0 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0490 con 0x55820d296100 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.510+0000 7f137abba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13701ad6d0 con 0x55820d29bc60 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f137abba740 10 client.0 _ll_register_callbacks cb 0x7f135c09c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33390/0 (socket says 172.21.15.59:33390) 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 -- 172.21.15.59:0/3466627252 learned_addr learned my addr 172.21.15.59:0/3466627252 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1374a00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d2a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375e00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 0x55820d2a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 -- 172.21.15.59:0/3466627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 msgr2=0x55820d2a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 0x55820d2a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 -- 172.21.15.59:0/3466627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 msgr2=0x55820d2a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d2a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 -- 172.21.15.59:0/3466627252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820d2a64f0 con 0x55820d296100 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375e00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 0x55820d2a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1374a00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d2a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1375400640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1364009780 tx=0x7f1364034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a0fb6a6b33daa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.511+0000 7f1356a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364052020 con 0x55820d296100 2024-04-02T17:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1364034640 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f136402c510 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1364039070 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f137abba740 10 client.26525 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820d312bf0 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.512+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364034640 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1364032cb0 con 0x55820d296100 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 10 client.26525 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 20 client.26525 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d3135c0 0x55820d3159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 4 client.26525 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55820d2092b0 con 0x55820d3135c0 2024-04-02T17:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f1374a00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d3135c0 0x55820d3159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f1374a00640 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d3135c0 0x55820d3159e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85500ff5815191c1 server_cookie=54a7a4f941e2ba0e in_seq=0 out_seq=0 2024-04-02T17:20:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.513+0000 7f1356a00640 10 client.26525 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f1356a00640 1 -- 172.21.15.59:0/3466627252 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55820d2092b0 con 0x55820d3135c0 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f1356a00640 10 client.26525 handle_command_reply: tid=0 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 client.26525 shutdown 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 msgr2=0x55820d2a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1364009780 tx=0x7f1364034440 comp rx=0 tx=0).stop 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 shutdown_connections 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55820d3135c0 0x55820d3159e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55820d296af0 0x55820d2a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f1356a00640 0 client.26525 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55820d29bc60 0x55820d2a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 --2- 172.21.15.59:0/3466627252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55820d296100 0x55820d2a4a00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 >> 172.21.15.59:0/3466627252 conn(0x55820d291670 msgr2=0x55820d2a04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 shutdown_connections 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.546+0000 7f137abba740 1 -- 172.21.15.59:0/3466627252 wait complete. 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f137abba740 20 client.26525 trim_cache size 0 max 16384 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 msgr2=0x7f13580a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 0x7f13580a6060 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1368023000 tx=0x7f1368024000 comp rx=0 tx=0).stop 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 msgr2=0x7f1370079df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 0x7f1370079df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f135c005fd0 tx=0x7f135c00ee50 comp rx=0 tx=0).stop 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 shutdown_connections 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370105730 0x7f1370076420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1370104d70 0x7f1370079df0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f13580a3ba0 0x7f13580a6060 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 --2- 172.21.15.59:0/2420496308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1370103b80 0x7f13700798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.547+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 >> 172.21.15.59:0/2420496308 conn(0x7f13700ff110 msgr2=0x7f1370108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.548+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 shutdown_connections 2024-04-02T17:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:20.548+0000 7f1376800640 1 -- 172.21.15.59:0/2420496308 wait complete. 2024-04-02T17:20:21.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 192083 3.10 MB/sec execute 228 sec latency 45.157 ms 2024-04-02T17:20:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:21 smithi059 ceph-mon[31664]: pgmap v606: 97 pgs: 97 active+clean; 64 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:21 smithi164 ceph-mon[36499]: pgmap v606: 97 pgs: 97 active+clean; 64 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:21.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:21 smithi154 ceph-mon[36479]: pgmap v606: 97 pgs: 97 active+clean; 64 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-04-02T17:20:22.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 193079 3.11 MB/sec execute 229 sec latency 23.662 ms 2024-04-02T17:20:23.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 194742 3.13 MB/sec execute 230 sec latency 52.805 ms 2024-04-02T17:20:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:23 smithi059 ceph-mon[31664]: pgmap v607: 97 pgs: 97 active+clean; 64 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:20:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:23 smithi164 ceph-mon[36499]: pgmap v607: 97 pgs: 97 active+clean; 64 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:20:23.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:23 smithi154 ceph-mon[36479]: pgmap v607: 97 pgs: 97 active+clean; 64 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-04-02T17:20:24.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 195452 3.14 MB/sec execute 231 sec latency 25.263 ms 2024-04-02T17:20:25.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 195848 3.13 MB/sec execute 232 sec latency 24.336 ms 2024-04-02T17:20:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:25 smithi059 ceph-mon[31664]: pgmap v608: 97 pgs: 97 active+clean; 61 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-02T17:20:25.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:25 smithi164 ceph-mon[36499]: pgmap v608: 97 pgs: 97 active+clean; 61 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-02T17:20:25.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:25 smithi154 ceph-mon[36479]: pgmap v608: 97 pgs: 97 active+clean; 61 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-02T17:20:26.076 INFO:tasks.workunit.client.0.smithi059.stdout: 1 197427 3.15 MB/sec execute 233 sec latency 24.515 ms 2024-04-02T17:20:27.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 198646 3.18 MB/sec execute 234 sec latency 17.010 ms 2024-04-02T17:20:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:27 smithi059 ceph-mon[31664]: pgmap v609: 97 pgs: 97 active+clean; 62 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-04-02T17:20:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:27 smithi164 ceph-mon[36499]: pgmap v609: 97 pgs: 97 active+clean; 62 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-04-02T17:20:27.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:27 smithi154 ceph-mon[36479]: pgmap v609: 97 pgs: 97 active+clean; 62 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-04-02T17:20:28.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 199071 3.17 MB/sec execute 235 sec latency 37.228 ms 2024-04-02T17:20:29.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 199618 3.16 MB/sec execute 236 sec latency 23.213 ms 2024-04-02T17:20:29.701 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:29 smithi154 ceph-mon[36479]: pgmap v610: 97 pgs: 97 active+clean; 62 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-02T17:20:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:29 smithi059 ceph-mon[31664]: pgmap v610: 97 pgs: 97 active+clean; 62 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-02T17:20:29.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:29 smithi164 ceph-mon[36499]: pgmap v610: 97 pgs: 97 active+clean; 62 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-02T17:20:30.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 201235 3.18 MB/sec execute 237 sec latency 16.536 ms 2024-04-02T17:20:30.717 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:30 smithi154 ceph-mon[36479]: pgmap v611: 97 pgs: 97 active+clean; 64 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-02T17:20:30.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:30 smithi059 ceph-mon[31664]: pgmap v611: 97 pgs: 97 active+clean; 64 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-02T17:20:30.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:30 smithi164 ceph-mon[36499]: pgmap v611: 97 pgs: 97 active+clean; 64 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-02T17:20:31.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 202428 3.21 MB/sec execute 238 sec latency 23.761 ms 2024-04-02T17:20:32.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 202668 3.20 MB/sec execute 239 sec latency 22.386 ms 2024-04-02T17:20:33.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 203476 3.19 MB/sec execute 240 sec latency 34.720 ms 2024-04-02T17:20:33.152 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:32 smithi154 ceph-mon[36479]: pgmap v612: 97 pgs: 97 active+clean; 64 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T17:20:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:32 smithi059 ceph-mon[31664]: pgmap v612: 97 pgs: 97 active+clean; 64 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T17:20:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:32 smithi164 ceph-mon[36499]: pgmap v612: 97 pgs: 97 active+clean; 64 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 -- 172.21.15.59:0/3620625858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0104f50 msgr2=0x7f04e01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 --2- 172.21.15.59:0/3620625858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0104f50 0x7f04e01053d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009900 tx=0x7f04cc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 -- 172.21.15.59:0/3620625858 shutdown_connections 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 --2- 172.21.15.59:0/3620625858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 0x7f04e010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 --2- 172.21.15.59:0/3620625858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0104f50 0x7f04e01053d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 --2- 172.21.15.59:0/3620625858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0103d60 0x7f04e0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.698+0000 7f04e6200640 1 -- 172.21.15.59:0/3620625858 >> 172.21.15.59:0/3620625858 conn(0x7f04e00ff2f0 msgr2=0x7f04e0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.699+0000 7f04e6200640 1 -- 172.21.15.59:0/3620625858 shutdown_connections 2024-04-02T17:20:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.699+0000 7f04e6200640 1 -- 172.21.15.59:0/3620625858 wait complete. 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 Processor -- start 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 -- start start 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 0x7f04e019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 0x7f04e01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010fa30 con 0x7f04e0104f50 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010f8b0 con 0x7f04e0103d60 2024-04-02T17:20:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04e6200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04e010fbb0 con 0x7f04e0105910 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04df400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04dea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 0x7f04e019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04df400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48478/0 (socket says 172.21.15.59:48478) 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04df400640 1 -- 172.21.15.59:0/3109828390 learned_addr learned my addr 172.21.15.59:0/3109828390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.700+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 0x7f04e01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 -- 172.21.15.59:0/3109828390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 msgr2=0x7f04e01a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 0x7f04e01a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 -- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 msgr2=0x7f04e019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 0x7f04e019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 -- 172.21.15.59:0/3109828390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e01a1ba0 con 0x7f04e0103d60 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04dea00640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 0x7f04e019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04df400640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f04d000ebc0 tx=0x7f04d000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04d7e00640 1 -- 172.21.15.59:0/3109828390 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0003b70 con 0x7f04e0103d60 2024-04-02T17:20:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04d7e00640 1 -- 172.21.15.59:0/3109828390 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0003d10 con 0x7f04e0103d60 2024-04-02T17:20:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.701+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e01a1e90 con 0x7f04e0103d60 2024-04-02T17:20:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.702+0000 7f04d7e00640 1 -- 172.21.15.59:0/3109828390 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0019980 con 0x7f04e0103d60 2024-04-02T17:20:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.702+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a2300 con 0x7f04e0103d60 2024-04-02T17:20:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.705+0000 7f04d7e00640 1 -- 172.21.15.59:0/3109828390 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f04d0019b20 con 0x7f04e0103d60 2024-04-02T17:20:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.705+0000 7f04e8e19740 1 Processor -- start 2024-04-02T17:20:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.705+0000 7f04e8e19740 1 -- start start 2024-04-02T17:20:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 0x55a0002883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a00028aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a0001e8d70 con 0x55a0002838c0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a0001912a0 con 0x55a000288a30 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04e8e19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a0001e4d20 con 0x55a000282ed0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41248/0 (socket says 172.21.15.59:41248) 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.706+0000 7f04dfe00640 1 -- 172.21.15.59:0/1061086977 learned_addr learned my addr 172.21.15.59:0/1061086977 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dea00640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a00028aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04df400640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 0x55a0002883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 -- 172.21.15.59:0/1061086977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 msgr2=0x55a00028aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a00028aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 -- 172.21.15.59:0/1061086977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 msgr2=0x55a0002883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 0x55a0002883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 -- 172.21.15.59:0/1061086977 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0001a5840 con 0x55a000282ed0 2024-04-02T17:20:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04df400640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 0x55a0002883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04d7e00640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 0x7f04c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04d7e00640 1 -- 172.21.15.59:0/3109828390 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f04d00d0c40 con 0x7f04e0103d60 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04dfe00640 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009ca0 tx=0x7f04d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bbd64650f206af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.707+0000 7f04df400640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 0x7f04c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.708+0000 7f04cbe00640 1 -- 172.21.15.59:0/1061086977 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d801e070 con 0x55a000282ed0 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.708+0000 7f04cbe00640 1 -- 172.21.15.59:0/1061086977 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d8009e50 con 0x55a000282ed0 2024-04-02T17:20:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.708+0000 7f04cbe00640 1 -- 172.21.15.59:0/1061086977 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d800eac0 con 0x55a000282ed0 2024-04-02T17:20:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- 172.21.15.59:0/1061086977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 msgr2=0x55a0002832f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009ca0 tx=0x7f04d8011810 comp rx=0 tx=0).stop 2024-04-02T17:20:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04df400640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 0x7f04c40a6190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009270 tx=0x7f04d000cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- 172.21.15.59:0/1061086977 shutdown_connections 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0002838c0 0x55a0002883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a00028aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1061086977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a0002832f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- 172.21.15.59:0/1061086977 >> 172.21.15.59:0/1061086977 conn(0x55a00027e440 msgr2=0x55a00027e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- 172.21.15.59:0/1061086977 shutdown_connections 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- 172.21.15.59:0/1061086977 wait complete. 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.709+0000 7f04e8e19740 1 -- start start 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 Processor -- start 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000282ed0 0x55a00028d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a000294970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a00028b9e0 con 0x55a000282ed0 2024-04-02T17:20:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a0001a4660 con 0x55a000288a30 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a00028bb60 con 0x55a0002838c0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04e8e19740 10 client.0 _ll_register_callbacks cb 0x55a0001a4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.710+0000 7f04dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41264/0 (socket says 172.21.15.59:41264) 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 -- 172.21.15.59:0/810532249 learned_addr learned my addr 172.21.15.59:0/810532249 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dfe00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000282ed0 0x55a00028d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dfe00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a000294970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 -- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 msgr2=0x55a000294970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a000294970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 -- 172.21.15.59:0/810532249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000282ed0 msgr2=0x55a00028d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000282ed0 0x55a00028d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 -- 172.21.15.59:0/810532249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0001b3e80 con 0x55a0002838c0 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dfe00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a000294970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:20:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04dea00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5b76fe1e566bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.711+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc005d50 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f04cc004ed0 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d8009e50 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04cc02cd30 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 10 client.26557 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559fffefd210 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d8009e50 con 0x55a0002838c0 2024-04-02T17:20:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f04cc053050 con 0x55a0002838c0 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 10 client.26557 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 20 client.26557 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.712+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a000299170 0x55a00029b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.713+0000 7f04e8e19740 4 client.26557 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.713+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a000198cd0 con 0x55a000299170 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.713+0000 7f04dfe00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a000299170 0x55a00029b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.713+0000 7f04dfe00640 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a000299170 0x55a00029b580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a64cb4af52ae3e0 server_cookie=a4b388e5faf5fa9e in_seq=0 out_seq=0 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.713+0000 7f04c8c00640 10 client.26557 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04c8c00640 1 -- 172.21.15.59:0/810532249 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a000198cd0 con 0x55a000299170 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04c8c00640 10 client.26557 handle_command_reply: tid=0 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 client.26557 shutdown 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 msgr2=0x55a00028d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006010 tx=0x7f04cc0048f0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 shutdown_connections 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a000299170 0x55a00029b580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000288a30 0x55a000294970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a00028d930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 --2- 172.21.15.59:0/810532249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000282ed0 0x55a00028d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04c8c00640 0 client.26557 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.714+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 >> 172.21.15.59:0/810532249 conn(0x55a00027e440 msgr2=0x55a00027d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.715+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 shutdown_connections 2024-04-02T17:20:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.715+0000 7f04e8e19740 1 -- 172.21.15.59:0/810532249 wait complete. 2024-04-02T17:20:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.715+0000 7f04e8e19740 20 client.26557 trim_cache size 0 max 16384 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.731+0000 7f04e8e19740 1 Processor -- start 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.731+0000 7f04e8e19740 1 -- start start 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000282ed0 0x55a0002792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a00027e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ab2a0 con 0x55a000288a30 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01acde0 con 0x55a000282ed0 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04e8e19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ab970 con 0x55a0002838c0 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000282ed0 0x55a0002792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41268/0 (socket says 172.21.15.59:41268) 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 -- 172.21.15.59:0/1122010334 learned_addr learned my addr 172.21.15.59:0/1122010334 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dfe00640 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a00027e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 -- 172.21.15.59:0/1122010334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000282ed0 msgr2=0x55a0002792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000282ed0 0x55a0002792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 -- 172.21.15.59:0/1122010334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 msgr2=0x55a00027e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a00027e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.732+0000 7f04dea00640 1 -- 172.21.15.59:0/1122010334 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffffc9620 con 0x55a0002838c0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04dea00640 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02c170 tx=0x7f04cc02c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74497383b359afaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04cb400640 1 -- 172.21.15.59:0/1122010334 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02cd30 con 0x55a0002838c0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04cb400640 1 -- 172.21.15.59:0/1122010334 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc059020 con 0x55a0002838c0 2024-04-02T17:20:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04cb400640 1 -- 172.21.15.59:0/1122010334 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0053b0 con 0x55a0002838c0 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04e8e19740 1 -- 172.21.15.59:0/1122010334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 msgr2=0x55a000294930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.733+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02c170 tx=0x7f04cc02c2b0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 -- 172.21.15.59:0/1122010334 shutdown_connections 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a00027e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a0002838c0 0x55a000294930 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 --2- 172.21.15.59:0/1122010334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a000282ed0 0x55a0002792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 -- 172.21.15.59:0/1122010334 >> 172.21.15.59:0/1122010334 conn(0x55a00027e440 msgr2=0x55a00028bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 -- 172.21.15.59:0/1122010334 shutdown_connections 2024-04-02T17:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 -- 172.21.15.59:0/1122010334 wait complete. 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 -- start start 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.734+0000 7f04e8e19740 1 Processor -- start 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a000291c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a0002930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ac710 con 0x55a000288a30 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ac040 con 0x55a0002838c0 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01a9280 con 0x55a000282ed0 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04e8e19740 10 client.0 _ll_register_callbacks cb 0x7f04cc004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48526/0 (socket says 172.21.15.59:48526) 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 -- 172.21.15.59:0/3567852699 learned_addr learned my addr 172.21.15.59:0/3567852699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a000291c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a0002930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 -- 172.21.15.59:0/3567852699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 msgr2=0x55a000291c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a000291c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 -- 172.21.15.59:0/3567852699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 msgr2=0x55a0002930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a0002930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dea00640 1 -- 172.21.15.59:0/3567852699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0002936a0 con 0x55a0002838c0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.735+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a0002930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04dea00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005ab0 tx=0x7f04cc005100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa6015886acdcee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02cd30 con 0x55a0002838c0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc059020 con 0x55a0002838c0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f04cc005500 con 0x55a0002838c0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04cc009520 con 0x55a0002838c0 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04e8e19740 10 client.26540 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0002949b0 con 0x55a0002838c0 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.736+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc059020 con 0x55a0002838c0 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f04cc053070 con 0x55a0002838c0 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 10 client.26540 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 20 client.26540 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a0002ffb90 0x55a000301fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 4 client.26540 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0001f62b0 con 0x55a0002ffb90 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.737+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a0002ffb90 0x55a000301fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.738+0000 7f04c8c00640 10 client.26540 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.738+0000 7f04dfe00640 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a0002ffb90 0x55a000301fb0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3282d2638b7a725f server_cookie=97e5441bba226b72 in_seq=0 out_seq=0 2024-04-02T17:20:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04c8c00640 1 -- 172.21.15.59:0/3567852699 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0001f62b0 con 0x55a0002ffb90 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04c8c00640 10 client.26540 handle_command_reply: tid=0 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 client.26540 shutdown 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 msgr2=0x55a000292160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005ab0 tx=0x7f04cc005100 comp rx=0 tx=0).stop 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 shutdown_connections 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a0002ffb90 0x55a000301fb0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a000288a30 0x55a0002930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a0002838c0 0x55a000292160 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 --2- 172.21.15.59:0/3567852699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a000282ed0 0x55a000291c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04c8c00640 0 client.26540 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 >> 172.21.15.59:0/3567852699 conn(0x55a00027e440 msgr2=0x55a00027e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 shutdown_connections 2024-04-02T17:20:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 1 -- 172.21.15.59:0/3567852699 wait complete. 2024-04-02T17:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.773+0000 7f04e8e19740 20 client.26540 trim_cache size 0 max 16384 2024-04-02T17:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 msgr2=0x7f04c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 0x7f04c40a6190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009270 tx=0x7f04d000cf90 comp rx=0 tx=0).stop 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 msgr2=0x7f04e0199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f04d000ebc0 tx=0x7f04d000c6a0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 shutdown_connections 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f04c40a3cd0 0x7f04c40a6190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04e0104f50 0x7f04e019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04e0105910 0x7f04e01a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 --2- 172.21.15.59:0/3109828390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0103d60 0x7f04e0199f80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 >> 172.21.15.59:0/3109828390 conn(0x7f04e00ff2f0 msgr2=0x7f04e0108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.774+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 shutdown_connections 2024-04-02T17:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:33.775+0000 7f04e6200640 1 -- 172.21.15.59:0/3109828390 wait complete. 2024-04-02T17:20:34.077 INFO:tasks.workunit.client.0.smithi059.stdout: 1 205394 3.23 MB/sec execute 241 sec latency 32.259 ms 2024-04-02T17:20:35.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 206099 3.24 MB/sec execute 242 sec latency 29.215 ms 2024-04-02T17:20:35.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:34 smithi154 ceph-mon[36479]: pgmap v613: 97 pgs: 97 active+clean; 65 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-02T17:20:35.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:34 smithi059 ceph-mon[31664]: pgmap v613: 97 pgs: 97 active+clean; 65 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-02T17:20:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:34 smithi164 ceph-mon[36499]: pgmap v613: 97 pgs: 97 active+clean; 65 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-04-02T17:20:36.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 206519 3.23 MB/sec execute 243 sec latency 22.385 ms 2024-04-02T17:20:37.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 207933 3.24 MB/sec execute 244 sec latency 27.465 ms 2024-04-02T17:20:37.100 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:36 smithi059 ceph-mon[31664]: pgmap v614: 97 pgs: 97 active+clean; 65 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:20:37.124 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:36 smithi164 ceph-mon[36499]: pgmap v614: 97 pgs: 97 active+clean; 65 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:20:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:36 smithi154 ceph-mon[36479]: pgmap v614: 97 pgs: 97 active+clean; 65 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:20:38.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 209238 3.27 MB/sec execute 245 sec latency 51.058 ms 2024-04-02T17:20:39.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 209736 3.26 MB/sec execute 246 sec latency 26.241 ms 2024-04-02T17:20:39.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:38 smithi154 ceph-mon[36479]: pgmap v615: 97 pgs: 97 active+clean; 65 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:20:39.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:38 smithi059 ceph-mon[31664]: pgmap v615: 97 pgs: 97 active+clean; 65 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:20:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:38 smithi164 ceph-mon[36499]: pgmap v615: 97 pgs: 97 active+clean; 65 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:20:40.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 210558 3.26 MB/sec execute 247 sec latency 22.368 ms 2024-04-02T17:20:41.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 212464 3.29 MB/sec execute 248 sec latency 16.578 ms 2024-04-02T17:20:41.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:40 smithi154 ceph-mon[36479]: pgmap v616: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-04-02T17:20:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:40 smithi059 ceph-mon[31664]: pgmap v616: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-04-02T17:20:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:40 smithi164 ceph-mon[36499]: pgmap v616: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-04-02T17:20:42.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 213171 3.30 MB/sec execute 249 sec latency 25.873 ms 2024-04-02T17:20:43.078 INFO:tasks.workunit.client.0.smithi059.stdout: 1 213551 3.29 MB/sec execute 250 sec latency 55.933 ms 2024-04-02T17:20:43.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:42 smithi154 ceph-mon[36479]: pgmap v617: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:20:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:42 smithi059 ceph-mon[31664]: pgmap v617: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:20:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:42 smithi164 ceph-mon[36499]: pgmap v617: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:20:44.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 215316 3.30 MB/sec execute 251 sec latency 76.019 ms 2024-04-02T17:20:45.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 216643 3.34 MB/sec execute 252 sec latency 26.397 ms 2024-04-02T17:20:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:44 smithi154 ceph-mon[36479]: pgmap v618: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-04-02T17:20:45.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:44 smithi059 ceph-mon[31664]: pgmap v618: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-04-02T17:20:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:44 smithi164 ceph-mon[36499]: pgmap v618: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-04-02T17:20:46.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 216885 3.32 MB/sec execute 253 sec latency 24.604 ms 2024-04-02T17:20:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.931+0000 7fed0bc00640 1 -- 172.21.15.59:0/3509497276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed04100180 msgr2=0x7fed041072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.931+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3509497276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed04100180 0x7fed041072d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecec009900 tx=0x7fecec02bc40 comp rx=0 tx=0).stop 2024-04-02T17:20:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 -- 172.21.15.59:0/3509497276 shutdown_connections 2024-04-02T17:20:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3509497276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed04100180 0x7fed041072d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3509497276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed04109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3509497276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed040ff790 0x7fed040ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 -- 172.21.15.59:0/3509497276 >> 172.21.15.59:0/3509497276 conn(0x7fed040fb2e0 msgr2=0x7fed040fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 -- 172.21.15.59:0/3509497276 shutdown_connections 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.932+0000 7fed0bc00640 1 -- 172.21.15.59:0/3509497276 wait complete. 2024-04-02T17:20:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.933+0000 7fed0bc00640 1 Processor -- start 2024-04-02T17:20:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 -- start start 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 0x7fed0419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 0x7fed0419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed041a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410d7d0 con 0x7fed040ff790 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410d650 con 0x7fed04107810 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.934+0000 7fed0bc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410d950 con 0x7fed04100180 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 0x7fed0419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed041a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 0x7fed0419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59946/0 (socket says 172.21.15.59:59946) 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fecfbe00640 1 -- 172.21.15.59:0/3522928932 learned_addr learned my addr 172.21.15.59:0/3522928932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09200640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 0x7fed0419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 -- 172.21.15.59:0/3522928932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 msgr2=0x7fed0419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 0x7fed0419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 -- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 msgr2=0x7fed0419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 0x7fed0419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09c00640 1 -- 172.21.15.59:0/3522928932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041a1c00 con 0x7fed04107810 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.935+0000 7fed09200640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 0x7fed0419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed09c00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed041a1590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00e940 tx=0x7fecfc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed02000640 1 -- 172.21.15.59:0/3522928932 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00ccc0 con 0x7fed04107810 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed02000640 1 -- 172.21.15.59:0/3522928932 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecfc005020 con 0x7fed04107810 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041a1ef0 con 0x7fed04107810 2024-04-02T17:20:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed02000640 1 -- 172.21.15.59:0/3522928932 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004a00 con 0x7fed04107810 2024-04-02T17:20:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.936+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a23b0 con 0x7fed04107810 2024-04-02T17:20:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.939+0000 7fed02000640 1 -- 172.21.15.59:0/3522928932 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fecfc0051d0 con 0x7fed04107810 2024-04-02T17:20:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.940+0000 7fed0e981740 1 Processor -- start 2024-04-02T17:20:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.940+0000 7fed0e981740 1 -- start start 2024-04-02T17:20:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.940+0000 7fed0e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e5902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.940+0000 7fed0e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.940+0000 7fed0e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e597ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed0e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56335e4f5d70 con 0x56335e58fed0 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed0e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56335e49e2a0 con 0x56335e595a30 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed0e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56335e4f1d20 con 0x56335e5908c0 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e5902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59958/0 (socket says 172.21.15.59:59958) 2024-04-02T17:20:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 -- 172.21.15.59:0/1907740568 learned_addr learned my addr 172.21.15.59:0/1907740568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fecfbe00640 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e597ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 -- 172.21.15.59:0/1907740568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 msgr2=0x56335e597ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e597ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 -- 172.21.15.59:0/1907740568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 msgr2=0x56335e5902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e5902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed09200640 1 -- 172.21.15.59:0/1907740568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335e4b2840 con 0x56335e5908c0 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fed02000640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 0x7fece80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.941+0000 7fecfbe00640 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e597ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fed02000640 1 -- 172.21.15.59:0/3522928932 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fecfc0d0440 con 0x7fed04107810 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fed09200640 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fecf400f390 tx=0x7fecf400f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109d4a3dca75829e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fecfbe00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 0x7fece80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fecfa000640 1 -- 172.21.15.59:0/1907740568 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4003b70 con 0x56335e5908c0 2024-04-02T17:20:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fecfa000640 1 -- 172.21.15.59:0/1907740568 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4003d10 con 0x56335e5908c0 2024-04-02T17:20:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.942+0000 7fecfa000640 1 -- 172.21.15.59:0/1907740568 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4011ac0 con 0x56335e5908c0 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.943+0000 7fecfbe00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 0x7fece80a6190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fecec000c00 tx=0x7fecec005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.943+0000 7fed0e981740 1 -- 172.21.15.59:0/1907740568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 msgr2=0x56335e5953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.943+0000 7fed0e981740 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fecf400f390 tx=0x7fecf400f850 comp rx=0 tx=0).stop 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.943+0000 7fed0e981740 1 -- 172.21.15.59:0/1907740568 shutdown_connections 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e597ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e5953c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 --2- 172.21.15.59:0/1907740568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e5902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 -- 172.21.15.59:0/1907740568 >> 172.21.15.59:0/1907740568 conn(0x56335e58b440 msgr2=0x56335e58b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 -- 172.21.15.59:0/1907740568 shutdown_connections 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 -- 172.21.15.59:0/1907740568 wait complete. 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 -- start start 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.944+0000 7fed0e981740 1 Processor -- start 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e5a1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56335e5989e0 con 0x56335e58fed0 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56335e4b1660 con 0x56335e595a30 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56335e598b60 con 0x56335e5908c0 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.945+0000 7fed0e981740 10 client.0 _ll_register_callbacks cb 0x56335e4b1050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e5a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41268/0 (socket says 172.21.15.59:41268) 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09200640 1 -- 172.21.15.59:0/4114319119 learned_addr learned my addr 172.21.15.59:0/4114319119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 -- 172.21.15.59:0/4114319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 msgr2=0x56335e59a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 -- 172.21.15.59:0/4114319119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 msgr2=0x56335e59a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 -- 172.21.15.59:0/4114319119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335e4c0e80 con 0x56335e595a30 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fed09c00640 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e5a1970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecfc099030 tx=0x7fecfc005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18269279381ade0c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.946+0000 7fecf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01c040 con 0x56335e595a30 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fecfc0a4e40 con 0x56335e595a30 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf4003d10 con 0x56335e595a30 2024-04-02T17:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fecfc0a0a10 con 0x56335e595a30 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fed0e981740 10 client.26552 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56335e20a210 con 0x56335e595a30 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.947+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4003d10 con 0x56335e595a30 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fecfc09bab0 con 0x56335e595a30 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 10 client.26552 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 20 client.26552 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e5a62c0 0x56335e5a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 4 client.26552 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335e4a5cd0 con 0x56335e5a62c0 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.948+0000 7fed09200640 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e5a62c0 0x56335e5a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.949+0000 7fed09200640 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e5a62c0 0x56335e5a8730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7dbc5109b6238a5 server_cookie=c06652aaf9e47aa0 in_seq=0 out_seq=0 2024-04-02T17:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.949+0000 7fecf2a00640 10 client.26552 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fecf2a00640 1 -- 172.21.15.59:0/4114319119 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56335e4a5cd0 con 0x56335e5a62c0 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fecf2a00640 10 client.26552 handle_command_reply: tid=0 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 client.26552 shutdown 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 msgr2=0x56335e5a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e5a1970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fecfc099030 tx=0x7fecfc005620 comp rx=0 tx=0).stop 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 shutdown_connections 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e5a62c0 0x56335e5a8730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e5a1970 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fecf2a00640 0 client.26552 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 --2- 172.21.15.59:0/4114319119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 >> 172.21.15.59:0/4114319119 conn(0x56335e58b440 msgr2=0x56335e58a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 shutdown_connections 2024-04-02T17:20:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.950+0000 7fed0e981740 1 -- 172.21.15.59:0/4114319119 wait complete. 2024-04-02T17:20:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.951+0000 7fed0e981740 20 client.26552 trim_cache size 0 max 16384 2024-04-02T17:20:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 Processor -- start 2024-04-02T17:20:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 -- start start 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e595a30 0x56335e598f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed041ab2d0 con 0x56335e595a30 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed041ace10 con 0x56335e5908c0 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed0e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed041ab9a0 con 0x56335e58fed0 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.970+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59968/0 (socket says 172.21.15.59:59968) 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e595a30 0x56335e598f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44692/0 (socket says 172.21.15.59:44692) 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 -- 172.21.15.59:0/3179000962 learned_addr learned my addr 172.21.15.59:0/3179000962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 -- 172.21.15.59:0/3179000962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 msgr2=0x56335e59a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 -- 172.21.15.59:0/3179000962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e595a30 msgr2=0x56335e598f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e595a30 0x56335e598f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 -- 172.21.15.59:0/3179000962 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335e2d6620 con 0x56335e58fed0 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fed09200640 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fecf4019000 tx=0x7fecf40092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6500e3fca6ee4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fecf9600640 1 -- 172.21.15.59:0/3179000962 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004cc0 con 0x56335e58fed0 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.971+0000 7fecf9600640 1 -- 172.21.15.59:0/3179000962 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4011ae0 con 0x56335e58fed0 2024-04-02T17:20:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fecf9600640 1 -- 172.21.15.59:0/3179000962 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4029710 con 0x56335e58fed0 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 -- 172.21.15.59:0/3179000962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 msgr2=0x56335e59a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fecf4019000 tx=0x7fecf40092a0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 -- 172.21.15.59:0/3179000962 shutdown_connections 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e595a30 0x56335e598f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e5908c0 0x56335e59a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 --2- 172.21.15.59:0/3179000962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59a3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 -- 172.21.15.59:0/3179000962 >> 172.21.15.59:0/3179000962 conn(0x56335e58b440 msgr2=0x56335e58b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 -- 172.21.15.59:0/3179000962 shutdown_connections 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.972+0000 7fed0e981740 1 -- 172.21.15.59:0/3179000962 wait complete. 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.973+0000 7fed0e981740 1 -- start start 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.973+0000 7fed0e981740 1 Processor -- start 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e5908c0 0x56335e59e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e59f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed041ac740 con 0x56335e5908c0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed041ac070 con 0x56335e595a30 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed041a92b0 con 0x56335e58fed0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed0e981740 10 client.0 _ll_register_callbacks cb 0x7fecfc09c250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed09200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59970/0 (socket says 172.21.15.59:59970) 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fed09200640 1 -- 172.21.15.59:0/1801034498 learned_addr learned my addr 172.21.15.59:0/1801034498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.974+0000 7fecfbe00640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e5908c0 0x56335e59e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09c00640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e59f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 -- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 msgr2=0x56335e59f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e59f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 -- 172.21.15.59:0/1801034498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e5908c0 msgr2=0x56335e59e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e5908c0 0x56335e59e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 -- 172.21.15.59:0/1801034498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335e59fcc0 con 0x56335e58fed0 2024-04-02T17:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09c00640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e59f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fed09200640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009f30 tx=0x7fecf4009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177778c9eb58a21d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.975+0000 7fecf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf402f050 con 0x56335e58fed0 2024-04-02T17:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fecf4011570 con 0x56335e58fed0 2024-04-02T17:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf4011ae0 con 0x56335e58fed0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fecf4011770 con 0x56335e58fed0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fed0e981740 10 client.26578 fetch_fsmap learned FSMap version 33 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.976+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56335e5a0fd0 con 0x56335e58fed0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4011ae0 con 0x56335e58fed0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fecf40054b0 con 0x56335e58fed0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 10 client.26578 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 10 client.26578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 20 client.26578 populate_metadata read hostname 'smithi059' 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e60cc40 0x56335e60f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 4 client.26578 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56335e5032b0 con 0x56335e60cc40 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.977+0000 7fecfbe00640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e60cc40 0x56335e60f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.978+0000 7fecfbe00640 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e60cc40 0x56335e60f030 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4c13a9080d867a server_cookie=f12a088738875588 in_seq=0 out_seq=0 2024-04-02T17:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:46.978+0000 7fecf2a00640 10 client.26578 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fecf2a00640 1 -- 172.21.15.59:0/1801034498 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56335e5032b0 con 0x56335e60cc40 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fecf2a00640 10 client.26578 handle_command_reply: tid=0 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 client.26578 shutdown 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 msgr2=0x56335e59e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009f30 tx=0x7fecf4009cc0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 shutdown_connections 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56335e60cc40 0x56335e60f030 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56335e5908c0 0x56335e59e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56335e595a30 0x56335e59f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 --2- 172.21.15.59:0/1801034498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56335e58fed0 0x56335e59e180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fecf2a00640 0 client.26578 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 >> 172.21.15.59:0/1801034498 conn(0x56335e58b440 msgr2=0x56335e591e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 shutdown_connections 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.010+0000 7fed0e981740 1 -- 172.21.15.59:0/1801034498 wait complete. 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0e981740 20 client.26578 trim_cache size 0 max 16384 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:20:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 msgr2=0x7fece80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 0x7fece80a6190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fecec000c00 tx=0x7fecec005b50 comp rx=0 tx=0).stop 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 msgr2=0x7fed041a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed041a1590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00e940 tx=0x7fecfc00ee00 comp rx=0 tx=0).stop 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 shutdown_connections 2024-04-02T17:20:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed04107810 0x7fed041a1590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed04100180 0x7fed0419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fece80a3cd0 0x7fece80a6190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 --2- 172.21.15.59:0/3522928932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed040ff790 0x7fed0419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.011+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 >> 172.21.15.59:0/3522928932 conn(0x7fed040fb2e0 msgr2=0x7fed040fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.012+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 shutdown_connections 2024-04-02T17:20:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:20:47.012+0000 7fed0bc00640 1 -- 172.21.15.59:0/3522928932 wait complete. 2024-04-02T17:20:47.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 217586 3.32 MB/sec execute 254 sec latency 29.697 ms 2024-04-02T17:20:47.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:46 smithi154 ceph-mon[36479]: pgmap v619: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-02T17:20:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:46 smithi059 ceph-mon[31664]: pgmap v619: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-02T17:20:47.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:46 smithi164 ceph-mon[36499]: pgmap v619: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-02T17:20:48.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 219342 3.35 MB/sec execute 255 sec latency 16.206 ms 2024-04-02T17:20:49.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 220240 3.36 MB/sec execute 256 sec latency 43.703 ms 2024-04-02T17:20:49.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:48 smithi154 ceph-mon[36479]: pgmap v620: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:20:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:48 smithi059 ceph-mon[31664]: pgmap v620: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:20:49.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:48 smithi164 ceph-mon[36499]: pgmap v620: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:20:50.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 220698 3.35 MB/sec execute 257 sec latency 24.237 ms 2024-04-02T17:20:51.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 222057 3.36 MB/sec execute 258 sec latency 24.472 ms 2024-04-02T17:20:51.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:50 smithi154 ceph-mon[36479]: pgmap v621: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-02T17:20:51.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:50 smithi059 ceph-mon[31664]: pgmap v621: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-02T17:20:51.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:50 smithi164 ceph-mon[36499]: pgmap v621: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-02T17:20:52.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 223327 3.38 MB/sec execute 259 sec latency 18.072 ms 2024-04-02T17:20:53.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 223898 3.38 MB/sec execute 260 sec latency 25.111 ms 2024-04-02T17:20:53.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:52 smithi154 ceph-mon[36479]: pgmap v622: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:20:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:52 smithi059 ceph-mon[31664]: pgmap v622: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:20:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:52 smithi164 ceph-mon[36499]: pgmap v622: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:20:54.079 INFO:tasks.workunit.client.0.smithi059.stdout: 1 224492 3.37 MB/sec execute 261 sec latency 40.291 ms 2024-04-02T17:20:55.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 226011 3.39 MB/sec execute 262 sec latency 23.412 ms 2024-04-02T17:20:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:54 smithi154 ceph-mon[36479]: pgmap v623: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:55.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:54 smithi059 ceph-mon[31664]: pgmap v623: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:54 smithi164 ceph-mon[36499]: pgmap v623: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-04-02T17:20:56.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 227267 3.42 MB/sec execute 263 sec latency 19.437 ms 2024-04-02T17:20:57.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 227527 3.40 MB/sec execute 264 sec latency 39.261 ms 2024-04-02T17:20:57.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:56 smithi154 ceph-mon[36479]: pgmap v624: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:20:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:56 smithi059 ceph-mon[31664]: pgmap v624: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:20:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:56 smithi164 ceph-mon[36499]: pgmap v624: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:20:58.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 228243 3.40 MB/sec execute 265 sec latency 21.971 ms 2024-04-02T17:20:59.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 229949 3.42 MB/sec execute 266 sec latency 18.393 ms 2024-04-02T17:20:59.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:20:58 smithi154 ceph-mon[36479]: pgmap v625: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:20:59.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:20:58 smithi059 ceph-mon[31664]: pgmap v625: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:20:59.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:20:58 smithi164 ceph-mon[36499]: pgmap v625: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:21:00.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 230864 3.44 MB/sec execute 267 sec latency 32.264 ms 2024-04-02T17:21:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 -- 172.21.15.59:0/743141055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 msgr2=0x7f5a1c104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 --2- 172.21.15.59:0/743141055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c104990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009900 tx=0x7f5a0c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 -- 172.21.15.59:0/743141055 shutdown_connections 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 --2- 172.21.15.59:0/743141055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c107800 0x7f5a1c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 --2- 172.21.15.59:0/743141055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c104ed0 0x7f5a1c1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 --2- 172.21.15.59:0/743141055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c104990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 -- 172.21.15.59:0/743141055 >> 172.21.15.59:0/743141055 conn(0x7f5a1c0fbf90 msgr2=0x7f5a1c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 -- 172.21.15.59:0/743141055 shutdown_connections 2024-04-02T17:21:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.169+0000 7f5a22000640 1 -- 172.21.15.59:0/743141055 wait complete. 2024-04-02T17:21:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.170+0000 7f5a22000640 1 Processor -- start 2024-04-02T17:21:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.170+0000 7f5a22000640 1 -- start start 2024-04-02T17:21:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c104ed0 0x7f5a1c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c107800 0x7f5a1c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10d7f0 con 0x7f5a1c104ed0 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10d670 con 0x7f5a1c1025a0 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a22000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10d970 con 0x7f5a1c107800 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33306/0 (socket says 172.21.15.59:33306) 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 -- 172.21.15.59:0/1748868657 learned_addr learned my addr 172.21.15.59:0/1748868657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1be00640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c107800 0x7f5a1c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c104ed0 0x7f5a1c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 -- 172.21.15.59:0/1748868657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c107800 msgr2=0x7f5a1c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c107800 0x7f5a1c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 -- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c104ed0 msgr2=0x7f5a1c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c104ed0 0x7f5a1c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.171+0000 7f5a1b400640 1 -- 172.21.15.59:0/1748868657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c19f9d0 con 0x7f5a1c1025a0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a1b400640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005ab0 tx=0x7f5a0c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a13e00640 1 -- 172.21.15.59:0/1748868657 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c043020 con 0x7f5a1c1025a0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a13e00640 1 -- 172.21.15.59:0/1748868657 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0054e0 con 0x7f5a1c1025a0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a13e00640 1 -- 172.21.15.59:0/1748868657 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c003ce0 con 0x7f5a1c1025a0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c19fc60 con 0x7f5a1c1025a0 2024-04-02T17:21:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.172+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1a01b0 con 0x7f5a1c1025a0 2024-04-02T17:21:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a13e00640 1 -- 172.21.15.59:0/1748868657 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a0c046020 con 0x7f5a1c1025a0 2024-04-02T17:21:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 Processor -- start 2024-04-02T17:21:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 -- start start 2024-04-02T17:21:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f3c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7e9dd70 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7e462a0 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.176+0000 7f5a24c63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7e99d20 con 0x55ebc7f41e40 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f3c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33312/0 (socket says 172.21.15.59:33312) 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 -- 172.21.15.59:0/2206869483 learned_addr learned my addr 172.21.15.59:0/2206869483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 -- 172.21.15.59:0/2206869483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 msgr2=0x55ebc7f442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 -- 172.21.15.59:0/2206869483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 msgr2=0x55ebc7f3c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f3c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1b400640 1 -- 172.21.15.59:0/2206869483 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc7e5a840 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.177+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a1b400640 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0f85b0 tx=0x7f5a0c0f86a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7c96d44732b6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a07e00640 1 -- 172.21.15.59:0/2206869483 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0f2e40 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a13e00640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 0x7f5a000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a07e00640 1 -- 172.21.15.59:0/2206869483 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0ff030 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a07e00640 1 -- 172.21.15.59:0/2206869483 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c104a50 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a13e00640 1 -- 172.21.15.59:0/1748868657 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5a0c0f2640 con 0x7f5a1c1025a0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.178+0000 7f5a1be00640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 0x7f5a000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 -- 172.21.15.59:0/2206869483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 msgr2=0x55ebc7f417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0f85b0 tx=0x7f5a0c0f86a0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 -- 172.21.15.59:0/2206869483 shutdown_connections 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f417d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 --2- 172.21.15.59:0/2206869483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f3c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 -- 172.21.15.59:0/2206869483 >> 172.21.15.59:0/2206869483 conn(0x55ebc7f33440 msgr2=0x55ebc7f33840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a1be00640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 0x7f5a000a6190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55ebc7f3d530 tx=0x7f5a14009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 -- 172.21.15.59:0/2206869483 shutdown_connections 2024-04-02T17:21:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.179+0000 7f5a24c63740 1 -- 172.21.15.59:0/2206869483 wait complete. 2024-04-02T17:21:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.180+0000 7f5a24c63740 1 -- start start 2024-04-02T17:21:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.180+0000 7f5a24c63740 1 Processor -- start 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7f44bb0 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7e59660 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ebc7f44d30 con 0x55ebc7f41e40 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a24c63740 10 client.0 _ll_register_callbacks cb 0x55ebc7e59050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33328/0 (socket says 172.21.15.59:33328) 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a1b400640 1 -- 172.21.15.59:0/404904145 learned_addr learned my addr 172.21.15.59:0/404904145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.181+0000 7f5a1be00640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 -- 172.21.15.59:0/404904145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 msgr2=0x55ebc7f4dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f4dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 -- 172.21.15.59:0/404904145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 msgr2=0x55ebc7f46aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f46aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 -- 172.21.15.59:0/404904145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc7e68e80 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1be00640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f4dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a1b400640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0c6fa0 tx=0x7f5a0c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3a0da267c2da93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a04c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0baa70 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c0ff030 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5a0c0bcb60 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c0bc9a0 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a24c63740 10 client.26567 fetch_fsmap learned FSMap version 33 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.182+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ebc7bb2210 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0ff030 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0c0bae40 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 10 client.26567 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 20 client.26567 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7f52150 0x55ebc7f54540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 4 client.26567 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:21:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebc7e4dcd0 con 0x55ebc7f52150 2024-04-02T17:21:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.183+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7f52150 0x55ebc7f54540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.184+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7f52150 0x55ebc7f54540 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46709851fa7f4c8e server_cookie=ad08bb3f7c5a5d89 in_seq=0 out_seq=0 2024-04-02T17:21:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.184+0000 7f5a04c00640 10 client.26567 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a04c00640 1 -- 172.21.15.59:0/404904145 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebc7e4dcd0 con 0x55ebc7f52150 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a04c00640 10 client.26567 handle_command_reply: tid=0 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 client.26567 shutdown 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 msgr2=0x55ebc7f46560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0c6fa0 tx=0x7f5a0c009260 comp rx=0 tx=0).stop 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 shutdown_connections 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7f52150 0x55ebc7f54540 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7f4dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 --2- 172.21.15.59:0/404904145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46560 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 >> 172.21.15.59:0/404904145 conn(0x55ebc7f33440 msgr2=0x55ebc7f32640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.185+0000 7f5a04c00640 0 client.26567 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.186+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 shutdown_connections 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.186+0000 7f5a24c63740 1 -- 172.21.15.59:0/404904145 wait complete. 2024-04-02T17:21:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.186+0000 7f5a24c63740 20 client.26567 trim_cache size 0 max 16384 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 Processor -- start 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 -- start start 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 0x55ebc7f31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a6e70 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a89b0 con 0x55ebc7f41e40 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a24c63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a7540 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.204+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33696/0 (socket says 172.21.15.59:33696) 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 -- 172.21.15.59:0/301467093 learned_addr learned my addr 172.21.15.59:0/301467093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1be00640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 0x55ebc7f31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 -- 172.21.15.59:0/301467093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f3ccd0 msgr2=0x55ebc7f4da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 -- 172.21.15.59:0/301467093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 msgr2=0x55ebc7f31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 0x55ebc7f31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 -- 172.21.15.59:0/301467093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebc7c7e620 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1be00640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 0x55ebc7f31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.205+0000 7f5a1b400640 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c0bc870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634dd16d86ab405 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.206+0000 7f5a07400640 1 -- 172.21.15.59:0/301467093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0be1c0 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.206+0000 7f5a07400640 1 -- 172.21.15.59:0/301467093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0ff070 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.206+0000 7f5a07400640 1 -- 172.21.15.59:0/301467093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0be3a0 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- 172.21.15.59:0/301467093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 msgr2=0x55ebc7f46880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c0bc870 comp rx=0 tx=0).stop 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- 172.21.15.59:0/301467093 shutdown_connections 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f41e40 0x55ebc7f31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 --2- 172.21.15.59:0/301467093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f46880 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- 172.21.15.59:0/301467093 >> 172.21.15.59:0/301467093 conn(0x55ebc7f33440 msgr2=0x55ebc7f46170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- 172.21.15.59:0/301467093 shutdown_connections 2024-04-02T17:21:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- 172.21.15.59:0/301467093 wait complete. 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.207+0000 7f5a24c63740 1 -- start start 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 Processor -- start 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7fb66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a82e0 con 0x55ebc7f3ccd0 2024-04-02T17:21:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a7c10 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1a4e50 con 0x55ebc7f41e40 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a24c63740 10 client.0 _ll_register_callbacks cb 0x7f5a0c0f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33344/0 (socket says 172.21.15.59:33344) 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a1b400640 1 -- 172.21.15.59:0/342391442 learned_addr learned my addr 172.21.15.59:0/342391442 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.208+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1be00640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7fb66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 -- 172.21.15.59:0/342391442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 msgr2=0x55ebc7fb66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7fb66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 -- 172.21.15.59:0/342391442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 msgr2=0x55ebc7f4ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 -- 172.21.15.59:0/342391442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebc7f4dc60 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1be00640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7fb66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:21:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a1b400640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0bac30 tx=0x7f5a0c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91783566901543e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a04c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0be630 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5a0c0bae40 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.209+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c0ff070 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c0be8b0 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 10 client.26573 fetch_fsmap learned FSMap version 33 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ebc7f4dc60 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0ff070 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0c0c7040 con 0x55ebc7f3c2e0 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 10 client.26573 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 10 client.26573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 20 client.26573 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7fb80e0 0x55ebc7fba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 4 client.26573 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:21:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebc7eab2b0 con 0x55ebc7fb80e0 2024-04-02T17:21:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.210+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7fb80e0 0x55ebc7fba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.211+0000 7f5a1aa00640 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7fb80e0 0x55ebc7fba4d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b04c4173fb912a9 server_cookie=7e67b1a999e5230c in_seq=0 out_seq=0 2024-04-02T17:21:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.211+0000 7f5a04c00640 10 client.26573 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a04c00640 1 -- 172.21.15.59:0/342391442 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebc7eab2b0 con 0x55ebc7fb80e0 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a04c00640 10 client.26573 handle_command_reply: tid=0 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 client.26573 shutdown 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 msgr2=0x55ebc7f4a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0bac30 tx=0x7f5a0c009560 comp rx=0 tx=0).stop 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 shutdown_connections 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ebc7fb80e0 0x55ebc7fba4d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebc7f3ccd0 0x55ebc7f4ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc7f41e40 0x55ebc7fb66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a04c00640 0 client.26573 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 --2- 172.21.15.59:0/342391442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebc7f3c2e0 0x55ebc7f4a650 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.243+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 >> 172.21.15.59:0/342391442 conn(0x55ebc7f33440 msgr2=0x55ebc7f366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.244+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 shutdown_connections 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.244+0000 7f5a24c63740 1 -- 172.21.15.59:0/342391442 wait complete. 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.244+0000 7f5a24c63740 20 client.26573 trim_cache size 0 max 16384 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:21:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 msgr2=0x7f5a000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 0x7f5a000a6190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55ebc7f3d530 tx=0x7f5a14009600 comp rx=0 tx=0).stop 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 msgr2=0x7f5a1c197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005ab0 tx=0x7f5a0c02c6c0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 shutdown_connections 2024-04-02T17:21:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5a000a3cd0 0x7f5a000a6190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a1c104ed0 0x7f5a1c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a1c107800 0x7f5a1c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 --2- 172.21.15.59:0/1748868657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a1c1025a0 0x7f5a1c197dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 >> 172.21.15.59:0/1748868657 conn(0x7f5a1c0fbf90 msgr2=0x7f5a1c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 shutdown_connections 2024-04-02T17:21:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:00.245+0000 7f5a22000640 1 -- 172.21.15.59:0/1748868657 wait complete. 2024-04-02T17:21:01.080 INFO:tasks.workunit.client.0.smithi059.stdout: 1 231184 3.42 MB/sec execute 268 sec latency 86.890 ms 2024-04-02T17:21:01.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:00 smithi154 ceph-mon[36479]: pgmap v626: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:21:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:00 smithi059 ceph-mon[31664]: pgmap v626: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:21:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:00 smithi164 ceph-mon[36499]: pgmap v626: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:21:02.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 232092 3.43 MB/sec execute 269 sec latency 25.013 ms 2024-04-02T17:21:03.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 233615 3.45 MB/sec execute 270 sec latency 18.556 ms 2024-04-02T17:21:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:02 smithi059 ceph-mon[31664]: pgmap v627: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-02T17:21:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:02 smithi164 ceph-mon[36499]: pgmap v627: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-02T17:21:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:02 smithi154 ceph-mon[36479]: pgmap v627: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-02T17:21:04.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 234451 3.46 MB/sec execute 271 sec latency 59.926 ms 2024-04-02T17:21:05.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 234869 3.44 MB/sec execute 272 sec latency 36.289 ms 2024-04-02T17:21:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:04 smithi059 ceph-mon[31664]: pgmap v628: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:21:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:04 smithi164 ceph-mon[36499]: pgmap v628: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:21:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:04 smithi154 ceph-mon[36479]: pgmap v628: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:21:06.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 236126 3.45 MB/sec execute 273 sec latency 31.820 ms 2024-04-02T17:21:07.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 237497 3.47 MB/sec execute 274 sec latency 18.907 ms 2024-04-02T17:21:07.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:06 smithi059 ceph-mon[31664]: pgmap v629: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:21:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:06 smithi164 ceph-mon[36499]: pgmap v629: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:21:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:06 smithi154 ceph-mon[36479]: pgmap v629: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:21:08.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 238069 3.48 MB/sec execute 275 sec latency 25.958 ms 2024-04-02T17:21:09.081 INFO:tasks.workunit.client.0.smithi059.stdout: 1 238477 3.46 MB/sec execute 276 sec latency 26.559 ms 2024-04-02T17:21:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:08 smithi059 ceph-mon[31664]: pgmap v630: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-04-02T17:21:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:08 smithi164 ceph-mon[36499]: pgmap v630: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-04-02T17:21:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:08 smithi154 ceph-mon[36479]: pgmap v630: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-04-02T17:21:10.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 239896 3.47 MB/sec execute 277 sec latency 21.050 ms 2024-04-02T17:21:11.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 241070 3.49 MB/sec execute 278 sec latency 17.510 ms 2024-04-02T17:21:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:10 smithi059 ceph-mon[31664]: pgmap v631: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-04-02T17:21:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:10 smithi164 ceph-mon[36499]: pgmap v631: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-04-02T17:21:11.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:10 smithi154 ceph-mon[36479]: pgmap v631: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-04-02T17:21:12.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 241633 3.49 MB/sec execute 279 sec latency 23.897 ms 2024-04-02T17:21:13.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 242162 3.48 MB/sec execute 280 sec latency 20.952 ms 2024-04-02T17:21:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:12 smithi059 ceph-mon[31664]: pgmap v632: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-02T17:21:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:12 smithi164 ceph-mon[36499]: pgmap v632: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 -- 172.21.15.59:0/285583546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 msgr2=0x7f729410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 --2- 172.21.15.59:0/285583546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f729410bfc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f728c009ca0 tx=0x7f728c02c0f0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 -- 172.21.15.59:0/285583546 shutdown_connections 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 --2- 172.21.15.59:0/285583546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f729410bfc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 --2- 172.21.15.59:0/285583546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f72941051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 --2- 172.21.15.59:0/285583546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 0x7f7294103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.395+0000 7f7293e00640 1 -- 172.21.15.59:0/285583546 >> 172.21.15.59:0/285583546 conn(0x7f72940ff110 msgr2=0x7f7294101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.396+0000 7f7293e00640 1 -- 172.21.15.59:0/285583546 shutdown_connections 2024-04-02T17:21:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.396+0000 7f7293e00640 1 -- 172.21.15.59:0/285583546 wait complete. 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.396+0000 7f7293e00640 1 Processor -- start 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 -- start start 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 0x7f7294199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f729410f7e0 con 0x7f7294103b80 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f729410f660 con 0x7f7294104d70 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f729410f960 con 0x7f7294105730 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40070/0 (socket says 172.21.15.59:40070) 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45502/0 (socket says 172.21.15.59:45502) 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7293400640 1 -- 172.21.15.59:0/583465789 learned_addr learned my addr 172.21.15.59:0/583465789 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.397+0000 7f7292a00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 0x7f7294199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 -- 172.21.15.59:0/583465789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 msgr2=0x7f729419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 -- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 msgr2=0x7f7294199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 0x7f7294199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 -- 172.21.15.59:0/583465789 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72941a19e0 con 0x7f7294105730 2024-04-02T17:21:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f728be00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.398+0000 7f7293400640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f728c02c630 tx=0x7f728c032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.399+0000 7f728b400640 1 -- 172.21.15.59:0/583465789 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c041020 con 0x7f7294105730 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.399+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72941a1c70 con 0x7f7294105730 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.399+0000 7f728b400640 1 -- 172.21.15.59:0/583465789 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728c034cd0 con 0x7f7294105730 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.399+0000 7f728b400640 1 -- 172.21.15.59:0/583465789 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c039040 con 0x7f7294105730 2024-04-02T17:21:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.399+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72941a2160 con 0x7f7294105730 2024-04-02T17:21:13.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:12 smithi154 ceph-mon[36479]: pgmap v632: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-02T17:21:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.403+0000 7f728b400640 1 -- 172.21.15.59:0/583465789 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f728c044020 con 0x7f7294105730 2024-04-02T17:21:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.403+0000 7f729c800740 1 Processor -- start 2024-04-02T17:21:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 -- start start 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1edeaf0 0x5557c1ee35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ee3c60 0x5557c1ee60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5557c1e43d70 con 0x5557c1edeaf0 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557c1dec2a0 con 0x5557c1ede100 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f729c800740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557c1e3fd20 con 0x5557c1ee3c60 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45508/0 (socket says 172.21.15.59:45508) 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f7292a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1edeaf0 0x5557c1ee35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f728be00640 1 -- 172.21.15.59:0/1226687477 learned_addr learned my addr 172.21.15.59:0/1226687477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.404+0000 7f728be00640 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ee3c60 0x5557c1ee60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 -- 172.21.15.59:0/1226687477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ee3c60 msgr2=0x5557c1ee60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ee3c60 0x5557c1ee60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 -- 172.21.15.59:0/1226687477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1edeaf0 msgr2=0x5557c1ee35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1edeaf0 0x5557c1ee35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 -- 172.21.15.59:0/1226687477 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c1e00840 con 0x5557c1ede100 2024-04-02T17:21:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f728be00640 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f727c0058c0 tx=0x7f727c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9ef2df3c3e164f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.405+0000 7f7283e00640 1 -- 172.21.15.59:0/1226687477 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c00fc50 con 0x5557c1ede100 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f7283e00640 1 -- 172.21.15.59:0/1226687477 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c00fdf0 con 0x5557c1ede100 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f7283e00640 1 -- 172.21.15.59:0/1226687477 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c003e20 con 0x5557c1ede100 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f728b400640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 0x7f72780a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f728b400640 1 -- 172.21.15.59:0/583465789 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f728c0f22a0 con 0x7f7294105730 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f7292a00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 0x7f72780a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.406+0000 7f729c800740 1 -- 172.21.15.59:0/1226687477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 msgr2=0x5557c1ede520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f727c0058c0 tx=0x7f727c009770 comp rx=0 tx=0).stop 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 -- 172.21.15.59:0/1226687477 shutdown_connections 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1edeaf0 0x5557c1ee35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ee3c60 0x5557c1ee60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 --2- 172.21.15.59:0/1226687477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1ede100 0x5557c1ede520 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 -- 172.21.15.59:0/1226687477 >> 172.21.15.59:0/1226687477 conn(0x5557c1ed9670 msgr2=0x5557c1ed9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 -- 172.21.15.59:0/1226687477 shutdown_connections 2024-04-02T17:21:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f7292a00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 0x7f72780a6060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5557c1edeef0 tx=0x7f7284005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 -- 172.21.15.59:0/1226687477 wait complete. 2024-04-02T17:21:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.407+0000 7f729c800740 1 -- start start 2024-04-02T17:21:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 Processor -- start 2024-04-02T17:21:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5557c1ee6c10 con 0x5557c1ee3c60 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5557c1dff660 con 0x5557c1edeaf0 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557c1ee6d90 con 0x5557c1ede100 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.408+0000 7f729c800740 10 client.0 _ll_register_callbacks cb 0x5557c1dff050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40094/0 (socket says 172.21.15.59:40094) 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45514/0 (socket says 172.21.15.59:45514) 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7292a00640 1 -- 172.21.15.59:0/4202878007 learned_addr learned my addr 172.21.15.59:0/4202878007 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 -- 172.21.15.59:0/4202878007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 msgr2=0x5557c1ee8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 -- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 msgr2=0x5557c1ee8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 -- 172.21.15.59:0/4202878007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c1e0ee80 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f728be00640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7292a00640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7293400640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefc00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f728c0efce0 tx=0x7f728c0be150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565baace5ee86a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.409+0000 7f7280c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0be550 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f728c0c6ca0 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c00fdf0 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f728c0c2900 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f729c800740 10 client.16494 fetch_fsmap learned FSMap version 33 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5557c1b58210 con 0x5557c1ee3c60 2024-04-02T17:21:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.410+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c00fdf0 con 0x5557c1ee3c60 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f728c03c030 con 0x5557c1ee3c60 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 10 client.16494 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 20 client.16494 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1ef4530 0x5557c1ef6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 4 client.16494 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c1df3cd0 con 0x5557c1ef4530 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f7292a00640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1ef4530 0x5557c1ef6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f7292a00640 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1ef4530 0x5557c1ef6970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf6dad9b15c4429 server_cookie=a3182967022e6142 in_seq=0 out_seq=0 2024-04-02T17:21:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.411+0000 7f7280c00640 10 client.16494 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.412+0000 7f7280c00640 1 -- 172.21.15.59:0/4202878007 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c1df3cd0 con 0x5557c1ef4530 2024-04-02T17:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.412+0000 7f7280c00640 10 client.16494 handle_command_reply: tid=0 2024-04-02T17:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.412+0000 7f729c800740 1 client.16494 shutdown 2024-04-02T17:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 msgr2=0x5557c1eefc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefc00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f728c0efce0 tx=0x7f728c0be150 comp rx=0 tx=0).stop 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 shutdown_connections 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1ef4530 0x5557c1ef6970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefc00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 --2- 172.21.15.59:0/4202878007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f7280c00640 0 client.16494 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 >> 172.21.15.59:0/4202878007 conn(0x5557c1ed9670 msgr2=0x5557c1ed8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 shutdown_connections 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 1 -- 172.21.15.59:0/4202878007 wait complete. 2024-04-02T17:21:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.413+0000 7f729c800740 20 client.16494 trim_cache size 0 max 16384 2024-04-02T17:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 Processor -- start 2024-04-02T17:21:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 -- start start 2024-04-02T17:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ab0a0 con 0x5557c1ee3c60 2024-04-02T17:21:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72941acbe0 con 0x5557c1edeaf0 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.432+0000 7f729c800740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ab770 con 0x5557c1ede100 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40104/0 (socket says 172.21.15.59:40104) 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 -- 172.21.15.59:0/3346617586 learned_addr learned my addr 172.21.15.59:0/3346617586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f728be00640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7293400640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 -- 172.21.15.59:0/3346617586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 msgr2=0x5557c1ee0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 -- 172.21.15.59:0/3346617586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 msgr2=0x5557c1eefbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 -- 172.21.15.59:0/3346617586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c1c24620 con 0x5557c1ede100 2024-04-02T17:21:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7293400640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:21:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.433+0000 7f7292a00640 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5557c1ef71e0 tx=0x7f7284022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00ff43eae9beb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f7283400640 1 -- 172.21.15.59:0/3346617586 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284003da0 con 0x5557c1ede100 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f7283400640 1 -- 172.21.15.59:0/3346617586 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284030070 con 0x5557c1ede100 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f7283400640 1 -- 172.21.15.59:0/3346617586 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728401fcc0 con 0x5557c1ede100 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 -- 172.21.15.59:0/3346617586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 msgr2=0x5557c1ee0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5557c1ef71e0 tx=0x7f7284022040 comp rx=0 tx=0).stop 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 -- 172.21.15.59:0/3346617586 shutdown_connections 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1eefbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1ee0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 --2- 172.21.15.59:0/3346617586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1ee0050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.434+0000 7f729c800740 1 -- 172.21.15.59:0/3346617586 >> 172.21.15.59:0/3346617586 conn(0x5557c1ed9670 msgr2=0x5557c1ee8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.435+0000 7f729c800740 1 -- 172.21.15.59:0/3346617586 shutdown_connections 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.435+0000 7f729c800740 1 -- 172.21.15.59:0/3346617586 wait complete. 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.435+0000 7f729c800740 1 -- start start 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.435+0000 7f729c800740 1 Processor -- start 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1eed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1f58aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ac510 con 0x5557c1ee3c60 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72941abe40 con 0x5557c1edeaf0 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72941a9080 con 0x5557c1ede100 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f729c800740 10 client.0 _ll_register_callbacks cb 0x7f728c0ba7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40118/0 (socket says 172.21.15.59:40118) 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 -- 172.21.15.59:0/46032987 learned_addr learned my addr 172.21.15.59:0/46032987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f728be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1eed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7293400640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1f58aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 -- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 msgr2=0x5557c1eed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1eed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 -- 172.21.15.59:0/46032987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 msgr2=0x5557c1f58aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1f58aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7292a00640 1 -- 172.21.15.59:0/46032987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c1f58fe0 con 0x5557c1ede100 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.436+0000 7f7293400640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1f58aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f7292a00640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f728402f000 tx=0x7f7284003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281b1630742fe7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f7280c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284039020 con 0x5557c1ede100 2024-04-02T17:21:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7284030070 con 0x5557c1ede100 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f728401f4e0 con 0x5557c1ede100 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7284044050 con 0x5557c1ede100 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f729c800740 10 client.26611 fetch_fsmap learned FSMap version 33 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.437+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5557c1f5a260 con 0x5557c1ede100 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284030070 con 0x5557c1ede100 2024-04-02T17:21:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f72840050d0 con 0x5557c1ede100 2024-04-02T17:21:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 10 client.26611 fetch_fsmap finished waiting for FSMap version 33 2024-04-02T17:21:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 20 client.26611 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1f5ac40 0x5557c1f5d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 4 client.26611 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557c1e512b0 con 0x5557c1f5ac40 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.438+0000 7f728be00640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1f5ac40 0x5557c1f5d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.439+0000 7f728be00640 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1f5ac40 0x5557c1f5d060 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a314dd16d3e34f server_cookie=f2082785229845d8 in_seq=0 out_seq=0 2024-04-02T17:21:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.439+0000 7f7280c00640 10 client.26611 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f7280c00640 1 -- 172.21.15.59:0/46032987 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557c1e512b0 con 0x5557c1f5ac40 2024-04-02T17:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f7280c00640 10 client.26611 handle_command_reply: tid=0 2024-04-02T17:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 client.26611 shutdown 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 msgr2=0x5557c1eecb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f728402f000 tx=0x7f7284003e90 comp rx=0 tx=0).stop 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 shutdown_connections 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5557c1f5ac40 0x5557c1f5d060 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5557c1ee3c60 0x5557c1f58aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5557c1edeaf0 0x5557c1eed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 --2- 172.21.15.59:0/46032987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557c1ede100 0x5557c1eecb50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f7280c00640 0 client.26611 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.471+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 >> 172.21.15.59:0/46032987 conn(0x5557c1ed9670 msgr2=0x5557c1ee8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.472+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 shutdown_connections 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.472+0000 7f729c800740 1 -- 172.21.15.59:0/46032987 wait complete. 2024-04-02T17:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.472+0000 7f729c800740 20 client.26611 trim_cache size 0 max 16384 2024-04-02T17:21:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:21:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:21:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:21:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 msgr2=0x7f72780a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 0x7f72780a6060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5557c1edeef0 tx=0x7f7284005f90 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 msgr2=0x7f72941a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f728c02c630 tx=0x7f728c032ea0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 shutdown_connections 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7294105730 0x7f72941a1370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7294104d70 0x7f729419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f72780a3ba0 0x7f72780a6060 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 --2- 172.21.15.59:0/583465789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7294103b80 0x7f7294199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 >> 172.21.15.59:0/583465789 conn(0x7f72940ff110 msgr2=0x7f7294108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 shutdown_connections 2024-04-02T17:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:13.473+0000 7f7293e00640 1 -- 172.21.15.59:0/583465789 wait complete. 2024-04-02T17:21:14.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 243690 3.50 MB/sec execute 281 sec latency 24.980 ms 2024-04-02T17:21:15.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 244995 3.52 MB/sec execute 282 sec latency 20.681 ms 2024-04-02T17:21:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:14 smithi059 ceph-mon[31664]: pgmap v633: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:21:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:14 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:21:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:14 smithi164 ceph-mon[36499]: pgmap v633: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:21:15.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:14 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:21:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:14 smithi154 ceph-mon[36479]: pgmap v633: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:21:15.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:14 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:21:16.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 245266 3.51 MB/sec execute 283 sec latency 35.703 ms 2024-04-02T17:21:16.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:21:16.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:21:16.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:21:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:21:16.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:21:16.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:21:16.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:21:16.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:21:16.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:21:17.082 INFO:tasks.workunit.client.0.smithi059.stdout: 1 245995 3.51 MB/sec execute 284 sec latency 20.962 ms 2024-04-02T17:21:17.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:16 smithi059 ceph-mon[31664]: pgmap v634: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:21:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:16 smithi164 ceph-mon[36499]: pgmap v634: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:21:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:16 smithi154 ceph-mon[36479]: pgmap v634: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-02T17:21:18.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 247501 3.53 MB/sec execute 285 sec latency 21.967 ms 2024-04-02T17:21:19.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 248592 3.54 MB/sec execute 286 sec latency 39.058 ms 2024-04-02T17:21:19.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:18 smithi059 ceph-mon[31664]: pgmap v635: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:21:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:18 smithi164 ceph-mon[36499]: pgmap v635: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:21:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:18 smithi154 ceph-mon[36479]: pgmap v635: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:21:20.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 248938 3.53 MB/sec execute 287 sec latency 25.735 ms 2024-04-02T17:21:21.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 249988 3.54 MB/sec execute 288 sec latency 22.790 ms 2024-04-02T17:21:21.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:20 smithi059 ceph-mon[31664]: pgmap v636: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-02T17:21:21.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:20 smithi164 ceph-mon[36499]: pgmap v636: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-02T17:21:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:20 smithi154 ceph-mon[36479]: pgmap v636: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-02T17:21:22.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 251535 3.55 MB/sec execute 289 sec latency 19.288 ms 2024-04-02T17:21:23.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 252245 3.56 MB/sec execute 290 sec latency 33.405 ms 2024-04-02T17:21:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:22 smithi059 ceph-mon[31664]: pgmap v637: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-04-02T17:21:23.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:22 smithi164 ceph-mon[36499]: pgmap v637: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-04-02T17:21:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:22 smithi154 ceph-mon[36479]: pgmap v637: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-04-02T17:21:24.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 252601 3.55 MB/sec execute 291 sec latency 30.725 ms 2024-04-02T17:21:25.083 INFO:tasks.workunit.client.0.smithi059.stdout: 1 253971 3.55 MB/sec execute 292 sec latency 28.091 ms 2024-04-02T17:21:25.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:24 smithi059 ceph-mon[31664]: pgmap v638: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-02T17:21:25.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:24 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:21:25.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:24 smithi164 ceph-mon[36499]: pgmap v638: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-02T17:21:25.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:24 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:21:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:24 smithi154 ceph-mon[36479]: pgmap v638: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-02T17:21:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:24 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:21:26.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 255247 3.57 MB/sec execute 293 sec latency 16.801 ms 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 -- 172.21.15.59:0/3895389102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740100850 msgr2=0x7fe740100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 --2- 172.21.15.59:0/3895389102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740100850 0x7fe740100c70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402bc30 comp rx=0 tx=0).stop 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 -- 172.21.15.59:0/3895389102 shutdown_connections 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 --2- 172.21.15.59:0/3895389102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740102400 0x7fe740108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 --2- 172.21.15.59:0/3895389102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740101a40 0x7fe740101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 --2- 172.21.15.59:0/3895389102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740100850 0x7fe740100c70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 -- 172.21.15.59:0/3895389102 >> 172.21.15.59:0/3895389102 conn(0x7fe7400fbde0 msgr2=0x7fe7400fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 -- 172.21.15.59:0/3895389102 shutdown_connections 2024-04-02T17:21:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.623+0000 7fe746600640 1 -- 172.21.15.59:0/3895389102 wait complete. 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 Processor -- start 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 -- start start 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 0x7fe740198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 0x7fe74019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010c510 con 0x7fe740101a40 2024-04-02T17:21:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010c390 con 0x7fe740102400 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.625+0000 7fe746600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010c690 con 0x7fe740100850 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 0x7fe74019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42344/0 (socket says 172.21.15.59:42344) 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 -- 172.21.15.59:0/2534186629 learned_addr learned my addr 172.21.15.59:0/2534186629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe73fe00640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 0x7fe740198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 -- 172.21.15.59:0/2534186629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 msgr2=0x7fe74019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 0x7fe74019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 -- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 msgr2=0x7fe740198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 0x7fe740198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745200640 1 -- 172.21.15.59:0/2534186629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74019f900 con 0x7fe740100850 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe745c00640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 0x7fe74019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.626+0000 7fe73fe00640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 0x7fe740198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe745200640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe73402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe73d600640 1 -- 172.21.15.59:0/2534186629 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734043020 con 0x7fe740100850 2024-04-02T17:21:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe73d600640 1 -- 172.21.15.59:0/2534186629 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7340054e0 con 0x7fe740100850 2024-04-02T17:21:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe73d600640 1 -- 172.21.15.59:0/2534186629 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734003ce0 con 0x7fe740100850 2024-04-02T17:21:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74019fb90 con 0x7fe740100850 2024-04-02T17:21:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.627+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7401a0060 con 0x7fe740100850 2024-04-02T17:21:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 Processor -- start 2024-04-02T17:21:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe73d600640 1 -- 172.21.15.59:0/2534186629 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe734046020 con 0x7fe740100850 2024-04-02T17:21:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 -- start start 2024-04-02T17:21:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2af0 0x5626492d75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 0x5626492da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562649237d70 con 0x5626492d2af0 2024-04-02T17:21:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626491e02a0 con 0x5626492d7c60 2024-04-02T17:21:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.632+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562649233d20 con 0x5626492d2100 2024-04-02T17:21:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 0x5626492da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42348/0 (socket says 172.21.15.59:42348) 2024-04-02T17:21:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 -- 172.21.15.59:0/1924588376 learned_addr learned my addr 172.21.15.59:0/1924588376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2af0 0x5626492d75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 -- 172.21.15.59:0/1924588376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 msgr2=0x5626492da0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 0x5626492da0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 -- 172.21.15.59:0/1924588376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2af0 msgr2=0x5626492d75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2af0 0x5626492d75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 -- 172.21.15.59:0/1924588376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626491f4840 con 0x5626492d2100 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe73fe00640 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 0x5626492da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.633+0000 7fe745c00640 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7380117e0 tx=0x7fe738011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b96a5cb68632fba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe732000640 1 -- 172.21.15.59:0/1924588376 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738011310 con 0x5626492d2100 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe732000640 1 -- 172.21.15.59:0/1924588376 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe738003e50 con 0x5626492d2100 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe732000640 1 -- 172.21.15.59:0/1924588376 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738004620 con 0x5626492d2100 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe73d600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 0x7fe7280a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe73d600640 1 -- 172.21.15.59:0/2534186629 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe7340f25b0 con 0x7fe740100850 2024-04-02T17:21:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.634+0000 7fe745200640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 0x7fe7280a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 -- 172.21.15.59:0/1924588376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 msgr2=0x5626492d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7380117e0 tx=0x7fe738011ca0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 -- 172.21.15.59:0/1924588376 shutdown_connections 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2af0 0x5626492d75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d7c60 0x5626492da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/1924588376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d2520 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 -- 172.21.15.59:0/1924588376 >> 172.21.15.59:0/1924588376 conn(0x5626492cd670 msgr2=0x5626492cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 -- 172.21.15.59:0/1924588376 shutdown_connections 2024-04-02T17:21:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe745200640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 0x7fe7280a60e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7340f03c0 tx=0x7fe7340f8110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.635+0000 7fe74a7f2740 1 -- 172.21.15.59:0/1924588376 wait complete. 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.636+0000 7fe74a7f2740 1 -- start start 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.636+0000 7fe74a7f2740 1 Processor -- start 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.636+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492e0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626492dac10 con 0x5626492d7c60 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626491f3660 con 0x5626492d2af0 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626492dad90 con 0x5626492d2100 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe74a7f2740 10 client.0 _ll_register_callbacks cb 0x5626491f3050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe745c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492e0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe745c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492e0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42356/0 (socket says 172.21.15.59:42356) 2024-04-02T17:21:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe745c00640 1 -- 172.21.15.59:0/873303919 learned_addr learned my addr 172.21.15.59:0/873303919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe745c00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 -- 172.21.15.59:0/873303919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 msgr2=0x5626492e0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492e0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 -- 172.21.15.59:0/873303919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 msgr2=0x5626492e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe73fe00640 1 -- 172.21.15.59:0/873303919 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562649202e80 con 0x5626492d2af0 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.637+0000 7fe745c00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe73fe00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e1030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005fd0 tx=0x7fe72c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b30ac68c450ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe726a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c005220 con 0x5626492d2af0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe738003e50 con 0x5626492d2af0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe72c0053c0 con 0x5626492d2af0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe72c0055c0 con 0x5626492d2af0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe74a7f2740 10 client.26603 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.638+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562648f4c210 con 0x5626492d2af0 2024-04-02T17:21:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.639+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe738003e50 con 0x5626492d2af0 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.639+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe72c022020 con 0x5626492d2af0 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 10 client.26603 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 20 client.26603 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626492ec9a0 0x5626492eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 4 client.26603 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:21:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626491e7cd0 con 0x5626492ec9a0 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe745c00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626492ec9a0 0x5626492eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe745c00640 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626492ec9a0 0x5626492eed90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90774df2cf8a008 server_cookie=e5d7e8cfec90c25 in_seq=0 out_seq=0 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.640+0000 7fe726a00640 10 client.26603 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.641+0000 7fe726a00640 1 -- 172.21.15.59:0/873303919 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626491e7cd0 con 0x5626492ec9a0 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.641+0000 7fe726a00640 10 client.26603 handle_command_reply: tid=0 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.641+0000 7fe74a7f2740 1 client.26603 shutdown 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 msgr2=0x5626492e1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e1030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005fd0 tx=0x7fe72c012820 comp rx=0 tx=0).stop 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 shutdown_connections 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626492ec9a0 0x5626492eed90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e1030 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/873303919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492e0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe726a00640 0 client.26603 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 >> 172.21.15.59:0/873303919 conn(0x5626492cd670 msgr2=0x5626492cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 shutdown_connections 2024-04-02T17:21:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 1 -- 172.21.15.59:0/873303919 wait complete. 2024-04-02T17:21:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.642+0000 7fe74a7f2740 20 client.26603 trim_cache size 0 max 16384 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 Processor -- start 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 -- start start 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a6e00 con 0x5626492d7c60 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a8970 con 0x5626492d2af0 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.664+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a74d0 con 0x5626492d2100 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57370/0 (socket says 172.21.15.59:57370) 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 -- 172.21.15.59:0/3246363256 learned_addr learned my addr 172.21.15.59:0/3246363256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe745c00640 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe745c00640 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 -- 172.21.15.59:0/3246363256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 msgr2=0x5626492d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 -- 172.21.15.59:0/3246363256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 msgr2=0x5626492f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.665+0000 7fe73fe00640 1 -- 172.21.15.59:0/3246363256 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562649018620 con 0x5626492d2af0 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.666+0000 7fe73fe00640 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011010 tx=0x7fe72c00b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7af7fe1475136c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.666+0000 7fe731600640 1 -- 172.21.15.59:0/3246363256 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c010a20 con 0x5626492d2af0 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.666+0000 7fe731600640 1 -- 172.21.15.59:0/3246363256 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00b610 con 0x5626492d2af0 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.666+0000 7fe731600640 1 -- 172.21.15.59:0/3246363256 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c01a7f0 con 0x5626492d2af0 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 -- 172.21.15.59:0/3246363256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 msgr2=0x5626492e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011010 tx=0x7fe72c00b7c0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 -- 172.21.15.59:0/3246363256 shutdown_connections 2024-04-02T17:21:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d7c60 0x5626492f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e8030 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/3246363256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d2100 0x5626492d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 -- 172.21.15.59:0/3246363256 >> 172.21.15.59:0/3246363256 conn(0x5626492cd670 msgr2=0x5626492d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 -- 172.21.15.59:0/3246363256 shutdown_connections 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.667+0000 7fe74a7f2740 1 -- 172.21.15.59:0/3246363256 wait complete. 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 -- start start 2024-04-02T17:21:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 Processor -- start 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2100 0x5626492e5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 0x5626492e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a82a0 con 0x5626492d2100 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a7ba0 con 0x5626492d2af0 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401a4de0 con 0x5626492d7c60 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe74a7f2740 10 client.0 _ll_register_callbacks cb 0x7fe72c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.668+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57384/0 (socket says 172.21.15.59:57384) 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe745c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2100 0x5626492e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 -- 172.21.15.59:0/979132107 learned_addr learned my addr 172.21.15.59:0/979132107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe745c00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 0x5626492e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 -- 172.21.15.59:0/979132107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 msgr2=0x5626492e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 0x5626492e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 -- 172.21.15.59:0/979132107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2100 msgr2=0x5626492e5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2100 0x5626492e5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 -- 172.21.15.59:0/979132107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626492e72c0 con 0x5626492d2af0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe745c00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 0x5626492e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe73fe00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe72c012860 tx=0x7fe72c024810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f343adebf4d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.669+0000 7fe726a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c012d20 con 0x5626492d2af0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe72c01a570 con 0x5626492d2af0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c00b610 con 0x5626492d2af0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe72c024b90 con 0x5626492d2af0 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe74a7f2740 10 client.26609 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562649352f30 con 0x5626492d2af0 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.670+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00b610 con 0x5626492d2af0 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe72c022040 con 0x5626492d2af0 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 10 client.26609 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 10 client.26609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 20 client.26609 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626493537f0 0x562649355c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 4 client.26609 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:21:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626492452b0 con 0x5626493537f0 2024-04-02T17:21:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.671+0000 7fe745c00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626493537f0 0x562649355c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.672+0000 7fe745c00640 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626493537f0 0x562649355c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb14e2550413b3b server_cookie=504893e9d8acd897 in_seq=0 out_seq=0 2024-04-02T17:21:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.672+0000 7fe726a00640 10 client.26609 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.709+0000 7fe726a00640 1 -- 172.21.15.59:0/979132107 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626492452b0 con 0x5626493537f0 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.709+0000 7fe726a00640 10 client.26609 handle_command_reply: tid=0 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.709+0000 7fe74a7f2740 1 client.26609 shutdown 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 msgr2=0x5626492e5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe72c012860 tx=0x7fe72c024810 comp rx=0 tx=0).stop 2024-04-02T17:21:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 shutdown_connections 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626493537f0 0x562649355c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626492d7c60 0x5626492e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626492d2af0 0x5626492e5cc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 --2- 172.21.15.59:0/979132107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626492d2100 0x5626492e5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 >> 172.21.15.59:0/979132107 conn(0x5626492cd670 msgr2=0x5626492cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe726a00640 0 client.26609 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 shutdown_connections 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 1 -- 172.21.15.59:0/979132107 wait complete. 2024-04-02T17:21:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.710+0000 7fe74a7f2740 20 client.26609 trim_cache size 0 max 16384 2024-04-02T17:21:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:21:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:21:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:21:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:21:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 msgr2=0x7fe7280a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 0x7fe7280a60e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7340f03c0 tx=0x7fe7340f8110 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 msgr2=0x7fe740197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe73402c6c0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 shutdown_connections 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe7280a3c20 0x7fe7280a60e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe740101a40 0x7fe740198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe740102400 0x7fe74019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 --2- 172.21.15.59:0/2534186629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe740100850 0x7fe740197cf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.711+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 >> 172.21.15.59:0/2534186629 conn(0x7fe7400fbde0 msgr2=0x7fe740069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.712+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 shutdown_connections 2024-04-02T17:21:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:26.712+0000 7fe746600640 1 -- 172.21.15.59:0/2534186629 wait complete. 2024-04-02T17:21:27.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 255823 3.58 MB/sec execute 294 sec latency 26.312 ms 2024-04-02T17:21:27.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:26 smithi059 ceph-mon[31664]: pgmap v639: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:21:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:26 smithi164 ceph-mon[36499]: pgmap v639: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:21:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:26 smithi154 ceph-mon[36479]: pgmap v639: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:21:28.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 256400 3.57 MB/sec execute 295 sec latency 31.149 ms 2024-04-02T17:21:29.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 257929 3.58 MB/sec execute 296 sec latency 17.201 ms 2024-04-02T17:21:29.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:28 smithi059 ceph-mon[31664]: pgmap v640: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-04-02T17:21:29.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:28 smithi164 ceph-mon[36499]: pgmap v640: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-04-02T17:21:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:28 smithi154 ceph-mon[36479]: pgmap v640: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-04-02T17:21:30.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 259125 3.60 MB/sec execute 297 sec latency 32.159 ms 2024-04-02T17:21:31.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 259415 3.59 MB/sec execute 298 sec latency 31.174 ms 2024-04-02T17:21:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:30 smithi059 ceph-mon[31664]: pgmap v641: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-02T17:21:31.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:30 smithi164 ceph-mon[36499]: pgmap v641: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-02T17:21:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:30 smithi154 ceph-mon[36479]: pgmap v641: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-04-02T17:21:32.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 260037 3.59 MB/sec execute 299 sec latency 25.751 ms 2024-04-02T17:21:33.084 INFO:tasks.workunit.client.0.smithi059.stdout: 1 261377 3.59 MB/sec execute 300 sec latency 223.158 ms 2024-04-02T17:21:33.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:32 smithi059 ceph-mon[31664]: pgmap v642: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:33.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:32 smithi164 ceph-mon[36499]: pgmap v642: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:32 smithi154 ceph-mon[36479]: pgmap v642: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:34.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 262575 3.62 MB/sec execute 301 sec latency 19.193 ms 2024-04-02T17:21:35.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 262937 3.61 MB/sec execute 302 sec latency 37.739 ms 2024-04-02T17:21:35.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:34 smithi059 ceph-mon[31664]: pgmap v643: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:35.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:34 smithi164 ceph-mon[36499]: pgmap v643: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:34 smithi154 ceph-mon[36479]: pgmap v643: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-04-02T17:21:36.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 263491 3.60 MB/sec execute 303 sec latency 21.682 ms 2024-04-02T17:21:37.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 265003 3.61 MB/sec execute 304 sec latency 21.503 ms 2024-04-02T17:21:37.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:36 smithi059 ceph-mon[31664]: pgmap v644: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:21:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:36 smithi164 ceph-mon[36499]: pgmap v644: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:21:37.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:36 smithi154 ceph-mon[36479]: pgmap v644: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:21:38.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 266255 3.63 MB/sec execute 305 sec latency 19.156 ms 2024-04-02T17:21:39.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 266506 3.62 MB/sec execute 306 sec latency 28.349 ms 2024-04-02T17:21:39.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:38 smithi059 ceph-mon[31664]: pgmap v645: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:39.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:38 smithi164 ceph-mon[36499]: pgmap v645: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:39.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:38 smithi154 ceph-mon[36479]: pgmap v645: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 -- 172.21.15.59:0/4278921717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 msgr2=0x7f0f98104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 --2- 172.21.15.59:0/4278921717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98104150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f8802bc40 comp rx=0 tx=0).stop 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 -- 172.21.15.59:0/4278921717 shutdown_connections 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 --2- 172.21.15.59:0/4278921717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 0x7f0f9810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 --2- 172.21.15.59:0/4278921717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 0x7f0f981053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 --2- 172.21.15.59:0/4278921717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98104150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 -- 172.21.15.59:0/4278921717 >> 172.21.15.59:0/4278921717 conn(0x7f0f980ff2c0 msgr2=0x7f0f981016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 -- 172.21.15.59:0/4278921717 shutdown_connections 2024-04-02T17:21:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.875+0000 7f0f9da00640 1 -- 172.21.15.59:0/4278921717 wait complete. 2024-04-02T17:21:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.876+0000 7f0f9da00640 1 Processor -- start 2024-04-02T17:21:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.876+0000 7f0f9da00640 1 -- start start 2024-04-02T17:21:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 0x7f0f9819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 0x7f0f981a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810f9a0 con 0x7f0f981058e0 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810f820 con 0x7f0f98104f20 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f9da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810fb20 con 0x7f0f98103d30 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47618/0 (socket says 172.21.15.59:47618) 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 -- 172.21.15.59:0/3433157410 learned_addr learned my addr 172.21.15.59:0/3433157410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f97400640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 0x7f0f981a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96000640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 0x7f0f9819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 -- 172.21.15.59:0/3433157410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 msgr2=0x7f0f9819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 0x7f0f9819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 -- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 msgr2=0x7f0f981a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 0x7f0f981a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.877+0000 7f0f96a00640 1 -- 172.21.15.59:0/3433157410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f981a1bd0 con 0x7f0f98103d30 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f96a00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000c00 tx=0x7f0f8802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f8f400640 1 -- 172.21.15.59:0/3433157410 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88041070 con 0x7f0f98103d30 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f8f400640 1 -- 172.21.15.59:0/3433157410 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88004450 con 0x7f0f98103d30 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f8f400640 1 -- 172.21.15.59:0/3433157410 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88034440 con 0x7f0f98103d30 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f981a1e60 con 0x7f0f98103d30 2024-04-02T17:21:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.878+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f981a2270 con 0x7f0f98103d30 2024-04-02T17:21:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.881+0000 7f0f8f400640 1 -- 172.21.15.59:0/3433157410 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f88044070 con 0x7f0f98103d30 2024-04-02T17:21:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.882+0000 7f0fa07d1740 1 Processor -- start 2024-04-02T17:21:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.882+0000 7f0fa07d1740 1 -- start start 2024-04-02T17:21:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e1632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e1683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e16aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55891e0c8d70 con 0x55891e168a30 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55891e0712a0 con 0x55891e162ed0 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55891e0c4d20 con 0x55891e1638c0 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f97400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e1632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f97400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e1632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60766/0 (socket says 172.21.15.59:60766) 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f97400640 1 -- 172.21.15.59:0/1308396898 learned_addr learned my addr 172.21.15.59:0/1308396898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e1683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f96000640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e16aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.883+0000 7f0f8f400640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 0x7f0f7c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 -- 172.21.15.59:0/1308396898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 msgr2=0x55891e1632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e1632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 -- 172.21.15.59:0/1308396898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 msgr2=0x55891e16aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e16aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f8f400640 1 -- 172.21.15.59:0/3433157410 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0f880f1e50 con 0x7f0f98103d30 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 -- 172.21.15.59:0/1308396898 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891e085840 con 0x55891e1638c0 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f97400640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 0x7f0f7c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96000640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e16aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e1683c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ef860 tx=0x7f0f88005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0a41b9b89a95da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f83e00640 1 -- 172.21.15.59:0/1308396898 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8804b070 con 0x55891e1638c0 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f83e00640 1 -- 172.21.15.59:0/1308396898 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88004c80 con 0x55891e1638c0 2024-04-02T17:21:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.884+0000 7f0f83e00640 1 -- 172.21.15.59:0/1308396898 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f880c6ce0 con 0x55891e1638c0 2024-04-02T17:21:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1308396898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 msgr2=0x55891e1683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0f97400640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 0x7f0f7c0a6280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009fd0 tx=0x7f0f90004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e1683c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ef860 tx=0x7f0f88005e50 comp rx=0 tx=0).stop 2024-04-02T17:21:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1308396898 shutdown_connections 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e16aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e1683c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1308396898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e1632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.885+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1308396898 >> 172.21.15.59:0/1308396898 conn(0x55891e15e440 msgr2=0x55891e15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.886+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1308396898 shutdown_connections 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.886+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1308396898 wait complete. 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.886+0000 7f0fa07d1740 1 -- start start 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.886+0000 7f0fa07d1740 1 Processor -- start 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e1638c0 0x55891e16d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 0x55891e174610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55891e16b7a0 con 0x55891e1638c0 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55891e084660 con 0x55891e168a30 2024-04-02T17:21:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55891e16b920 con 0x55891e162ed0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0fa07d1740 10 client.0 _ll_register_callbacks cb 0x55891e084050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47630/0 (socket says 172.21.15.59:47630) 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0f96a00640 1 -- 172.21.15.59:0/2445210988 learned_addr learned my addr 172.21.15.59:0/2445210988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0f97400640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 0x55891e174610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.887+0000 7f0f96000640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e1638c0 0x55891e16d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 -- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 msgr2=0x55891e174610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 0x55891e174610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 -- 172.21.15.59:0/2445210988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e1638c0 msgr2=0x55891e16d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e1638c0 0x55891e16d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 -- 172.21.15.59:0/2445210988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891e093e80 con 0x55891e162ed0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f97400640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 0x55891e174610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f880bab80 tx=0x7f0f88003990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b34c82affb193a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8804b070 con 0x55891e162ed0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0f880f5360 con 0x55891e162ed0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f88004c80 con 0x55891e162ed0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f880c6ce0 con 0x55891e162ed0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0fa07d1740 10 client.26641 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.888+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55891dddd210 con 0x55891e162ed0 2024-04-02T17:21:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88004c80 con 0x55891e162ed0 2024-04-02T17:21:39.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0f8803ed10 con 0x55891e162ed0 2024-04-02T17:21:39.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 10 client.26641 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 20 client.26641 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e178d80 0x55891e17b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 4 client.26641 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891e078cd0 con 0x55891e178d80 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.889+0000 7f0f96000640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e178d80 0x55891e17b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.890+0000 7f0f96000640 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e178d80 0x55891e17b190 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d17006bcdde14c5 server_cookie=118a8f4ef9632d7b in_seq=0 out_seq=0 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.890+0000 7f0f80c00640 10 client.26641 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0f80c00640 1 -- 172.21.15.59:0/2445210988 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55891e078cd0 con 0x55891e178d80 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0f80c00640 10 client.26641 handle_command_reply: tid=0 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 client.26641 shutdown 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 msgr2=0x55891e16d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f880bab80 tx=0x7f0f88003990 comp rx=0 tx=0).stop 2024-04-02T17:21:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 shutdown_connections 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e178d80 0x55891e17b190 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e1638c0 0x55891e16d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e168a30 0x55891e174610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2445210988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e162ed0 0x55891e16d090 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 >> 172.21.15.59:0/2445210988 conn(0x55891e15e440 msgr2=0x55891e15d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0f80c00640 0 client.26641 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 shutdown_connections 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.891+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2445210988 wait complete. 2024-04-02T17:21:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.892+0000 7f0fa07d1740 20 client.26641 trim_cache size 0 max 16384 2024-04-02T17:21:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.913+0000 7f0fa07d1740 1 Processor -- start 2024-04-02T17:21:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.913+0000 7f0fa07d1740 1 -- start start 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.913+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.913+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.913+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981ab210 con 0x55891e168a30 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981acd80 con 0x55891e162ed0 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981ab8e0 con 0x55891e1638c0 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60782/0 (socket says 172.21.15.59:60782) 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 -- 172.21.15.59:0/1860842481 learned_addr learned my addr 172.21.15.59:0/1860842481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47632/0 (socket says 172.21.15.59:47632) 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f97400640 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 -- 172.21.15.59:0/1860842481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 msgr2=0x55891e16d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 -- 172.21.15.59:0/1860842481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 msgr2=0x55891e15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96a00640 1 -- 172.21.15.59:0/1860842481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891dea9620 con 0x55891e162ed0 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.914+0000 7f0f96000640 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.915+0000 7f0f96a00640 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f880b9e90 tx=0x7f0f88005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363d6c0b9c0bd8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.915+0000 7f0f83400640 1 -- 172.21.15.59:0/1860842481 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8804b070 con 0x55891e162ed0 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.915+0000 7f0f83400640 1 -- 172.21.15.59:0/1860842481 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88004c80 con 0x55891e162ed0 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.915+0000 7f0f83400640 1 -- 172.21.15.59:0/1860842481 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f880c6850 con 0x55891e162ed0 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1860842481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 msgr2=0x55891e16cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f880b9e90 tx=0x7f0f88005e50 comp rx=0 tx=0).stop 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1860842481 shutdown_connections 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e16d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/1860842481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e16cca0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1860842481 >> 172.21.15.59:0/1860842481 conn(0x55891e15e440 msgr2=0x55891e15e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1860842481 shutdown_connections 2024-04-02T17:21:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- 172.21.15.59:0/1860842481 wait complete. 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.916+0000 7f0fa07d1740 1 -- start start 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 Processor -- start 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e172bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981ac6b0 con 0x55891e168a30 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981abfb0 con 0x55891e162ed0 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981a91f0 con 0x55891e1638c0 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0fa07d1740 10 client.0 _ll_register_callbacks cb 0x7f0f880bdb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f96000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f96a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:60794/0 (socket says 172.21.15.59:60794) 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f96000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47638/0 (socket says 172.21.15.59:47638) 2024-04-02T17:21:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f96a00640 1 -- 172.21.15.59:0/2712863020 learned_addr learned my addr 172.21.15.59:0/2712863020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.917+0000 7f0f97400640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e172bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 -- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 msgr2=0x55891e171670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 -- 172.21.15.59:0/2712863020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 msgr2=0x55891e172bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e172bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 -- 172.21.15.59:0/2712863020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891e1731b0 con 0x55891e1638c0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f97400640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e172bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f96000640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55891e17b9b0 tx=0x7f0f84004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2591bc956fa1ad8d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84024020 con 0x55891e1638c0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0f84003800 con 0x55891e1638c0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.918+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f88004c80 con 0x55891e1638c0 2024-04-02T17:21:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0f8402f050 con 0x55891e1638c0 2024-04-02T17:21:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 10 client.26647 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55891e1df7f0 con 0x55891e1638c0 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88004c80 con 0x55891e1638c0 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0f84010ac0 con 0x55891e1638c0 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 10 client.26647 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 10 client.26647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 20 client.26647 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e1e01f0 0x55891e1e2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.919+0000 7f0fa07d1740 4 client.26647 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.920+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55891e0d62b0 con 0x55891e1e01f0 2024-04-02T17:21:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.920+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e1e01f0 0x55891e1e2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.920+0000 7f0f80c00640 10 client.26647 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.920+0000 7f0f96a00640 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e1e01f0 0x55891e1e2610 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602a65fa7e721326 server_cookie=c5d27ba1e41ebc96 in_seq=0 out_seq=0 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0f80c00640 1 -- 172.21.15.59:0/2712863020 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55891e0d62b0 con 0x55891e1e01f0 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0f80c00640 10 client.26647 handle_command_reply: tid=0 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 client.26647 shutdown 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 msgr2=0x55891e171bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55891e17b9b0 tx=0x7f0f84004410 comp rx=0 tx=0).stop 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 shutdown_connections 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891e1e01f0 0x55891e1e2610 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891e168a30 0x55891e172bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891e1638c0 0x55891e171bb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 --2- 172.21.15.59:0/2712863020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891e162ed0 0x55891e171670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0f80c00640 0 client.26647 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 >> 172.21.15.59:0/2712863020 conn(0x55891e15e440 msgr2=0x55891e16cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 shutdown_connections 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 1 -- 172.21.15.59:0/2712863020 wait complete. 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.957+0000 7f0fa07d1740 20 client.26647 trim_cache size 0 max 16384 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:21:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 msgr2=0x7f0f7c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 0x7f0f7c0a6280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009fd0 tx=0x7f0f90004a40 comp rx=0 tx=0).stop 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 msgr2=0x7f0f98199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f88000c00 tx=0x7f0f8802c650 comp rx=0 tx=0).stop 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 shutdown_connections 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f0f7c0a3dc0 0x7f0f7c0a6280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f981058e0 0x7f0f981a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f98104f20 0x7f0f9819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 --2- 172.21.15.59:0/3433157410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f98103d30 0x7f0f98199fb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 >> 172.21.15.59:0/3433157410 conn(0x7f0f980ff2c0 msgr2=0x7f0f98108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 shutdown_connections 2024-04-02T17:21:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:39.958+0000 7f0f9da00640 1 -- 172.21.15.59:0/3433157410 wait complete. 2024-04-02T17:21:40.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 267268 3.62 MB/sec execute 307 sec latency 28.934 ms 2024-04-02T17:21:41.085 INFO:tasks.workunit.client.0.smithi059.stdout: 1 268835 3.64 MB/sec execute 308 sec latency 28.469 ms 2024-04-02T17:21:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:41 smithi059 ceph-mon[31664]: pgmap v646: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-02T17:21:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:41 smithi164 ceph-mon[36499]: pgmap v646: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-02T17:21:41.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:41 smithi154 ceph-mon[36479]: pgmap v646: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-02T17:21:42.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 269843 3.65 MB/sec execute 309 sec latency 24.023 ms 2024-04-02T17:21:43.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 270088 3.64 MB/sec execute 310 sec latency 22.883 ms 2024-04-02T17:21:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:43 smithi059 ceph-mon[31664]: pgmap v647: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-02T17:21:43.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:43 smithi164 ceph-mon[36499]: pgmap v647: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-02T17:21:43.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:43 smithi154 ceph-mon[36479]: pgmap v647: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-04-02T17:21:44.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 270823 3.63 MB/sec execute 311 sec latency 24.409 ms 2024-04-02T17:21:45.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 272377 3.65 MB/sec execute 312 sec latency 30.906 ms 2024-04-02T17:21:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:45 smithi059 ceph-mon[31664]: pgmap v648: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-02T17:21:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:45 smithi164 ceph-mon[36499]: pgmap v648: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-02T17:21:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:45 smithi154 ceph-mon[36479]: pgmap v648: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-04-02T17:21:46.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 273391 3.66 MB/sec execute 313 sec latency 35.341 ms 2024-04-02T17:21:47.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 273632 3.65 MB/sec execute 314 sec latency 24.358 ms 2024-04-02T17:21:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:47 smithi059 ceph-mon[31664]: pgmap v649: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:21:47.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:47 smithi164 ceph-mon[36499]: pgmap v649: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:21:47.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:47 smithi154 ceph-mon[36479]: pgmap v649: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:21:48.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 274339 3.65 MB/sec execute 315 sec latency 23.289 ms 2024-04-02T17:21:49.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 276043 3.67 MB/sec execute 316 sec latency 17.694 ms 2024-04-02T17:21:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:49 smithi059 ceph-mon[31664]: pgmap v650: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-04-02T17:21:49.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:49 smithi164 ceph-mon[36499]: pgmap v650: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-04-02T17:21:49.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:49 smithi154 ceph-mon[36479]: pgmap v650: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-04-02T17:21:50.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 276969 3.68 MB/sec execute 317 sec latency 57.074 ms 2024-04-02T17:21:50.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:50 smithi154 ceph-mon[36479]: pgmap v651: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-04-02T17:21:51.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:50 smithi059 ceph-mon[31664]: pgmap v651: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-04-02T17:21:51.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:50 smithi164 ceph-mon[36499]: pgmap v651: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.6 MiB/s wr, 194 op/s 2024-04-02T17:21:51.086 INFO:tasks.workunit.client.0.smithi059.stdout: 1 277360 3.67 MB/sec execute 318 sec latency 25.315 ms 2024-04-02T17:21:52.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 278672 3.67 MB/sec execute 319 sec latency 20.465 ms 2024-04-02T17:21:53.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 280123 3.69 MB/sec execute 320 sec latency 15.913 ms 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.116+0000 7f8912a00640 1 -- 172.21.15.59:0/3008208790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c104ef0 msgr2=0x7f890c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.116+0000 7f8912a00640 1 --2- 172.21.15.59:0/3008208790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c104ef0 0x7f890c1072e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009860 tx=0x7f88f802bb70 comp rx=0 tx=0).stop 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 -- 172.21.15.59:0/3008208790 shutdown_connections 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 --2- 172.21.15.59:0/3008208790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 0x7f890c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 --2- 172.21.15.59:0/3008208790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c104ef0 0x7f890c1072e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 --2- 172.21.15.59:0/3008208790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c1025c0 0x7f890c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 -- 172.21.15.59:0/3008208790 >> 172.21.15.59:0/3008208790 conn(0x7f890c0fbfb0 msgr2=0x7f890c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 -- 172.21.15.59:0/3008208790 shutdown_connections 2024-04-02T17:21:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.117+0000 7f8912a00640 1 -- 172.21.15.59:0/3008208790 wait complete. 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 Processor -- start 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 -- start start 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 0x7f890c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 0x7f890c1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10d7b0 con 0x7f890c107820 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10d630 con 0x7f890c104ef0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.118+0000 7f8912a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10d930 con 0x7f890c1025c0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 0x7f890c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46066/0 (socket says 172.21.15.59:46066) 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890be00640 1 -- 172.21.15.59:0/1322411291 learned_addr learned my addr 172.21.15.59:0/1322411291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f8910a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 0x7f890c1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 -- 172.21.15.59:0/1322411291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 msgr2=0x7f890c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 -- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 msgr2=0x7f890c1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 0x7f890c1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890b400640 1 -- 172.21.15.59:0/1322411291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1a1ba0 con 0x7f890c104ef0 2024-04-02T17:21:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.119+0000 7f890be00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.120+0000 7f890b400640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 0x7f890c19a4c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009780 tx=0x7f88f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.120+0000 7f8908c00640 1 -- 172.21.15.59:0/1322411291 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8041020 con 0x7f890c104ef0 2024-04-02T17:21:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.120+0000 7f8908c00640 1 -- 172.21.15.59:0/1322411291 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f80323b0 con 0x7f890c104ef0 2024-04-02T17:21:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.120+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a1e30 con 0x7f890c104ef0 2024-04-02T17:21:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.120+0000 7f8908c00640 1 -- 172.21.15.59:0/1322411291 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8039040 con 0x7f890c104ef0 2024-04-02T17:21:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.121+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a2290 con 0x7f890c104ef0 2024-04-02T17:21:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.124+0000 7f89157dc740 1 Processor -- start 2024-04-02T17:21:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.124+0000 7f89157dc740 1 -- start start 2024-04-02T17:21:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.124+0000 7f8908c00640 1 -- 172.21.15.59:0/1322411291 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f88f8044020 con 0x7f890c104ef0 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a801f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8021ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f7fd70 con 0x55b1a801a8c0 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f282a0 con 0x55b1a801fa30 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f89157dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f7bd20 con 0x55b1a8019ed0 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46080/0 (socket says 172.21.15.59:46080) 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 -- 172.21.15.59:0/670205684 learned_addr learned my addr 172.21.15.59:0/670205684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f890b400640 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8021ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f890be00640 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a801f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 -- 172.21.15.59:0/670205684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 msgr2=0x55b1a8021ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8021ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 -- 172.21.15.59:0/670205684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 msgr2=0x55b1a801f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a801f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.125+0000 7f8910a00640 1 -- 172.21.15.59:0/670205684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a7f3c840 con 0x55b1a8019ed0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.126+0000 7f8910a00640 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8904010370 tx=0x7f8904010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480852f814de342c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.126+0000 7f8901200640 1 -- 172.21.15.59:0/670205684 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904016070 con 0x55b1a8019ed0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.126+0000 7f8901200640 1 -- 172.21.15.59:0/670205684 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904004d20 con 0x55b1a8019ed0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.126+0000 7f8901200640 1 -- 172.21.15.59:0/670205684 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904005480 con 0x55b1a8019ed0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.126+0000 7f8908c00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 0x7f88f00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f8908c00640 1 -- 172.21.15.59:0/1322411291 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f88f80f1ce0 con 0x7f890c104ef0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f890be00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 0x7f88f00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 -- 172.21.15.59:0/670205684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 msgr2=0x55b1a801a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8904010370 tx=0x7f8904010830 comp rx=0 tx=0).stop 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 -- 172.21.15.59:0/670205684 shutdown_connections 2024-04-02T17:21:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a801f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8021ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 --2- 172.21.15.59:0/670205684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a801a2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 -- 172.21.15.59:0/670205684 >> 172.21.15.59:0/670205684 conn(0x55b1a8015440 msgr2=0x55b1a8015840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 -- 172.21.15.59:0/670205684 shutdown_connections 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.127+0000 7f89157dc740 1 -- 172.21.15.59:0/670205684 wait complete. 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f890be00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 0x7f88f00a6110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55b1a80163b0 tx=0x7f88fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f89157dc740 1 -- start start 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f89157dc740 1 Processor -- start 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f89157dc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f89157dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a8024990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.128+0000 7f89157dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801fa30 0x55b1a802b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f89157dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b1a80229e0 con 0x55b1a801fa30 2024-04-02T17:21:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f89157dc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f3b660 con 0x55b1a8019ed0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f89157dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1a8022b60 con 0x55b1a801a8c0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f89157dc740 10 client.0 _ll_register_callbacks cb 0x55b1a7f3b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39610/0 (socket says 172.21.15.59:39610) 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890be00640 1 -- 172.21.15.59:0/1796668955 learned_addr learned my addr 172.21.15.59:0/1796668955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a8024990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f8910a00640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801fa30 0x55b1a802b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 -- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 msgr2=0x55b1a8024450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 -- 172.21.15.59:0/1796668955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801fa30 msgr2=0x55b1a802b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801fa30 0x55b1a802b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 -- 172.21.15.59:0/1796668955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a7f4ae80 con 0x55b1a801a8c0 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890be00640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f890b400640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a8024990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88f80ef7d0 tx=0x7f88f80044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a689c0f2a190959d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.129+0000 7f88f5600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f80ba200 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f88f80c6740 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8904004d20 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88f80bd8c0 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f89157dc740 10 client.26659 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b1a7c94210 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904004d20 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.130+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f88f803c030 con 0x55b1a801a8c0 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 10 client.26659 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 20 client.26659 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a8030390 0x55b1a8032780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 4 client.26659 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:21:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a7f2fcd0 con 0x55b1a8030390 2024-04-02T17:21:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f890be00640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a8030390 0x55b1a8032780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f890be00640 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a8030390 0x55b1a8032780 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cac99e19ca58d95 server_cookie=3fde8ef1597e7d5 in_seq=0 out_seq=0 2024-04-02T17:21:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.131+0000 7f88f5600640 10 client.26659 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.132+0000 7f88f5600640 1 -- 172.21.15.59:0/1796668955 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1a7f2fcd0 con 0x55b1a8030390 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.132+0000 7f88f5600640 10 client.26659 handle_command_reply: tid=0 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.132+0000 7f89157dc740 1 client.26659 shutdown 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 msgr2=0x55b1a8024990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a8024990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88f80ef7d0 tx=0x7f88f80044e0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 shutdown_connections 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a8030390 0x55b1a8032780 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801fa30 0x55b1a802b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a8024990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f88f5600640 0 client.26659 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 --2- 172.21.15.59:0/1796668955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a8019ed0 0x55b1a8024450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 >> 172.21.15.59:0/1796668955 conn(0x55b1a8015440 msgr2=0x55b1a8014640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 shutdown_connections 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 1 -- 172.21.15.59:0/1796668955 wait complete. 2024-04-02T17:21:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.133+0000 7f89157dc740 20 client.26659 trim_cache size 0 max 16384 2024-04-02T17:21:53.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:52 smithi154 ceph-mon[36479]: pgmap v652: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.152+0000 7f89157dc740 1 Processor -- start 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 -- start start 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a80142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ab230 con 0x55b1a8019ed0 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1acda0 con 0x55b1a801fa30 2024-04-02T17:21:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f89157dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ab900 con 0x55b1a801a8c0 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f890b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f890be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f8910a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a80142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f890be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60660/0 (socket says 172.21.15.59:60660) 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f890be00640 1 -- 172.21.15.59:0/2334221798 learned_addr learned my addr 172.21.15.59:0/2334221798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.153+0000 7f890b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46092/0 (socket says 172.21.15.59:46092) 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 -- 172.21.15.59:0/2334221798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 msgr2=0x55b1a802b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 -- 172.21.15.59:0/2334221798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 msgr2=0x55b1a80142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a80142f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 -- 172.21.15.59:0/2334221798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a7d60620 con 0x55b1a8019ed0 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890b400640 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f8910a00640 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a80142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f890be00640 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55b1a8032fc0 tx=0x7f88fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f994cfd0cb5fd78f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f88f7e00640 1 -- 172.21.15.59:0/2334221798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc02b530 con 0x55b1a8019ed0 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f88f7e00640 1 -- 172.21.15.59:0/2334221798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc005500 con 0x55b1a8019ed0 2024-04-02T17:21:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.154+0000 7f88f7e00640 1 -- 172.21.15.59:0/2334221798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc011b90 con 0x55b1a8019ed0 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.155+0000 7f89157dc740 1 -- 172.21.15.59:0/2334221798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 msgr2=0x55b1a8013db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.155+0000 7f89157dc740 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55b1a8032fc0 tx=0x7f88fc01b000 comp rx=0 tx=0).stop 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 -- 172.21.15.59:0/2334221798 shutdown_connections 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a80142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a801a8c0 0x55b1a802b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 --2- 172.21.15.59:0/2334221798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a8019ed0 0x55b1a8013db0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 -- 172.21.15.59:0/2334221798 >> 172.21.15.59:0/2334221798 conn(0x55b1a8015440 msgr2=0x55b1a801be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 -- 172.21.15.59:0/2334221798 shutdown_connections 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 -- 172.21.15.59:0/2334221798 wait complete. 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 -- start start 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.156+0000 7f89157dc740 1 Processor -- start 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a8028f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8029ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ac6d0 con 0x55b1a801a8c0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1abfd0 con 0x55b1a801fa30 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9210 con 0x55b1a8019ed0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f89157dc740 10 client.0 _ll_register_callbacks cb 0x7f88f8005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f890be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f8910a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8029ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a8028f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f890be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46106/0 (socket says 172.21.15.59:46106) 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f890be00640 1 -- 172.21.15.59:0/792389657 learned_addr learned my addr 172.21.15.59:0/792389657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.157+0000 7f8910a00640 1 -- 172.21.15.59:0/792389657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 msgr2=0x55b1a80289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f8910a00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f8910a00640 1 -- 172.21.15.59:0/792389657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 msgr2=0x55b1a8028f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f8910a00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a8028f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f8910a00640 1 -- 172.21.15.59:0/792389657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a802a4e0 con 0x55b1a801fa30 2024-04-02T17:21:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f890be00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f8910a00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8029ef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f8904003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd519b7b1c2a45a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f88f5600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890402a060 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc005500 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f89040052b0 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8904016070 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f89157dc740 10 client.26645 fetch_fsmap learned FSMap version 34 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.158+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b1a802b860 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc005500 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f890402e030 con 0x55b1a801fa30 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 10 client.26645 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 10 client.26645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 20 client.26645 populate_metadata read hostname 'smithi059' 2024-04-02T17:21:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a80969f0 0x55b1a8098de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:21:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 4 client.26645 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:21:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1a7f8d2b0 con 0x55b1a80969f0 2024-04-02T17:21:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.159+0000 7f890be00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a80969f0 0x55b1a8098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:21:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.160+0000 7f890be00640 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a80969f0 0x55b1a8098de0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ebf39801dd81b9 server_cookie=3e6b6ec904bfa46d in_seq=0 out_seq=0 2024-04-02T17:21:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.160+0000 7f88f5600640 10 client.26645 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f88f5600640 1 -- 172.21.15.59:0/792389657 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1a7f8d2b0 con 0x55b1a80969f0 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f88f5600640 10 client.26645 handle_command_reply: tid=0 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 client.26645 shutdown 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 msgr2=0x55b1a8029ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8029ef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8904006010 tx=0x7f8904003ba0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 shutdown_connections 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b1a80969f0 0x55b1a8098de0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1a801a8c0 0x55b1a8028f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1a801fa30 0x55b1a8029ef0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 --2- 172.21.15.59:0/792389657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a8019ed0 0x55b1a80289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f88f5600640 0 client.26645 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.196+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 >> 172.21.15.59:0/792389657 conn(0x55b1a8015440 msgr2=0x55b1a80186a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 shutdown_connections 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f89157dc740 1 -- 172.21.15.59:0/792389657 wait complete. 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f89157dc740 20 client.26645 trim_cache size 0 max 16384 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:21:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 msgr2=0x7f88f00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 0x7f88f00a6110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55b1a80163b0 tx=0x7f88fc009260 comp rx=0 tx=0).stop 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 msgr2=0x7f890c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.197+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 0x7f890c19a4c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009780 tx=0x7f88f80042f0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 shutdown_connections 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88f00a3c50 0x7f88f00a6110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c107820 0x7f890c1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c104ef0 0x7f890c19a4c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 --2- 172.21.15.59:0/1322411291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f890c1025c0 0x7f890c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 >> 172.21.15.59:0/1322411291 conn(0x7f890c0fbfb0 msgr2=0x7f890c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 shutdown_connections 2024-04-02T17:21:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:21:53.198+0000 7f8912a00640 1 -- 172.21.15.59:0/1322411291 wait complete. 2024-04-02T17:21:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:52 smithi059 ceph-mon[31664]: pgmap v652: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:52 smithi164 ceph-mon[36499]: pgmap v652: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-02T17:21:54.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 280610 3.69 MB/sec execute 321 sec latency 147.405 ms 2024-04-02T17:21:55.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 281126 3.68 MB/sec execute 322 sec latency 54.531 ms 2024-04-02T17:21:55.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:54 smithi154 ceph-mon[36479]: pgmap v653: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:21:55.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:54 smithi059 ceph-mon[31664]: pgmap v653: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:21:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:54 smithi164 ceph-mon[36499]: pgmap v653: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:21:56.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 282672 3.69 MB/sec execute 323 sec latency 21.975 ms 2024-04-02T17:21:57.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 283935 3.71 MB/sec execute 324 sec latency 18.392 ms 2024-04-02T17:21:57.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:56 smithi154 ceph-mon[36479]: pgmap v654: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T17:21:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:56 smithi059 ceph-mon[31664]: pgmap v654: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T17:21:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:56 smithi164 ceph-mon[36499]: pgmap v654: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T17:21:58.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 284245 3.71 MB/sec execute 325 sec latency 33.062 ms 2024-04-02T17:21:59.087 INFO:tasks.workunit.client.0.smithi059.stdout: 1 284968 3.70 MB/sec execute 326 sec latency 19.478 ms 2024-04-02T17:21:59.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:21:58 smithi154 ceph-mon[36479]: pgmap v655: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-02T17:21:59.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:21:58 smithi059 ceph-mon[31664]: pgmap v655: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-02T17:21:59.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:21:58 smithi164 ceph-mon[36499]: pgmap v655: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-04-02T17:22:00.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 286685 3.72 MB/sec execute 327 sec latency 49.540 ms 2024-04-02T17:22:01.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 287600 3.73 MB/sec execute 328 sec latency 49.562 ms 2024-04-02T17:22:01.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:00 smithi154 ceph-mon[36479]: pgmap v656: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-02T17:22:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:00 smithi059 ceph-mon[31664]: pgmap v656: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-02T17:22:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:00 smithi164 ceph-mon[36499]: pgmap v656: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-04-02T17:22:02.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 287937 3.72 MB/sec execute 329 sec latency 30.769 ms 2024-04-02T17:22:03.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 289000 3.72 MB/sec execute 330 sec latency 23.876 ms 2024-04-02T17:22:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:02 smithi059 ceph-mon[31664]: pgmap v657: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T17:22:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:02 smithi164 ceph-mon[36499]: pgmap v657: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T17:22:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:02 smithi154 ceph-mon[36479]: pgmap v657: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T17:22:04.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 290671 3.74 MB/sec execute 331 sec latency 16.081 ms 2024-04-02T17:22:05.088 INFO:tasks.workunit.client.0.smithi059.stdout: 1 291267 3.74 MB/sec execute 332 sec latency 52.712 ms 2024-04-02T17:22:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:04 smithi059 ceph-mon[31664]: pgmap v658: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-04-02T17:22:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:04 smithi164 ceph-mon[36499]: pgmap v658: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-04-02T17:22:05.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:04 smithi154 ceph-mon[36479]: pgmap v658: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-04-02T17:22:06.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 291865 3.73 MB/sec execute 333 sec latency 23.176 ms 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 -- 172.21.15.59:0/1286920728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18105910 msgr2=0x7f0a1810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/1286920728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18105910 0x7f0a1810c1a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a1002c100 comp rx=0 tx=0).stop 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 -- 172.21.15.59:0/1286920728 shutdown_connections 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/1286920728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18104f50 0x7f0a181053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/1286920728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18105910 0x7f0a1810c1a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/1286920728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 -- 172.21.15.59:0/1286920728 >> 172.21.15.59:0/1286920728 conn(0x7f0a180ff2f0 msgr2=0x7f0a18101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.347+0000 7f0a1ce00640 1 -- 172.21.15.59:0/1286920728 shutdown_connections 2024-04-02T17:22:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.348+0000 7f0a1ce00640 1 -- 172.21.15.59:0/1286920728 wait complete. 2024-04-02T17:22:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 Processor -- start 2024-04-02T17:22:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 -- start start 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18104f50 0x7f0a1819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18105910 0x7f0a181a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810f9e0 con 0x7f0a18105910 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810f860 con 0x7f0a18104f50 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a1ce00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810fb60 con 0x7f0a18103d60 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a15e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a15e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44816/0 (socket says 172.21.15.59:44816) 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a15e00640 1 -- 172.21.15.59:0/3884145254 learned_addr learned my addr 172.21.15.59:0/3884145254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a15400640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18104f50 0x7f0a1819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.349+0000 7f0a16800640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18105910 0x7f0a181a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 -- 172.21.15.59:0/3884145254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18104f50 msgr2=0x7f0a1819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18104f50 0x7f0a1819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 -- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18105910 msgr2=0x7f0a181a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18105910 0x7f0a181a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 -- 172.21.15.59:0/3884145254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a181a1b90 con 0x7f0a18103d60 2024-04-02T17:22:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a15e00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e9c0 tx=0x7f0a0800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a0ea00640 1 -- 172.21.15.59:0/3884145254 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800cd80 con 0x7f0a18103d60 2024-04-02T17:22:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181a1e80 con 0x7f0a18103d60 2024-04-02T17:22:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a0ea00640 1 -- 172.21.15.59:0/3884145254 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a080035f0 con 0x7f0a18103d60 2024-04-02T17:22:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.350+0000 7f0a0ea00640 1 -- 172.21.15.59:0/3884145254 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010690 con 0x7f0a18103d60 2024-04-02T17:22:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.351+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181a22e0 con 0x7f0a18103d60 2024-04-02T17:22:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.354+0000 7f0a0ea00640 1 -- 172.21.15.59:0/3884145254 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0a080037a0 con 0x7f0a18103d60 2024-04-02T17:22:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.354+0000 7f0a1fb67740 1 Processor -- start 2024-04-02T17:22:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.354+0000 7f0a1fb67740 1 -- start start 2024-04-02T17:22:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d7f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6d81ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563ed6cdfd70 con 0x563ed6d7fa30 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ed6c882a0 con 0x563ed6d79ed0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ed6cdbd20 con 0x563ed6d7a8c0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55070/0 (socket says 172.21.15.59:55070) 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 -- 172.21.15.59:0/2734138762 learned_addr learned my addr 172.21.15.59:0/2734138762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a15e00640 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d7f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a15400640 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6d81ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 -- 172.21.15.59:0/2734138762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 msgr2=0x563ed6d7f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d7f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 -- 172.21.15.59:0/2734138762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 msgr2=0x563ed6d81ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6d81ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.355+0000 7f0a16800640 1 -- 172.21.15.59:0/2734138762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed6c9c840 con 0x563ed6d79ed0 2024-04-02T17:22:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a0ea00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a16800640 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0a1002c710 tx=0x7f0a100323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d38506b636ff9da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a0ea00640 1 -- 172.21.15.59:0/3884145254 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0a080d07a0 con 0x7f0a18103d60 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a15e00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a03400640 1 -- 172.21.15.59:0/2734138762 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10043070 con 0x563ed6d79ed0 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a03400640 1 -- 172.21.15.59:0/2734138762 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10032c90 con 0x563ed6d79ed0 2024-04-02T17:22:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.356+0000 7f0a03400640 1 -- 172.21.15.59:0/2734138762 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10039040 con 0x563ed6d79ed0 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2734138762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 msgr2=0x563ed6d7a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0a1002c710 tx=0x7f0a100323d0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a15e00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a080a5100 tx=0x7f0a080d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2734138762 shutdown_connections 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6d81ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d7f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2734138762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d7a2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2734138762 >> 172.21.15.59:0/2734138762 conn(0x563ed6d75440 msgr2=0x563ed6d75840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2734138762 shutdown_connections 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.357+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2734138762 wait complete. 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 -- start start 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 Processor -- start 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7fa30 0x563ed6d8b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563ed6d82980 con 0x563ed6d79ed0 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ed6c9b660 con 0x563ed6d7fa30 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ed6d82b00 con 0x563ed6d7a8c0 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.358+0000 7f0a1fb67740 10 client.0 _ll_register_callbacks cb 0x563ed6c9b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a16800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a16800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58730/0 (socket says 172.21.15.59:58730) 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a16800640 1 -- 172.21.15.59:0/1451116181 learned_addr learned my addr 172.21.15.59:0/1451116181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a16800640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7fa30 0x563ed6d8b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 -- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7fa30 msgr2=0x563ed6d8b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7fa30 0x563ed6d8b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 -- 172.21.15.59:0/1451116181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 msgr2=0x563ed6d843b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 -- 172.21.15.59:0/1451116181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed6caae80 con 0x563ed6d7a8c0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a16800640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f0a15400640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d848f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a1819b480 tx=0x7f0a04012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbca4a816ca93cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.359+0000 7f09f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04005220 con 0x563ed6d7a8c0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a10032c90 con 0x563ed6d7a8c0 2024-04-02T17:22:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0a040053c0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a040055c0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f0a1fb67740 10 client.26677 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.360+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563ed69f4210 con 0x563ed6d7a8c0 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10032c90 con 0x563ed6d7a8c0 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0a04022020 con 0x563ed6d7a8c0 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 10 client.26677 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 10 client.26677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 20 client.26677 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6d90290 0x563ed6d926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 4 client.26677 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:22:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed6c8fcd0 con 0x563ed6d90290 2024-04-02T17:22:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.361+0000 7f0a16800640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6d90290 0x563ed6d926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.362+0000 7f0a16800640 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6d90290 0x563ed6d926e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493ef072613d6a54 server_cookie=b910c62f1446219b in_seq=0 out_seq=0 2024-04-02T17:22:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.362+0000 7f09f7e00640 10 client.26677 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f09f7e00640 1 -- 172.21.15.59:0/1451116181 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ed6c8fcd0 con 0x563ed6d90290 2024-04-02T17:22:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f09f7e00640 10 client.26677 handle_command_reply: tid=0 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 client.26677 shutdown 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 msgr2=0x563ed6d848f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d848f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a1819b480 tx=0x7f0a04012820 comp rx=0 tx=0).stop 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 shutdown_connections 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6d90290 0x563ed6d926e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7fa30 0x563ed6d8b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7a8c0 0x563ed6d848f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1451116181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d79ed0 0x563ed6d843b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 >> 172.21.15.59:0/1451116181 conn(0x563ed6d75440 msgr2=0x563ed6d74640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f09f7e00640 0 client.26677 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 shutdown_connections 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.363+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1451116181 wait complete. 2024-04-02T17:22:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.364+0000 7f0a1fb67740 20 client.26677 trim_cache size 0 max 16384 2024-04-02T17:22:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 Processor -- start 2024-04-02T17:22:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 -- start start 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 0x563ed6d8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7fa30 0x563ed6d73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181ab270 con 0x563ed6d7a8c0 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181acde0 con 0x563ed6d79ed0 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181ab940 con 0x563ed6d7fa30 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a16800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a16800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55082/0 (socket says 172.21.15.59:55082) 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a16800640 1 -- 172.21.15.59:0/2572028063 learned_addr learned my addr 172.21.15.59:0/2572028063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a15400640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 0x563ed6d8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.383+0000 7f0a16800640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7fa30 0x563ed6d73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a16800640 1 -- 172.21.15.59:0/2572028063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7fa30 msgr2=0x563ed6d73db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a16800640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7fa30 0x563ed6d73db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a16800640 1 -- 172.21.15.59:0/2572028063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 msgr2=0x563ed6d8b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a16800640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 0x563ed6d8b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a16800640 1 -- 172.21.15.59:0/2572028063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed6ac0620 con 0x563ed6d79ed0 2024-04-02T17:22:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.384+0000 7f0a15400640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 0x563ed6d8b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.385+0000 7f0a16800640 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a10009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf54411c0bbf718 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.385+0000 7f0a02a00640 1 -- 172.21.15.59:0/2572028063 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10051040 con 0x563ed6d79ed0 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.385+0000 7f0a02a00640 1 -- 172.21.15.59:0/2572028063 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a100547b0 con 0x563ed6d79ed0 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.385+0000 7f0a02a00640 1 -- 172.21.15.59:0/2572028063 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10039040 con 0x563ed6d79ed0 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2572028063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 msgr2=0x563ed6d75820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a10009540 comp rx=0 tx=0).stop 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2572028063 shutdown_connections 2024-04-02T17:22:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7a8c0 0x563ed6d8b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d7fa30 0x563ed6d73db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/2572028063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d79ed0 0x563ed6d75820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2572028063 >> 172.21.15.59:0/2572028063 conn(0x563ed6d75440 msgr2=0x563ed6d82ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2572028063 shutdown_connections 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.386+0000 7f0a1fb67740 1 -- 172.21.15.59:0/2572028063 wait complete. 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 -- start start 2024-04-02T17:22:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 Processor -- start 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 0x563ed6d88870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6df4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181ac710 con 0x563ed6d7fa30 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181ac010 con 0x563ed6d7a8c0 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.387+0000 7f0a1fb67740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a9250 con 0x563ed6d79ed0 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a1fb67740 10 client.0 _ll_register_callbacks cb 0x7f0a0400b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55096/0 (socket says 172.21.15.59:55096) 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 -- 172.21.15.59:0/1560300590 learned_addr learned my addr 172.21.15.59:0/1560300590 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a16800640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 0x563ed6d88870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a16800640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6df4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 -- 172.21.15.59:0/1560300590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 msgr2=0x563ed6d88870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 0x563ed6d88870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 -- 172.21.15.59:0/1560300590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 msgr2=0x563ed6df4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6df4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 -- 172.21.15.59:0/1560300590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed6df4e40 con 0x563ed6d7a8c0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a16800640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 0x563ed6d88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f0a15400640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x563ed6d85890 tx=0x7f0a040109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ade80147fdddd6a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.388+0000 7f09f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04024a20 con 0x563ed6d7a8c0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f0a04024bc0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a100547b0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a04024dc0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f0a1fb67740 10 client.26663 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563ed6df6190 con 0x563ed6d7a8c0 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.389+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a100547b0 con 0x563ed6d7a8c0 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0a04037050 con 0x563ed6d7a8c0 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 10 client.26663 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 10 client.26663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 20 client.26663 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6df6b10 0x563ed6df8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 4 client.26663 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ed6ced2b0 con 0x563ed6df6b10 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.390+0000 7f0a16800640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6df6b10 0x563ed6df8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.391+0000 7f0a16800640 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6df6b10 0x563ed6df8f30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bab022381c63c5 server_cookie=d68ee5e924f81cec in_seq=0 out_seq=0 2024-04-02T17:22:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.391+0000 7f09f7e00640 10 client.26663 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f09f7e00640 1 -- 172.21.15.59:0/1560300590 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ed6ced2b0 con 0x563ed6df6b10 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f09f7e00640 10 client.26663 handle_command_reply: tid=0 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 client.26663 shutdown 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 msgr2=0x563ed6d88db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x563ed6d85890 tx=0x7f0a040109e0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 shutdown_connections 2024-04-02T17:22:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563ed6df6b10 0x563ed6df8f30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ed6d7fa30 0x563ed6df4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ed6d7a8c0 0x563ed6d88db0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 --2- 172.21.15.59:0/1560300590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ed6d79ed0 0x563ed6d88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 >> 172.21.15.59:0/1560300590 conn(0x563ed6d75440 msgr2=0x563ed6d75820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f09f7e00640 0 client.26663 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.432+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 shutdown_connections 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.433+0000 7f0a1fb67740 1 -- 172.21.15.59:0/1560300590 wait complete. 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.433+0000 7f0a1fb67740 20 client.26663 trim_cache size 0 max 16384 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:22:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:22:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:22:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:22:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 msgr2=0x7f09fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a080a5100 tx=0x7f0a080d6530 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 msgr2=0x7f0a18199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e9c0 tx=0x7f0a0800ee80 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 shutdown_connections 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a18105910 0x7f0a181a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a18104f50 0x7f0a1819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 --2- 172.21.15.59:0/3884145254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a18103d60 0x7f0a18199f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 >> 172.21.15.59:0/3884145254 conn(0x7f0a180ff2f0 msgr2=0x7f0a18108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 shutdown_connections 2024-04-02T17:22:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:06.434+0000 7f0a1ce00640 1 -- 172.21.15.59:0/3884145254 wait complete. 2024-04-02T17:22:07.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 293470 3.74 MB/sec execute 334 sec latency 17.393 ms 2024-04-02T17:22:07.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:06 smithi059 ceph-mon[31664]: pgmap v659: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:22:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:06 smithi164 ceph-mon[36499]: pgmap v659: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:22:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:06 smithi154 ceph-mon[36479]: pgmap v659: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-02T17:22:08.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 294664 3.77 MB/sec execute 335 sec latency 27.243 ms 2024-04-02T17:22:09.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 294980 3.76 MB/sec execute 336 sec latency 19.120 ms 2024-04-02T17:22:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:08 smithi059 ceph-mon[31664]: pgmap v660: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:22:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:08 smithi164 ceph-mon[36499]: pgmap v660: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:22:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:08 smithi154 ceph-mon[36479]: pgmap v660: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:22:10.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 295850 3.76 MB/sec execute 337 sec latency 22.246 ms 2024-04-02T17:22:11.089 INFO:tasks.workunit.client.0.smithi059.stdout: 1 297422 3.77 MB/sec execute 338 sec latency 29.700 ms 2024-04-02T17:22:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:10 smithi059 ceph-mon[31664]: pgmap v661: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-02T17:22:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:10 smithi164 ceph-mon[36499]: pgmap v661: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-02T17:22:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:10 smithi154 ceph-mon[36479]: pgmap v661: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-02T17:22:12.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 298295 3.78 MB/sec execute 339 sec latency 36.639 ms 2024-04-02T17:22:13.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 298728 3.77 MB/sec execute 340 sec latency 20.357 ms 2024-04-02T17:22:13.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:12 smithi059 ceph-mon[31664]: pgmap v662: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:22:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:12 smithi164 ceph-mon[36499]: pgmap v662: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:22:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:12 smithi154 ceph-mon[36479]: pgmap v662: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-02T17:22:14.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 300337 3.78 MB/sec execute 341 sec latency 22.667 ms 2024-04-02T17:22:15.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 301520 3.80 MB/sec execute 342 sec latency 27.078 ms 2024-04-02T17:22:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:14 smithi059 ceph-mon[31664]: pgmap v663: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-02T17:22:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:14 smithi164 ceph-mon[36499]: pgmap v663: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-02T17:22:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:14 smithi154 ceph-mon[36479]: pgmap v663: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-02T17:22:16.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 301968 3.79 MB/sec execute 343 sec latency 26.305 ms 2024-04-02T17:22:16.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:15 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:22:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:15 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:22:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:15 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:22:17.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 302924 3.79 MB/sec execute 344 sec latency 21.103 ms 2024-04-02T17:22:17.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:16 smithi059 ceph-mon[31664]: pgmap v664: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-04-02T17:22:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:16 smithi164 ceph-mon[36499]: pgmap v664: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-04-02T17:22:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:16 smithi154 ceph-mon[36479]: pgmap v664: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-04-02T17:22:18.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 304509 3.80 MB/sec execute 345 sec latency 19.453 ms 2024-04-02T17:22:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:22:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:22:18.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:17 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:22:18.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:22:18.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:22:18.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:17 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:22:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:22:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:22:18.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:17 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:22:19.090 INFO:tasks.workunit.client.0.smithi059.stdout: 1 305351 3.81 MB/sec execute 346 sec latency 23.635 ms 2024-04-02T17:22:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:19 smithi154 ceph-mon[36479]: pgmap v665: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-02T17:22:19.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:19 smithi059 ceph-mon[31664]: pgmap v665: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-02T17:22:19.577 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:19 smithi164 ceph-mon[36499]: pgmap v665: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 -- 172.21.15.59:0/1430028219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c40764c0 msgr2=0x7f93c40768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 --2- 172.21.15.59:0/1430028219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c40764c0 0x7f93c40768e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b402bc40 comp rx=0 tx=0).stop 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 -- 172.21.15.59:0/1430028219 shutdown_connections 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 --2- 172.21.15.59:0/1430028219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40780a0 0x7f93c4078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 --2- 172.21.15.59:0/1430028219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c4077700 0x7f93c4077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 --2- 172.21.15.59:0/1430028219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c40764c0 0x7f93c40768e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 -- 172.21.15.59:0/1430028219 >> 172.21.15.59:0/1430028219 conn(0x7f93c40ff110 msgr2=0x7f93c4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 -- 172.21.15.59:0/1430028219 shutdown_connections 2024-04-02T17:22:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.592+0000 7f93c8e00640 1 -- 172.21.15.59:0/1430028219 wait complete. 2024-04-02T17:22:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 Processor -- start 2024-04-02T17:22:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 -- start start 2024-04-02T17:22:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c4077700 0x7f93c419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c40780a0 0x7f93c41a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4113c10 con 0x7f93c4077700 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4113a90 con 0x7f93c40780a0 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.593+0000 7f93c8e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4113d90 con 0x7f93c40764c0 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44472/0 (socket says 172.21.15.59:44472) 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c40780a0 0x7f93c41a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 -- 172.21.15.59:0/3408078090 learned_addr learned my addr 172.21.15.59:0/3408078090 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93bbe00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c4077700 0x7f93c419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 -- 172.21.15.59:0/3408078090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c40780a0 msgr2=0x7f93c41a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c40780a0 0x7f93c41a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 -- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c4077700 msgr2=0x7f93c419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c4077700 0x7f93c419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.594+0000 7f93c3400640 1 -- 172.21.15.59:0/3408078090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41a5e70 con 0x7f93c40764c0 2024-04-02T17:22:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c3400640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c0c00640 1 -- 172.21.15.59:0/3408078090 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4041070 con 0x7f93c40764c0 2024-04-02T17:22:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c41a6100 con 0x7f93c40764c0 2024-04-02T17:22:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c0c00640 1 -- 172.21.15.59:0/3408078090 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b4004450 con 0x7f93c40764c0 2024-04-02T17:22:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c0c00640 1 -- 172.21.15.59:0/3408078090 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4003d90 con 0x7f93c40764c0 2024-04-02T17:22:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.595+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a6640 con 0x7f93c40764c0 2024-04-02T17:22:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.598+0000 7f93c0c00640 1 -- 172.21.15.59:0/3408078090 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93b4044070 con 0x7f93c40764c0 2024-04-02T17:22:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.599+0000 7f93cd1f0740 1 Processor -- start 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.599+0000 7f93cd1f0740 1 -- start start 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0dac60 0x56554e0dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56554e03ad70 con 0x56554e0dac60 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56554dfe32a0 con 0x56554e0d5100 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56554e036d20 con 0x56554e0d5af0 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93c3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39794/0 (socket says 172.21.15.59:39794) 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93c3e00640 1 -- 172.21.15.59:0/3223215058 learned_addr learned my addr 172.21.15.59:0/3223215058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.600+0000 7f93bbe00640 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0dac60 0x56554e0dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 -- 172.21.15.59:0/3223215058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 msgr2=0x56554e0d5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 -- 172.21.15.59:0/3223215058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0dac60 msgr2=0x56554e0dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0dac60 0x56554e0dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 -- 172.21.15.59:0/3223215058 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554dff7840 con 0x56554e0d5af0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3400640 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0da5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f93b40efa50 tx=0x7f93b40fa1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf52e690ba31ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c0c00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 0x7f93a80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93b8c00640 1 -- 172.21.15.59:0/3223215058 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40ba500 con 0x56554e0d5af0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93b8c00640 1 -- 172.21.15.59:0/3223215058 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40ff030 con 0x56554e0d5af0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93b8c00640 1 -- 172.21.15.59:0/3223215058 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40bcd90 con 0x56554e0d5af0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c0c00640 1 -- 172.21.15.59:0/3408078090 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f93b40f1fe0 con 0x7f93c40764c0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.601+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 0x7f93a80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3223215058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 msgr2=0x56554e0da5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0da5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f93b40efa50 tx=0x7f93b40fa1a0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 0x7f93a80a6060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56554e0d6350 tx=0x7f93bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3223215058 shutdown_connections 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0dac60 0x56554e0dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0da5f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3223215058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0d5100 0x56554e0d5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.602+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3223215058 >> 172.21.15.59:0/3223215058 conn(0x56554e0d0670 msgr2=0x56554e0d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.603+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3223215058 shutdown_connections 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.603+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3223215058 wait complete. 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.603+0000 7f93cd1f0740 1 -- start start 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.603+0000 7f93cd1f0740 1 Processor -- start 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56554e0dd9d0 con 0x56554e0d5af0 2024-04-02T17:22:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56554dff6660 con 0x56554e0dac60 2024-04-02T17:22:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56554e0ddb50 con 0x56554e0d5100 2024-04-02T17:22:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93cd1f0740 10 client.0 _ll_register_callbacks cb 0x56554dff6050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39802/0 (socket says 172.21.15.59:39802) 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93c3e00640 1 -- 172.21.15.59:0/3920227544 learned_addr learned my addr 172.21.15.59:0/3920227544 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.604+0000 7f93c3400640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 -- 172.21.15.59:0/3920227544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 msgr2=0x56554e0df2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0df2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 -- 172.21.15.59:0/3920227544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 msgr2=0x56554e0df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0df7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 -- 172.21.15.59:0/3920227544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554e005e80 con 0x56554e0dac60 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93bc003f50 tx=0x7f93bc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2e41c225096096 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93b1600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc012df0 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f93bc0044c0 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.605+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b40ff030 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f93bc0046c0 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 10 client.26675 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56554dd4f210 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40ff030 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f93bc031020 con 0x56554e0dac60 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 10 client.26675 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 20 client.26675 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e0eb2d0 0x56554e0ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 4 client.26675 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.606+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554dfeacd0 con 0x56554e0eb2d0 2024-04-02T17:22:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.607+0000 7f93c3400640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e0eb2d0 0x56554e0ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.607+0000 7f93c3400640 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e0eb2d0 0x56554e0ed750 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344834a10fce5277 server_cookie=60dddd370856c281 in_seq=0 out_seq=0 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.607+0000 7f93b1600640 10 client.26675 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93b1600640 1 -- 172.21.15.59:0/3920227544 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56554dfeacd0 con 0x56554e0eb2d0 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93b1600640 10 client.26675 handle_command_reply: tid=0 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 client.26675 shutdown 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 msgr2=0x56554e0e6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93bc003f50 tx=0x7f93bc004ab0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 shutdown_connections 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e0eb2d0 0x56554e0ed750 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e6820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3920227544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0df2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.608+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 >> 172.21.15.59:0/3920227544 conn(0x56554e0d0670 msgr2=0x56554e0cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.609+0000 7f93b1600640 0 client.26675 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.609+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 shutdown_connections 2024-04-02T17:22:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.609+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3920227544 wait complete. 2024-04-02T17:22:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.609+0000 7f93cd1f0740 20 client.26675 trim_cache size 0 max 16384 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.627+0000 7f93cd1f0740 1 Processor -- start 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.627+0000 7f93cd1f0740 1 -- start start 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41af610 con 0x56554e0d5af0 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b1180 con 0x56554e0dac60 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41afce0 con 0x56554e0d5100 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93c3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93c3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44508/0 (socket says 172.21.15.59:44508) 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93c3400640 1 -- 172.21.15.59:0/2995095298 learned_addr learned my addr 172.21.15.59:0/2995095298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.628+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 -- 172.21.15.59:0/2995095298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 msgr2=0x56554e0e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 -- 172.21.15.59:0/2995095298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 msgr2=0x56554e0d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0d7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 -- 172.21.15.59:0/2995095298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554de1b620 con 0x56554e0d5100 2024-04-02T17:22:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3e00640 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93c3400640 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93b40efad0 tx=0x7f93b403ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ef6b83fb3a4a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93b3e00640 1 -- 172.21.15.59:0/2995095298 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b404b070 con 0x56554e0d5100 2024-04-02T17:22:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93b3e00640 1 -- 172.21.15.59:0/2995095298 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40ff030 con 0x56554e0d5100 2024-04-02T17:22:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.629+0000 7f93b3e00640 1 -- 172.21.15.59:0/2995095298 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40bcf60 con 0x56554e0d5100 2024-04-02T17:22:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 -- 172.21.15.59:0/2995095298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 msgr2=0x56554e0d6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93b40efad0 tx=0x7f93b403ffa0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 -- 172.21.15.59:0/2995095298 shutdown_connections 2024-04-02T17:22:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5af0 0x56554e0d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e0e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/2995095298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5100 0x56554e0d6f10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 -- 172.21.15.59:0/2995095298 >> 172.21.15.59:0/2995095298 conn(0x56554e0d0670 msgr2=0x56554e0df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 -- 172.21.15.59:0/2995095298 shutdown_connections 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.630+0000 7f93cd1f0740 1 -- 172.21.15.59:0/2995095298 wait complete. 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 -- start start 2024-04-02T17:22:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 Processor -- start 2024-04-02T17:22:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 0x56554e0e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e14f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b0ab0 con 0x56554e0d5100 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b03b0 con 0x56554e0dac60 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ad5f0 con 0x56554e0d5af0 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.631+0000 7f93cd1f0740 10 client.0 _ll_register_callbacks cb 0x7f93bc003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 0x56554e0e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44518/0 (socket says 172.21.15.59:44518) 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 -- 172.21.15.59:0/3495454349 learned_addr learned my addr 172.21.15.59:0/3495454349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 0x56554e0e3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50170/0 (socket says 172.21.15.59:50170) 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93c3e00640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 -- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 msgr2=0x56554e14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 -- 172.21.15.59:0/3495454349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 msgr2=0x56554e0e3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 0x56554e0e3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 -- 172.21.15.59:0/3495454349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554e0e3fb0 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93bbe00640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0128e0 tx=0x7f93ac012da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44228d88e06b0f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.632+0000 7f93b1600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac022020 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f93ac004230 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b40ff030 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f93ac004430 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93cd1f0740 10 client.26698 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56554e150f40 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.633+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b40ff030 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f93ac026020 con 0x56554e0d5af0 2024-04-02T17:22:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 10 client.26698 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 20 client.26698 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e1519a0 0x56554e153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 4 client.26698 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56554e0482b0 con 0x56554e1519a0 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93c3400640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e1519a0 0x56554e153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93c3400640 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e1519a0 0x56554e153dc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e07a97819f213fa server_cookie=29ff0da8a7b7da95 in_seq=0 out_seq=0 2024-04-02T17:22:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.634+0000 7f93b1600640 10 client.26698 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93b1600640 1 -- 172.21.15.59:0/3495454349 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56554e0482b0 con 0x56554e1519a0 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93b1600640 10 client.26698 handle_command_reply: tid=0 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 client.26698 shutdown 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 msgr2=0x56554e0e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0128e0 tx=0x7f93ac012da0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 shutdown_connections 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56554e1519a0 0x56554e153dc0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56554e0dac60 0x56554e14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56554e0d5af0 0x56554e0e39c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 --2- 172.21.15.59:0/3495454349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56554e0d5100 0x56554e0e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93b1600640 0 client.26698 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 >> 172.21.15.59:0/3495454349 conn(0x56554e0d0670 msgr2=0x56554e0d6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 shutdown_connections 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.674+0000 7f93cd1f0740 1 -- 172.21.15.59:0/3495454349 wait complete. 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.675+0000 7f93cd1f0740 20 client.26698 trim_cache size 0 max 16384 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:22:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.675+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 msgr2=0x7f93a80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.675+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 0x7f93a80a6060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56554e0d6350 tx=0x7f93bc009500 comp rx=0 tx=0).stop 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.675+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 msgr2=0x7f93c419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.675+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b402c660 comp rx=0 tx=0).stop 2024-04-02T17:22:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 shutdown_connections 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f93a80a3ba0 0x7f93a80a6060 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93c4077700 0x7f93c419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93c40780a0 0x7f93c41a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 --2- 172.21.15.59:0/3408078090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40764c0 0x7f93c419e250 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 >> 172.21.15.59:0/3408078090 conn(0x7f93c40ff110 msgr2=0x7f93c4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 shutdown_connections 2024-04-02T17:22:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:19.676+0000 7f93c8e00640 1 -- 172.21.15.59:0/3408078090 wait complete. 2024-04-02T17:22:20.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 305727 3.80 MB/sec execute 347 sec latency 40.983 ms 2024-04-02T17:22:21.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 306624 3.81 MB/sec execute 348 sec latency 54.406 ms 2024-04-02T17:22:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:21 smithi059 ceph-mon[31664]: pgmap v666: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-02T17:22:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:21 smithi164 ceph-mon[36499]: pgmap v666: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-02T17:22:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:21 smithi154 ceph-mon[36479]: pgmap v666: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-02T17:22:22.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 308040 3.82 MB/sec execute 349 sec latency 21.414 ms 2024-04-02T17:22:23.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 308877 3.82 MB/sec execute 350 sec latency 24.770 ms 2024-04-02T17:22:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:23 smithi059 ceph-mon[31664]: pgmap v667: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:22:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:23 smithi164 ceph-mon[36499]: pgmap v667: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:22:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:23 smithi154 ceph-mon[36479]: pgmap v667: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:22:24.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 309146 3.81 MB/sec execute 351 sec latency 25.878 ms 2024-04-02T17:22:25.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 309801 3.81 MB/sec execute 352 sec latency 27.851 ms 2024-04-02T17:22:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:25 smithi059 ceph-mon[31664]: pgmap v668: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-02T17:22:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:25 smithi164 ceph-mon[36499]: pgmap v668: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-02T17:22:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:25 smithi154 ceph-mon[36479]: pgmap v668: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-02T17:22:26.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 311326 3.82 MB/sec execute 353 sec latency 35.582 ms 2024-04-02T17:22:27.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 312373 3.83 MB/sec execute 354 sec latency 42.575 ms 2024-04-02T17:22:27.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:27 smithi059 ceph-mon[31664]: pgmap v669: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:22:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:27 smithi164 ceph-mon[36499]: pgmap v669: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:22:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:27 smithi154 ceph-mon[36479]: pgmap v669: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:22:28.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 312609 3.82 MB/sec execute 355 sec latency 28.603 ms 2024-04-02T17:22:29.091 INFO:tasks.workunit.client.0.smithi059.stdout: 1 313263 3.82 MB/sec execute 356 sec latency 38.699 ms 2024-04-02T17:22:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:29 smithi059 ceph-mon[31664]: pgmap v670: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:22:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:29 smithi164 ceph-mon[36499]: pgmap v670: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:22:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:29 smithi154 ceph-mon[36479]: pgmap v670: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:22:30.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 314714 3.82 MB/sec execute 357 sec latency 50.150 ms 2024-04-02T17:22:31.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 315864 3.84 MB/sec execute 358 sec latency 37.678 ms 2024-04-02T17:22:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:31 smithi059 ceph-mon[31664]: pgmap v671: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-04-02T17:22:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:31 smithi164 ceph-mon[36499]: pgmap v671: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-04-02T17:22:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:31 smithi154 ceph-mon[36479]: pgmap v671: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-04-02T17:22:32.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 316115 3.83 MB/sec execute 359 sec latency 158.874 ms 2024-04-02T17:22:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 -- 172.21.15.59:0/3508499653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4109d10 msgr2=0x7f4cc41105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/3508499653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4109d10 0x7f4cc41105a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009ca0 tx=0x7f4cbc02c100 comp rx=0 tx=0).stop 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 -- 172.21.15.59:0/3508499653 shutdown_connections 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/3508499653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109350 0x7f4cc41097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/3508499653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4109d10 0x7f4cc41105a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/3508499653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4108160 0x7f4cc4108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 -- 172.21.15.59:0/3508499653 >> 172.21.15.59:0/3508499653 conn(0x7f4cc4077110 msgr2=0x7f4cc4079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 -- 172.21.15.59:0/3508499653 shutdown_connections 2024-04-02T17:22:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.832+0000 7f4cc9c00640 1 -- 172.21.15.59:0/3508499653 wait complete. 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.833+0000 7f4cc9c00640 1 Processor -- start 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.833+0000 7f4cc9c00640 1 -- start start 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 0x7f4cc419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 0x7f4cc419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 0x7f4cc41a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4113df0 con 0x7f4cc4109d10 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4113c70 con 0x7f4cc4108160 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4113f70 con 0x7f4cc4109350 2024-04-02T17:22:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 0x7f4cc419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 0x7f4cc419e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52210/0 (socket says 172.21.15.59:52210) 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc2a00640 1 -- 172.21.15.59:0/938428924 learned_addr learned my addr 172.21.15.59:0/938428924 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 0x7f4cc419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.834+0000 7f4cc3400640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 0x7f4cc41a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 -- 172.21.15.59:0/938428924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 msgr2=0x7f4cc419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 0x7f4cc419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 -- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 msgr2=0x7f4cc41a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 0x7f4cc41a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 -- 172.21.15.59:0/938428924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc41a5f80 con 0x7f4cc4109350 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc3400640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 0x7f4cc41a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc2000640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 0x7f4cc419e8a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00e9f0 tx=0x7f4cac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cbb400640 1 -- 172.21.15.59:0/938428924 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00ccc0 con 0x7f4cc4109350 2024-04-02T17:22:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc41a6270 con 0x7f4cc4109350 2024-04-02T17:22:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cbb400640 1 -- 172.21.15.59:0/938428924 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac010910 con 0x7f4cc4109350 2024-04-02T17:22:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cbb400640 1 -- 172.21.15.59:0/938428924 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac018960 con 0x7f4cc4109350 2024-04-02T17:22:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.835+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41a6810 con 0x7f4cc4109350 2024-04-02T17:22:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 Processor -- start 2024-04-02T17:22:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 -- start start 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb132f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4cbb400640 1 -- 172.21.15.59:0/938428924 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4cac010ac0 con 0x7f4cc4109350 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb1331ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbb128fd70 con 0x55bbb132fa30 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbb12382a0 con 0x55bbb132a8c0 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.839+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbb128bd20 con 0x55bbb1329ed0 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb132f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47122/0 (socket says 172.21.15.59:47122) 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc3400640 1 -- 172.21.15.59:0/4068280332 learned_addr learned my addr 172.21.15.59:0/4068280332 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2000640 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb1331ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4068280332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 msgr2=0x55bbb132a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4068280332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 msgr2=0x55bbb1331ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb1331ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4068280332 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb124c840 con 0x55bbb132a8c0 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.840+0000 7f4cc3400640 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.841+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb132f3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40058c0 tx=0x7f4cb4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57454a95af6b159 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.841+0000 7f4cb3e00640 1 -- 172.21.15.59:0/4068280332 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4004220 con 0x55bbb132a8c0 2024-04-02T17:22:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.841+0000 7f4cb3e00640 1 -- 172.21.15.59:0/4068280332 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4009e20 con 0x55bbb132a8c0 2024-04-02T17:22:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.841+0000 7f4cb3e00640 1 -- 172.21.15.59:0/4068280332 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4019930 con 0x55bbb132a8c0 2024-04-02T17:22:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.841+0000 7f4cbb400640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 0x7f4ca80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4cbb400640 1 -- 172.21.15.59:0/938428924 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4cac0d0800 con 0x7f4cc4109350 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4068280332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 msgr2=0x55bbb132f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb132f3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40058c0 tx=0x7f4cb4011810 comp rx=0 tx=0).stop 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4cc3400640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 0x7f4ca80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4068280332 shutdown_connections 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb1331ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb132f3c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4068280332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb132a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4068280332 >> 172.21.15.59:0/4068280332 conn(0x55bbb1325440 msgr2=0x55bbb1325840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4068280332 shutdown_connections 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.842+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4068280332 wait complete. 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.843+0000 7f4ccc97b740 1 -- start start 2024-04-02T17:22:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.843+0000 7f4ccc97b740 1 Processor -- start 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.843+0000 7f4cc3400640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 0x7f4ca80a6110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.843+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.843+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb13349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb133b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbb13329e0 con 0x55bbb132fa30 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bbb124b660 con 0x55bbb132a8c0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bbb1332b60 con 0x55bbb1329ed0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4ccc97b740 10 client.0 _ll_register_callbacks cb 0x55bbb124b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47134/0 (socket says 172.21.15.59:47134) 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4237657144 learned_addr learned my addr 172.21.15.59:0/4237657144 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2000640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb13349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc3400640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb133b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 msgr2=0x55bbb13349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb13349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4237657144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 msgr2=0x55bbb133b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb133b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.844+0000 7f4cc2a00640 1 -- 172.21.15.59:0/4237657144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb125ae80 con 0x55bbb1329ed0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4011d10 tx=0x7f4cb4005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb43648a9df2050d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4cb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4028020 con 0x55bbb1329ed0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4cb40048c0 con 0x55bbb1329ed0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb4009e20 con 0x55bbb1329ed0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4cb4033050 con 0x55bbb1329ed0 2024-04-02T17:22:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4ccc97b740 10 client.26710 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.845+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bbb0fa4210 con 0x55bbb1329ed0 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4009e20 con 0x55bbb1329ed0 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f4cb4003b80 con 0x55bbb1329ed0 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 10 client.26710 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 20 client.26710 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13400b0 0x55bbb13424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 4 client.26710 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.846+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb123fcd0 con 0x55bbb13400b0 2024-04-02T17:22:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.847+0000 7f4cc2000640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13400b0 0x55bbb13424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.847+0000 7f4cc2000640 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13400b0 0x55bbb13424a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f9ac223142ca50 server_cookie=aa4684434e134037 in_seq=0 out_seq=0 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.847+0000 7f4cb0c00640 10 client.26710 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4cb0c00640 1 -- 172.21.15.59:0/4237657144 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbb123fcd0 con 0x55bbb13400b0 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4cb0c00640 10 client.26710 handle_command_reply: tid=0 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 client.26710 shutdown 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 msgr2=0x55bbb1334470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4011d10 tx=0x7f4cb4005270 comp rx=0 tx=0).stop 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 shutdown_connections 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13400b0 0x55bbb13424a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132fa30 0x55bbb133b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb13349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/4237657144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1334470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4cb0c00640 0 client.26710 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 >> 172.21.15.59:0/4237657144 conn(0x55bbb1325440 msgr2=0x55bbb1324640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.848+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 shutdown_connections 2024-04-02T17:22:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.849+0000 7f4ccc97b740 1 -- 172.21.15.59:0/4237657144 wait complete. 2024-04-02T17:22:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.849+0000 7f4ccc97b740 20 client.26710 trim_cache size 0 max 16384 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.870+0000 7f4ccc97b740 1 Processor -- start 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.870+0000 7f4ccc97b740 1 -- start start 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.870+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb1329ed0 0x55bbb1332f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb132fa30 0x55bbb13242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41af760 con 0x55bbb1329ed0 2024-04-02T17:22:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41b12d0 con 0x55bbb132a8c0 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41afe30 con 0x55bbb132fa30 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb1329ed0 0x55bbb1332f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52230/0 (socket says 172.21.15.59:52230) 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 -- 172.21.15.59:0/1335268898 learned_addr learned my addr 172.21.15.59:0/1335268898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc3400640 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb132fa30 0x55bbb13242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 -- 172.21.15.59:0/1335268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb132fa30 msgr2=0x55bbb13242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb132fa30 0x55bbb13242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 -- 172.21.15.59:0/1335268898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb1329ed0 msgr2=0x55bbb1332f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb1329ed0 0x55bbb1332f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.871+0000 7f4cc2000640 1 -- 172.21.15.59:0/1335268898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb1070620 con 0x55bbb132a8c0 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.872+0000 7f4cc2000640 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00e9c0 tx=0x7f4cac0d2c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de631dd100f3343 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.872+0000 7f4cb3400640 1 -- 172.21.15.59:0/1335268898 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0a0dd0 con 0x55bbb132a8c0 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.872+0000 7f4cb3400640 1 -- 172.21.15.59:0/1335268898 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac0a5690 con 0x55bbb132a8c0 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.872+0000 7f4cb3400640 1 -- 172.21.15.59:0/1335268898 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0a5460 con 0x55bbb132a8c0 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 -- 172.21.15.59:0/1335268898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 msgr2=0x55bbb1323db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00e9c0 tx=0x7f4cac0d2c50 comp rx=0 tx=0).stop 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 -- 172.21.15.59:0/1335268898 shutdown_connections 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb132fa30 0x55bbb13242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132a8c0 0x55bbb1323db0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/1335268898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb1329ed0 0x55bbb1332f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 -- 172.21.15.59:0/1335268898 >> 172.21.15.59:0/1335268898 conn(0x55bbb1325440 msgr2=0x55bbb1325820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 -- 172.21.15.59:0/1335268898 shutdown_connections 2024-04-02T17:22:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.873+0000 7f4ccc97b740 1 -- 172.21.15.59:0/1335268898 wait complete. 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.874+0000 7f4ccc97b740 1 -- start start 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.874+0000 7f4ccc97b740 1 Processor -- start 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132a8c0 0x55bbb1338250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132fa30 0x55bbb13a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41b0c00 con 0x55bbb132a8c0 2024-04-02T17:22:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41b0500 con 0x55bbb132fa30 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41ad740 con 0x55bbb1329ed0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4ccc97b740 10 client.0 _ll_register_callbacks cb 0x7f4cb4004360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47144/0 (socket says 172.21.15.59:47144) 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 -- 172.21.15.59:0/669633610 learned_addr learned my addr 172.21.15.59:0/669633610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc3400640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132fa30 0x55bbb13a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2000640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132a8c0 0x55bbb1338250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 -- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132fa30 msgr2=0x55bbb13a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132fa30 0x55bbb13a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 -- 172.21.15.59:0/669633610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132a8c0 msgr2=0x55bbb1338250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132a8c0 0x55bbb1338250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.875+0000 7f4cc2a00640 1 -- 172.21.15.59:0/669633610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb13a4b50 con 0x55bbb1329ed0 2024-04-02T17:22:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4cc2a00640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009380 tx=0x7f4cb4009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf1ff9893c0411b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4cb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb403c060 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f4cb4019880 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cac0a5690 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4cb4028040 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4ccc97b740 10 client.26716 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.876+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bbb13a5ce0 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac0a5690 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f4cb4040030 con 0x55bbb1329ed0 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 10 client.26716 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 10 client.26716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 20 client.26716 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13a6610 0x55bbb13a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 4 client.26716 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:22:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbb129d2b0 con 0x55bbb13a6610 2024-04-02T17:22:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.877+0000 7f4cc2000640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13a6610 0x55bbb13a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.878+0000 7f4cc2000640 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13a6610 0x55bbb13a8a30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e5132c5b090c69 server_cookie=a6100e20b7e16e6c in_seq=0 out_seq=0 2024-04-02T17:22:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.878+0000 7f4cb0c00640 10 client.26716 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.915+0000 7f4cb0c00640 1 -- 172.21.15.59:0/669633610 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbb129d2b0 con 0x55bbb13a6610 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.915+0000 7f4cb0c00640 10 client.26716 handle_command_reply: tid=0 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 client.26716 shutdown 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 msgr2=0x55bbb1337d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009380 tx=0x7f4cb4009b20 comp rx=0 tx=0).stop 2024-04-02T17:22:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 shutdown_connections 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bbb13a6610 0x55bbb13a8a30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb132a8c0 0x55bbb1338250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bbb132fa30 0x55bbb13a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 --2- 172.21.15.59:0/669633610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbb1329ed0 0x55bbb1337d10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4cb0c00640 0 client.26716 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 >> 172.21.15.59:0/669633610 conn(0x55bbb1325440 msgr2=0x55bbb1325820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.916+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 shutdown_connections 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.917+0000 7f4ccc97b740 1 -- 172.21.15.59:0/669633610 wait complete. 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.917+0000 7f4ccc97b740 20 client.26716 trim_cache size 0 max 16384 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:22:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 msgr2=0x7f4ca80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 0x7f4ca80a6110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc032590 comp rx=0 tx=0).stop 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 msgr2=0x7f4cc419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 0x7f4cc419e8a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00e9f0 tx=0x7f4cac00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 shutdown_connections 2024-04-02T17:22:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4ca80a3c50 0x7f4ca80a6110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc4109d10 0x7f4cc41a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cc4109350 0x7f4cc419e8a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 --2- 172.21.15.59:0/938428924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cc4108160 0x7f4cc419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 >> 172.21.15.59:0/938428924 conn(0x7f4cc4077110 msgr2=0x7f4cc410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 shutdown_connections 2024-04-02T17:22:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:32.918+0000 7f4cc9c00640 1 -- 172.21.15.59:0/938428924 wait complete. 2024-04-02T17:22:33.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 316684 3.83 MB/sec execute 360 sec latency 23.413 ms 2024-04-02T17:22:33.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:33 smithi154 ceph-mon[36479]: pgmap v672: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-04-02T17:22:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:33 smithi059 ceph-mon[31664]: pgmap v672: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-04-02T17:22:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:33 smithi164 ceph-mon[36499]: pgmap v672: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2024-04-02T17:22:34.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 318146 3.83 MB/sec execute 361 sec latency 22.293 ms 2024-04-02T17:22:35.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 319336 3.85 MB/sec execute 362 sec latency 17.533 ms 2024-04-02T17:22:35.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:35 smithi059 ceph-mon[31664]: pgmap v673: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:22:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:35 smithi164 ceph-mon[36499]: pgmap v673: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:22:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:35 smithi154 ceph-mon[36479]: pgmap v673: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:22:36.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 319667 3.84 MB/sec execute 363 sec latency 30.215 ms 2024-04-02T17:22:37.092 INFO:tasks.workunit.client.0.smithi059.stdout: 1 320183 3.84 MB/sec execute 364 sec latency 23.409 ms 2024-04-02T17:22:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:37 smithi059 ceph-mon[31664]: pgmap v674: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:22:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:37 smithi164 ceph-mon[36499]: pgmap v674: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:22:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:37 smithi154 ceph-mon[36479]: pgmap v674: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-02T17:22:38.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 321677 3.84 MB/sec execute 365 sec latency 22.188 ms 2024-04-02T17:22:39.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 322864 3.86 MB/sec execute 366 sec latency 19.073 ms 2024-04-02T17:22:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:39 smithi059 ceph-mon[31664]: pgmap v675: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:22:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:39 smithi164 ceph-mon[36499]: pgmap v675: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:22:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:39 smithi154 ceph-mon[36479]: pgmap v675: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:22:40.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 323179 3.85 MB/sec execute 367 sec latency 34.508 ms 2024-04-02T17:22:41.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 323678 3.85 MB/sec execute 368 sec latency 28.464 ms 2024-04-02T17:22:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:41 smithi059 ceph-mon[31664]: pgmap v676: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-02T17:22:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:41 smithi164 ceph-mon[36499]: pgmap v676: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-02T17:22:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:41 smithi154 ceph-mon[36479]: pgmap v676: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-04-02T17:22:42.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 325133 3.85 MB/sec execute 369 sec latency 24.322 ms 2024-04-02T17:22:43.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 326163 3.86 MB/sec execute 370 sec latency 18.152 ms 2024-04-02T17:22:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:43 smithi059 ceph-mon[31664]: pgmap v677: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:22:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:43 smithi164 ceph-mon[36499]: pgmap v677: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:22:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:43 smithi154 ceph-mon[36479]: pgmap v677: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:22:44.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 326712 3.86 MB/sec execute 371 sec latency 27.045 ms 2024-04-02T17:22:45.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 327122 3.86 MB/sec execute 372 sec latency 30.506 ms 2024-04-02T17:22:45.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:45 smithi059 ceph-mon[31664]: pgmap v678: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-02T17:22:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:45 smithi164 ceph-mon[36499]: pgmap v678: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-02T17:22:45.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:45 smithi154 ceph-mon[36479]: pgmap v678: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-02T17:22:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 -- 172.21.15.59:0/1622050670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4102640 msgr2=0x7fbad4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 --2- 172.21.15.59:0/1622050670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4102640 0x7fbad4104a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009ca0 tx=0x7fbac402c100 comp rx=0 tx=0).stop 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 -- 172.21.15.59:0/1622050670 shutdown_connections 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 --2- 172.21.15.59:0/1622050670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41078a0 0x7fbad4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 --2- 172.21.15.59:0/1622050670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad4104f70 0x7fbad4107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 --2- 172.21.15.59:0/1622050670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4102640 0x7fbad4104a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 -- 172.21.15.59:0/1622050670 >> 172.21.15.59:0/1622050670 conn(0x7fbad40fbfd0 msgr2=0x7fbad40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 -- 172.21.15.59:0/1622050670 shutdown_connections 2024-04-02T17:22:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.084+0000 7fbadba00640 1 -- 172.21.15.59:0/1622050670 wait complete. 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 Processor -- start 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 -- start start 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4102640 0x7fbad4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 0x7fbad419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad410d800 con 0x7fbad4102640 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbad410d680 con 0x7fbad41078a0 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.085+0000 7fbadba00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbad410d980 con 0x7fbad4104f70 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 0x7fbad419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4102640 0x7fbad4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38602/0 (socket says 172.21.15.59:38602) 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 0x7fbad419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34074/0 (socket says 172.21.15.59:34074) 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 -- 172.21.15.59:0/1740236938 learned_addr learned my addr 172.21.15.59:0/1740236938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 -- 172.21.15.59:0/1740236938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 msgr2=0x7fbad419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 0x7fbad419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 -- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4102640 msgr2=0x7fbad4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4102640 0x7fbad4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 -- 172.21.15.59:0/1740236938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41a1c10 con 0x7fbad41078a0 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.086+0000 7fbad9a00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e970 tx=0x7fbacc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.087+0000 7fbad1600640 1 -- 172.21.15.59:0/1740236938 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ccc0 con 0x7fbad41078a0 2024-04-02T17:22:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.087+0000 7fbad1600640 1 -- 172.21.15.59:0/1740236938 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc00ce60 con 0x7fbad41078a0 2024-04-02T17:22:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.087+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41a1f00 con 0x7fbad41078a0 2024-04-02T17:22:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.087+0000 7fbad1600640 1 -- 172.21.15.59:0/1740236938 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc010490 con 0x7fbad41078a0 2024-04-02T17:22:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.087+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a2370 con 0x7fbad41078a0 2024-04-02T17:22:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.091+0000 7fbade7e8740 1 Processor -- start 2024-04-02T17:22:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbad1600640 1 -- 172.21.15.59:0/1740236938 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbacc005020 con 0x7fbad41078a0 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 -- start start 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba3292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba32ea30 0x5619ba330ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5619ba28ed70 con 0x5619ba328ed0 2024-04-02T17:22:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5619ba2372a0 con 0x5619ba3298c0 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbade7e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5619ba28ad20 con 0x5619ba32ea30 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbad9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbad9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38606/0 (socket says 172.21.15.59:38606) 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.092+0000 7fbad9000640 1 -- 172.21.15.59:0/1350273044 learned_addr learned my addr 172.21.15.59:0/1350273044 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba3292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 -- 172.21.15.59:0/1350273044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba32ea30 msgr2=0x5619ba330ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba32ea30 0x5619ba330ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 -- 172.21.15.59:0/1350273044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 msgr2=0x5619ba3292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba3292f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 -- 172.21.15.59:0/1350273044 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba24b840 con 0x5619ba3298c0 2024-04-02T17:22:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba3292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.093+0000 7fbad9000640 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005e30 tx=0x7fbac4005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c2a90ec6c795598 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbad1600640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 0x7fbab80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbaca000640 1 -- 172.21.15.59:0/1350273044 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4043070 con 0x5619ba3298c0 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbaca000640 1 -- 172.21.15.59:0/1350273044 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4003970 con 0x5619ba3298c0 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbaca000640 1 -- 172.21.15.59:0/1350273044 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4039040 con 0x5619ba3298c0 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbad1600640 1 -- 172.21.15.59:0/1740236938 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fbacc0d0930 con 0x7fbad41078a0 2024-04-02T17:22:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.094+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 0x7fbab80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 328333 3.86 MB/sec execute 373 sec latency 27.853 ms 2024-04-02T17:22:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- 172.21.15.59:0/1350273044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 msgr2=0x5619ba32e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005e30 tx=0x7fbac4005f60 comp rx=0 tx=0).stop 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- 172.21.15.59:0/1350273044 shutdown_connections 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba32ea30 0x5619ba330ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba32e3c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 --2- 172.21.15.59:0/1350273044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba3292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- 172.21.15.59:0/1350273044 >> 172.21.15.59:0/1350273044 conn(0x5619ba324440 msgr2=0x5619ba324840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 0x7fbab80a6110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009710 tx=0x7fbac0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- 172.21.15.59:0/1350273044 shutdown_connections 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- 172.21.15.59:0/1350273044 wait complete. 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.095+0000 7fbade7e8740 1 -- start start 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 Processor -- start 2024-04-02T17:22:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba3298c0 0x5619ba3338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba33a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5619ba331980 con 0x5619ba328ed0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5619ba24a660 con 0x5619ba32ea30 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5619ba331b00 con 0x5619ba3298c0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbade7e8740 10 client.0 _ll_register_callbacks cb 0x5619ba24a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbad9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba3298c0 0x5619ba3338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbad9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54048/0 (socket says 172.21.15.59:54048) 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba33a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.096+0000 7fbad9000640 1 -- 172.21.15.59:0/4046747760 learned_addr learned my addr 172.21.15.59:0/4046747760 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 -- 172.21.15.59:0/4046747760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba3298c0 msgr2=0x5619ba3338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba3298c0 0x5619ba3338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 -- 172.21.15.59:0/4046747760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 msgr2=0x5619ba333390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 -- 172.21.15.59:0/4046747760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ba259e80 con 0x5619ba32ea30 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9000640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbad9a00640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba33a910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbacc004fd0 tx=0x7fbacc09cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d94baba488bd9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbabea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0bd070 con 0x5619ba32ea30 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fbacc005310 con 0x5619ba32ea30 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.097+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac4003970 con 0x5619ba32ea30 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0a5330 con 0x5619ba32ea30 2024-04-02T17:22:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 10 client.26711 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619b9fa3210 con 0x5619ba32ea30 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4003970 con 0x5619ba32ea30 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbacc09ba10 con 0x5619ba32ea30 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 10 client.26711 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 20 client.26711 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba33f200 0x5619ba341630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 4 client.26711 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.098+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ba23ecd0 con 0x5619ba33f200 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.099+0000 7fbad9000640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba33f200 0x5619ba341630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.099+0000 7fbad9000640 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba33f200 0x5619ba341630 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f481f96b341f9 server_cookie=d6ae58b945cffed2 in_seq=0 out_seq=0 2024-04-02T17:22:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.099+0000 7fbabea00640 10 client.26711 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbabea00640 1 -- 172.21.15.59:0/4046747760 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619ba23ecd0 con 0x5619ba33f200 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbabea00640 10 client.26711 handle_command_reply: tid=0 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 client.26711 shutdown 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 msgr2=0x5619ba33a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba33a910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbacc004fd0 tx=0x7fbacc09cb50 comp rx=0 tx=0).stop 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 shutdown_connections 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba33f200 0x5619ba341630 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba33a910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba3298c0 0x5619ba3338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4046747760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba328ed0 0x5619ba333390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 >> 172.21.15.59:0/4046747760 conn(0x5619ba324440 msgr2=0x5619ba323640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbabea00640 0 client.26711 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 shutdown_connections 2024-04-02T17:22:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 1 -- 172.21.15.59:0/4046747760 wait complete. 2024-04-02T17:22:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.101+0000 7fbade7e8740 20 client.26711 trim_cache size 0 max 16384 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 Processor -- start 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 -- start start 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 0x5619ba3323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba322db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ab350 con 0x5619ba3298c0 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41acec0 con 0x5619ba32ea30 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbade7e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41aba20 con 0x5619ba328ed0 2024-04-02T17:22:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.119+0000 7fbad9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:34104/0 (socket says 172.21.15.59:34104) 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 -- 172.21.15.59:0/4229608921 learned_addr learned my addr 172.21.15.59:0/4229608921 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9a00640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba322db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 0x5619ba3323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 -- 172.21.15.59:0/4229608921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 msgr2=0x5619ba322db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba322db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 -- 172.21.15.59:0/4229608921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 msgr2=0x5619ba3323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 0x5619ba3323e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 -- 172.21.15.59:0/4229608921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba06f620 con 0x5619ba328ed0 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 0x5619ba3323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:22:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbad9000640 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009ca0 tx=0x7fbac4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b977e53ef6fa4ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.120+0000 7fbac9600640 1 -- 172.21.15.59:0/4229608921 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4039040 con 0x5619ba328ed0 2024-04-02T17:22:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbac9600640 1 -- 172.21.15.59:0/4229608921 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4003e50 con 0x5619ba328ed0 2024-04-02T17:22:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbac9600640 1 -- 172.21.15.59:0/4229608921 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4043040 con 0x5619ba328ed0 2024-04-02T17:22:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 -- 172.21.15.59:0/4229608921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 msgr2=0x5619ba331ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009ca0 tx=0x7fbac4005ef0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 -- 172.21.15.59:0/4229608921 shutdown_connections 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba3298c0 0x5619ba3323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba32ea30 0x5619ba322db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 --2- 172.21.15.59:0/4229608921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba331ea0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.121+0000 7fbade7e8740 1 -- 172.21.15.59:0/4229608921 >> 172.21.15.59:0/4229608921 conn(0x5619ba324440 msgr2=0x5619ba333390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.122+0000 7fbade7e8740 1 -- 172.21.15.59:0/4229608921 shutdown_connections 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.122+0000 7fbade7e8740 1 -- 172.21.15.59:0/4229608921 wait complete. 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.122+0000 7fbade7e8740 1 -- start start 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.122+0000 7fbade7e8740 1 Processor -- start 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba336b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba337040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba32ea30 0x5619ba3a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ac7f0 con 0x5619ba32ea30 2024-04-02T17:22:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ac0f0 con 0x5619ba3298c0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41a9330 con 0x5619ba328ed0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbade7e8740 10 client.0 _ll_register_callbacks cb 0x7fbacc09c4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba337040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba336b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba337040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38634/0 (socket says 172.21.15.59:38634) 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad3e00640 1 -- 172.21.15.59:0/2527828800 learned_addr learned my addr 172.21.15.59:0/2527828800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9a00640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba32ea30 0x5619ba3a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 -- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 msgr2=0x5619ba337040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba337040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 -- 172.21.15.59:0/2527828800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba32ea30 msgr2=0x5619ba3a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba32ea30 0x5619ba3a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.123+0000 7fbad9000640 1 -- 172.21.15.59:0/2527828800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ba33a0d0 con 0x5619ba328ed0 2024-04-02T17:22:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbad9000640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba336b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005e30 tx=0x7fbac4032e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50dc407c14ab7162 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbabea00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4039070 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fbac4050ae0 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac4003e50 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.124+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbac4043040 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 10 client.26731 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619ba3a4d20 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4003e50 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbac4004480 con 0x5619ba328ed0 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 10 client.26731 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 10 client.26731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 20 client.26731 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba3a5340 0x5619ba3a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 4 client.26731 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:22:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.125+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619ba29c2b0 con 0x5619ba3a5340 2024-04-02T17:22:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.126+0000 7fbad3e00640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba3a5340 0x5619ba3a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.126+0000 7fbad3e00640 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba3a5340 0x5619ba3a7730 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a43b45c9ce5f93 server_cookie=5fcd8930de9f2408 in_seq=0 out_seq=0 2024-04-02T17:22:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.126+0000 7fbabea00640 10 client.26731 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbabea00640 1 -- 172.21.15.59:0/2527828800 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619ba29c2b0 con 0x5619ba3a5340 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbabea00640 10 client.26731 handle_command_reply: tid=0 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 client.26731 shutdown 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 msgr2=0x5619ba336b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba336b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005e30 tx=0x7fbac4032e80 comp rx=0 tx=0).stop 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 shutdown_connections 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5619ba3a5340 0x5619ba3a7730 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619ba32ea30 0x5619ba3a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba3298c0 0x5619ba337040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 --2- 172.21.15.59:0/2527828800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5619ba328ed0 0x5619ba336b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbabea00640 0 client.26731 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 >> 172.21.15.59:0/2527828800 conn(0x5619ba324440 msgr2=0x5619ba333310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 shutdown_connections 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 1 -- 172.21.15.59:0/2527828800 wait complete. 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.174+0000 7fbade7e8740 20 client.26731 trim_cache size 0 max 16384 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:22:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:22:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 msgr2=0x7fbab80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 0x7fbab80a6110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009710 tx=0x7fbac0009920 comp rx=0 tx=0).stop 2024-04-02T17:22:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 msgr2=0x7fbad41a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e970 tx=0x7fbacc00ed70 comp rx=0 tx=0).stop 2024-04-02T17:22:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 shutdown_connections 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbad41078a0 0x7fbad41a15a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbad4104f70 0x7fbad419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbab80a3c50 0x7fbab80a6110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 --2- 172.21.15.59:0/1740236938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4102640 0x7fbad4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 >> 172.21.15.59:0/1740236938 conn(0x7fbad40fbfd0 msgr2=0x7fbad4101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 shutdown_connections 2024-04-02T17:22:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:46.175+0000 7fbadba00640 1 -- 172.21.15.59:0/1740236938 wait complete. 2024-04-02T17:22:47.093 INFO:tasks.workunit.client.0.smithi059.stdout: 1 329674 3.87 MB/sec execute 374 sec latency 20.428 ms 2024-04-02T17:22:47.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:47 smithi059 ceph-mon[31664]: pgmap v679: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-02T17:22:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:47 smithi164 ceph-mon[36499]: pgmap v679: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-02T17:22:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:47 smithi154 ceph-mon[36479]: pgmap v679: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-02T17:22:48.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 330234 3.87 MB/sec execute 375 sec latency 39.582 ms 2024-04-02T17:22:49.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 330618 3.86 MB/sec execute 376 sec latency 23.060 ms 2024-04-02T17:22:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:49 smithi059 ceph-mon[31664]: pgmap v680: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:22:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:49 smithi164 ceph-mon[36499]: pgmap v680: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:22:49.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:49 smithi154 ceph-mon[36479]: pgmap v680: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T17:22:50.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 331711 3.87 MB/sec execute 377 sec latency 26.308 ms 2024-04-02T17:22:51.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 333187 3.88 MB/sec execute 378 sec latency 22.225 ms 2024-04-02T17:22:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:51 smithi059 ceph-mon[31664]: pgmap v681: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-02T17:22:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:51 smithi164 ceph-mon[36499]: pgmap v681: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-02T17:22:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:51 smithi154 ceph-mon[36479]: pgmap v681: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-02T17:22:52.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 333823 3.88 MB/sec execute 379 sec latency 46.284 ms 2024-04-02T17:22:53.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 334372 3.88 MB/sec execute 380 sec latency 21.101 ms 2024-04-02T17:22:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:53 smithi059 ceph-mon[31664]: pgmap v682: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-04-02T17:22:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:53 smithi164 ceph-mon[36499]: pgmap v682: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-04-02T17:22:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:53 smithi154 ceph-mon[36479]: pgmap v682: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-04-02T17:22:54.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 336023 3.88 MB/sec execute 381 sec latency 20.473 ms 2024-04-02T17:22:55.094 INFO:tasks.workunit.client.0.smithi059.stdout: 1 337215 3.90 MB/sec execute 382 sec latency 23.557 ms 2024-04-02T17:22:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:55 smithi059 ceph-mon[31664]: pgmap v683: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-02T17:22:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:55 smithi164 ceph-mon[36499]: pgmap v683: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-02T17:22:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:55 smithi154 ceph-mon[36479]: pgmap v683: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-02T17:22:56.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 337527 3.89 MB/sec execute 383 sec latency 29.856 ms 2024-04-02T17:22:57.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 338470 3.90 MB/sec execute 384 sec latency 25.181 ms 2024-04-02T17:22:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:57 smithi059 ceph-mon[31664]: pgmap v684: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:22:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:57 smithi164 ceph-mon[36499]: pgmap v684: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:22:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:57 smithi154 ceph-mon[36479]: pgmap v684: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:22:58.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 340171 3.91 MB/sec execute 385 sec latency 20.047 ms 2024-04-02T17:22:59.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 340882 3.91 MB/sec execute 386 sec latency 27.357 ms 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 -- 172.21.15.59:0/4072883626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 msgr2=0x7f02781053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 --2- 172.21.15.59:0/4072883626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f02781053c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0264009900 tx=0x7f026402bc30 comp rx=0 tx=0).stop 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 -- 172.21.15.59:0/4072883626 shutdown_connections 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 --2- 172.21.15.59:0/4072883626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f027810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 --2- 172.21.15.59:0/4072883626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f02781053c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 --2- 172.21.15.59:0/4072883626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 0x7f0278104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 -- 172.21.15.59:0/4072883626 >> 172.21.15.59:0/4072883626 conn(0x7f02780ff2c0 msgr2=0x7f0278101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.326+0000 7f027de00640 1 -- 172.21.15.59:0/4072883626 shutdown_connections 2024-04-02T17:22:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.327+0000 7f027de00640 1 -- 172.21.15.59:0/4072883626 wait complete. 2024-04-02T17:22:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 Processor -- start 2024-04-02T17:22:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 -- start start 2024-04-02T17:22:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 0x7f0278199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f02781a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f027810f9b0 con 0x7f0278103d50 2024-04-02T17:22:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f027810f830 con 0x7f0278105900 2024-04-02T17:22:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.328+0000 7f027de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f027810fb30 con 0x7f0278104f40 2024-04-02T17:22:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0276000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0276a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 0x7f0278199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f02781a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0276000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43734/0 (socket says 172.21.15.59:43734) 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0276000640 1 -- 172.21.15.59:0/327545754 learned_addr learned my addr 172.21.15.59:0/327545754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 -- 172.21.15.59:0/327545754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 msgr2=0x7f027819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 -- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 msgr2=0x7f0278199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 0x7f0278199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 -- 172.21.15.59:0/327545754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a1b80 con 0x7f0278105900 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0276000640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.329+0000 7f0277400640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f02781a1510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f027000e940 tx=0x7f027000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.330+0000 7f027ca00640 1 -- 172.21.15.59:0/327545754 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270004d20 con 0x7f0278105900 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.330+0000 7f027ca00640 1 -- 172.21.15.59:0/327545754 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0270005020 con 0x7f0278105900 2024-04-02T17:22:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.330+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a1e70 con 0x7f0278105900 2024-04-02T17:22:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.330+0000 7f027ca00640 1 -- 172.21.15.59:0/327545754 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700116b0 con 0x7f0278105900 2024-04-02T17:22:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.330+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a2330 con 0x7f0278105900 2024-04-02T17:22:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.333+0000 7f027ca00640 1 -- 172.21.15.59:0/327545754 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f02700054d0 con 0x7f0278105900 2024-04-02T17:22:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.333+0000 7f0280be6740 1 Processor -- start 2024-04-02T17:22:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 -- start start 2024-04-02T17:22:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74003c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f740043ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f73ffa1d70 con 0x55f740041a30 2024-04-02T17:22:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f73ff4a2a0 con 0x55f74003c8c0 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0280be6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f73ff9dd20 con 0x55f74003bed0 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0276a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0276a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51186/0 (socket says 172.21.15.59:51186) 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0276a00640 1 -- 172.21.15.59:0/3556654041 learned_addr learned my addr 172.21.15.59:0/3556654041 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.334+0000 7f0276000640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74003c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276000640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f740043ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 -- 172.21.15.59:0/3556654041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 msgr2=0x55f74003c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74003c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 -- 172.21.15.59:0/3556654041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 msgr2=0x55f740043ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f740043ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 -- 172.21.15.59:0/3556654041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73ff5e840 con 0x55f74003c8c0 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276000640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74003c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f027ca00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 0x7f025c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.335+0000 7f0276a00640 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0268009ca0 tx=0x7f0268011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29c63e36855b2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f027ca00640 1 -- 172.21.15.59:0/327545754 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f02700d0480 con 0x7f0278105900 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0263e00640 1 -- 172.21.15.59:0/3556654041 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268004220 con 0x55f74003c8c0 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0263e00640 1 -- 172.21.15.59:0/3556654041 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02680043c0 con 0x55f74003c8c0 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0263e00640 1 -- 172.21.15.59:0/3556654041 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268019930 con 0x55f74003c8c0 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0276000640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 0x7f025c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0280be6740 1 -- 172.21.15.59:0/3556654041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 msgr2=0x55f7400413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0280be6740 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0268009ca0 tx=0x7f0268011810 comp rx=0 tx=0).stop 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.336+0000 7f0276000640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 0x7f025c0a6190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0264000c00 tx=0x7f026402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 -- 172.21.15.59:0/3556654041 shutdown_connections 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f740043ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f7400413c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 --2- 172.21.15.59:0/3556654041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74003c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 -- 172.21.15.59:0/3556654041 >> 172.21.15.59:0/3556654041 conn(0x55f740037440 msgr2=0x55f740037840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 -- 172.21.15.59:0/3556654041 shutdown_connections 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 -- 172.21.15.59:0/3556654041 wait complete. 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 -- start start 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.337+0000 7f0280be6740 1 Processor -- start 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003bed0 0x55f7400463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003c8c0 0x55f740046930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7400449e0 con 0x55f74003bed0 2024-04-02T17:22:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f73ff5d660 con 0x55f740041a30 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f740044b60 con 0x55f74003c8c0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0280be6740 10 client.0 _ll_register_callbacks cb 0x55f73ff5d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0277400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0276000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003c8c0 0x55f740046930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0277400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51200/0 (socket says 172.21.15.59:51200) 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0277400640 1 -- 172.21.15.59:0/2833080584 learned_addr learned my addr 172.21.15.59:0/2833080584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.338+0000 7f0276a00640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003bed0 0x55f7400463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 -- 172.21.15.59:0/2833080584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003c8c0 msgr2=0x55f740046930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003c8c0 0x55f740046930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 -- 172.21.15.59:0/2833080584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003bed0 msgr2=0x55f7400463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003bed0 0x55f7400463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 -- 172.21.15.59:0/2833080584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73ff6ce80 con 0x55f740041a30 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0277400640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02700a4de0 tx=0x7f027009c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d226eb1d2993356f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0260c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027001f040 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02680043c0 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.339+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f02700870c0 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f02700a5310 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 10 client.26729 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f73fcb6210 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02680043c0 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0270085ce0 con 0x55f740041a30 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 10 client.26729 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 20 client.26729 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f740052260 0x55f740054670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 4 client.26729 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:22:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.340+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73ff51cd0 con 0x55f740052260 2024-04-02T17:22:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.341+0000 7f0276a00640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f740052260 0x55f740054670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.342+0000 7f0276a00640 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f740052260 0x55f740054670 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad287cc1c5a5ac9 server_cookie=c9403054054c5334 in_seq=0 out_seq=0 2024-04-02T17:22:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.342+0000 7f0260c00640 10 client.26729 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.342+0000 7f0260c00640 1 -- 172.21.15.59:0/2833080584 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f73ff51cd0 con 0x55f740052260 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.342+0000 7f0260c00640 10 client.26729 handle_command_reply: tid=0 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.342+0000 7f0280be6740 1 client.26729 shutdown 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 msgr2=0x55f74004d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02700a4de0 tx=0x7f027009c750 comp rx=0 tx=0).stop 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 shutdown_connections 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f740052260 0x55f740054670 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f74004d970 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003c8c0 0x55f740046930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 --2- 172.21.15.59:0/2833080584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003bed0 0x55f7400463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 >> 172.21.15.59:0/2833080584 conn(0x55f740037440 msgr2=0x55f740036640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0260c00640 0 client.26729 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 shutdown_connections 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 1 -- 172.21.15.59:0/2833080584 wait complete. 2024-04-02T17:22:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.343+0000 7f0280be6740 20 client.26729 trim_cache size 0 max 16384 2024-04-02T17:22:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 Processor -- start 2024-04-02T17:22:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 -- start start 2024-04-02T17:22:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 0x55f740035db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f7400362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ab270 con 0x55f74003c8c0 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02781acde0 con 0x55f740041a30 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.361+0000 7f0280be6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ab940 con 0x55f74003bed0 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0277400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f7400362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:43752/0 (socket says 172.21.15.59:43752) 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 -- 172.21.15.59:0/2600596107 learned_addr learned my addr 172.21.15.59:0/2600596107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276000640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 0x55f740035db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 -- 172.21.15.59:0/2600596107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 msgr2=0x55f7400362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f7400362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 -- 172.21.15.59:0/2600596107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 msgr2=0x55f740035db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 0x55f740035db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276a00640 1 -- 172.21.15.59:0/2600596107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73fd82620 con 0x55f74003bed0 2024-04-02T17:22:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0276000640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 0x55f740035db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:22:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.362+0000 7f0277400640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f7400362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.363+0000 7f0276a00640 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55f74003d330 tx=0x7f02680096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e61999bda04f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.363+0000 7f0263400640 1 -- 172.21.15.59:0/2600596107 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268031070 con 0x55f74003bed0 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.363+0000 7f0263400640 1 -- 172.21.15.59:0/2600596107 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0268005520 con 0x55f74003bed0 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.363+0000 7f0263400640 1 -- 172.21.15.59:0/2600596107 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268003770 con 0x55f74003bed0 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- 172.21.15.59:0/2600596107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 msgr2=0x55f740045df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55f74003d330 tx=0x7f02680096a0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- 172.21.15.59:0/2600596107 shutdown_connections 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f74003c8c0 0x55f740035db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f740041a30 0x55f7400362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 --2- 172.21.15.59:0/2600596107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f740045df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- 172.21.15.59:0/2600596107 >> 172.21.15.59:0/2600596107 conn(0x55f740037440 msgr2=0x55f7400456e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- 172.21.15.59:0/2600596107 shutdown_connections 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- 172.21.15.59:0/2600596107 wait complete. 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.364+0000 7f0280be6740 1 -- start start 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 Processor -- start 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74004a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f74004a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ac710 con 0x55f740041a30 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02781ac010 con 0x55f74003c8c0 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02781a9250 con 0x55f74003bed0 2024-04-02T17:22:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.365+0000 7f0280be6740 10 client.0 _ll_register_callbacks cb 0x7f027009c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0276000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f74004a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49674/0 (socket says 172.21.15.59:49674) 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 -- 172.21.15.59:0/3903424037 learned_addr learned my addr 172.21.15.59:0/3903424037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0276a00640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74004a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 -- 172.21.15.59:0/3903424037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 msgr2=0x55f74004a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74004a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 -- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 msgr2=0x55f74004a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f74004a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0277400640 1 -- 172.21.15.59:0/3903424037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74004bf30 con 0x55f740041a30 2024-04-02T17:22:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.366+0000 7f0276000640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f74004a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0277400640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0270099120 tx=0x7f027000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1623155c5a4ac20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0260c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027001f040 con 0x55f740041a30 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f027009bda0 con 0x55f740041a30 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0268005520 con 0x55f740041a30 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0270087180 con 0x55f740041a30 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0280be6740 10 client.16638 fetch_fsmap learned FSMap version 34 2024-04-02T17:22:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f7400b87c0 con 0x55f740041a30 2024-04-02T17:22:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.367+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0268005520 con 0x55f740041a30 2024-04-02T17:22:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f0270085ce0 con 0x55f740041a30 2024-04-02T17:22:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 10 client.16638 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:22:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:22:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 20 client.16638 populate_metadata read hostname 'smithi059' 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7400b9020 0x55f7400bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 4 client.16638 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f73ffaf2b0 con 0x55f7400b9020 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.368+0000 7f0276a00640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7400b9020 0x55f7400bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.369+0000 7f0276a00640 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7400b9020 0x55f7400bb440 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d8f927a63b051a server_cookie=7f94b539f9badbe0 in_seq=0 out_seq=0 2024-04-02T17:22:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.369+0000 7f0260c00640 10 client.16638 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0260c00640 1 -- 172.21.15.59:0/3903424037 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f73ffaf2b0 con 0x55f7400b9020 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0260c00640 10 client.16638 handle_command_reply: tid=0 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 client.16638 shutdown 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 msgr2=0x55f74004b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0270099120 tx=0x7f027000c770 comp rx=0 tx=0).stop 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 shutdown_connections 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55f7400b9020 0x55f7400bb440 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f740041a30 0x55f74004b940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f74003c8c0 0x55f74004a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 --2- 172.21.15.59:0/3903424037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f74003bed0 0x55f74004a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 >> 172.21.15.59:0/3903424037 conn(0x55f740037440 msgr2=0x55f74003a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0260c00640 0 client.16638 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 shutdown_connections 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 1 -- 172.21.15.59:0/3903424037 wait complete. 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.403+0000 7f0280be6740 20 client.16638 trim_cache size 0 max 16384 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:22:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.404+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 msgr2=0x7f025c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.404+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 0x7f025c0a6190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0264000c00 tx=0x7f026402bc30 comp rx=0 tx=0).stop 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.404+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 msgr2=0x7f02781a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.404+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f02781a1510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f027000e940 tx=0x7f027000ee00 comp rx=0 tx=0).stop 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 shutdown_connections 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0278105900 0x7f02781a1510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0278104f40 0x7f027819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f025c0a3cd0 0x7f025c0a6190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 --2- 172.21.15.59:0/327545754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0278103d50 0x7f0278199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:22:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 >> 172.21.15.59:0/327545754 conn(0x7f02780ff2c0 msgr2=0x7f02781081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:22:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 shutdown_connections 2024-04-02T17:22:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:22:59.405+0000 7f027de00640 1 -- 172.21.15.59:0/327545754 wait complete. 2024-04-02T17:22:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:22:59 smithi059 ceph-mon[31664]: pgmap v685: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:22:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:22:59 smithi164 ceph-mon[36499]: pgmap v685: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:22:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:22:59 smithi154 ceph-mon[36479]: pgmap v685: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:23:00.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 341392 3.90 MB/sec execute 387 sec latency 36.350 ms 2024-04-02T17:23:01.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 342960 3.91 MB/sec execute 388 sec latency 19.414 ms 2024-04-02T17:23:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:01 smithi059 ceph-mon[31664]: pgmap v686: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:23:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:01 smithi164 ceph-mon[36499]: pgmap v686: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:23:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:01 smithi154 ceph-mon[36479]: pgmap v686: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-02T17:23:02.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 344204 3.93 MB/sec execute 389 sec latency 33.453 ms 2024-04-02T17:23:03.095 INFO:tasks.workunit.client.0.smithi059.stdout: 1 344513 3.92 MB/sec execute 390 sec latency 35.860 ms 2024-04-02T17:23:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:03 smithi059 ceph-mon[31664]: pgmap v687: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-04-02T17:23:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:03 smithi164 ceph-mon[36499]: pgmap v687: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-04-02T17:23:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:03 smithi154 ceph-mon[36479]: pgmap v687: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-04-02T17:23:04.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 345139 3.92 MB/sec execute 391 sec latency 23.765 ms 2024-04-02T17:23:05.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 346829 3.93 MB/sec execute 392 sec latency 39.178 ms 2024-04-02T17:23:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:05 smithi059 ceph-mon[31664]: pgmap v688: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-04-02T17:23:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:05 smithi164 ceph-mon[36499]: pgmap v688: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-04-02T17:23:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:05 smithi154 ceph-mon[36479]: pgmap v688: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-04-02T17:23:06.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 347865 3.94 MB/sec execute 393 sec latency 22.753 ms 2024-04-02T17:23:07.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 348230 3.93 MB/sec execute 394 sec latency 21.142 ms 2024-04-02T17:23:07.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:07 smithi059 ceph-mon[31664]: pgmap v689: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:23:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:07 smithi164 ceph-mon[36499]: pgmap v689: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:23:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:07 smithi154 ceph-mon[36479]: pgmap v689: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.5 MiB/s wr, 193 op/s 2024-04-02T17:23:08.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 349316 3.93 MB/sec execute 395 sec latency 20.489 ms 2024-04-02T17:23:09.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 350948 3.95 MB/sec execute 396 sec latency 17.232 ms 2024-04-02T17:23:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:09 smithi059 ceph-mon[31664]: pgmap v690: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:23:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:09 smithi164 ceph-mon[36499]: pgmap v690: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:23:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:09 smithi154 ceph-mon[36479]: pgmap v690: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-04-02T17:23:10.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 351537 3.95 MB/sec execute 397 sec latency 20.697 ms 2024-04-02T17:23:11.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 352029 3.94 MB/sec execute 398 sec latency 21.668 ms 2024-04-02T17:23:11.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:11 smithi059 ceph-mon[31664]: pgmap v691: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:23:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:11 smithi164 ceph-mon[36499]: pgmap v691: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:23:11.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:11 smithi154 ceph-mon[36479]: pgmap v691: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-02T17:23:12.096 INFO:tasks.workunit.client.0.smithi059.stdout: 1 353567 3.95 MB/sec execute 399 sec latency 41.756 ms 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 -- 172.21.15.59:0/1103769374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 msgr2=0x7f5994073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 --2- 172.21.15.59:0/1103769374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f5994073f90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5984009900 tx=0x7f598402bc50 comp rx=0 tx=0).stop 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 -- 172.21.15.59:0/1103769374 shutdown_connections 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 --2- 172.21.15.59:0/1103769374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f599410e100 0x7f59941104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 --2- 172.21.15.59:0/1103769374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5994074560 0x7f59940749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 --2- 172.21.15.59:0/1103769374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f5994073f90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.571+0000 7f599ae00640 1 -- 172.21.15.59:0/1103769374 >> 172.21.15.59:0/1103769374 conn(0x7f59940fd0c0 msgr2=0x7f59940ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.572+0000 7f599ae00640 1 -- 172.21.15.59:0/1103769374 shutdown_connections 2024-04-02T17:23:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.572+0000 7f599ae00640 1 -- 172.21.15.59:0/1103769374 wait complete. 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 Processor -- start 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 -- start start 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5994074560 0x7f599419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 0x7f59941a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5994113e10 con 0x7f5994074560 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5994113c90 con 0x7f599410e100 2024-04-02T17:23:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f599ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5994113f90 con 0x7f5994075b40 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f5993400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.573+0000 7f5998e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 0x7f59941a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55880/0 (socket says 172.21.15.59:55880) 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 -- 172.21.15.59:0/2011436695 learned_addr learned my addr 172.21.15.59:0/2011436695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5998e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 0x7f59941a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50270/0 (socket says 172.21.15.59:50270) 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993e00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5994074560 0x7f599419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 -- 172.21.15.59:0/2011436695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 msgr2=0x7f59941a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 0x7f59941a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 -- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5994074560 msgr2=0x7f599419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5994074560 0x7f599419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.574+0000 7f5993400640 1 -- 172.21.15.59:0/2011436695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59941a5f90 con 0x7f5994075b40 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f5993400640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f5990c00640 1 -- 172.21.15.59:0/2011436695 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598000cd80 con 0x7f5994075b40 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f5990c00640 1 -- 172.21.15.59:0/2011436695 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980005500 con 0x7f5994075b40 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59941a6280 con 0x7f5994075b40 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f5990c00640 1 -- 172.21.15.59:0/2011436695 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980010690 con 0x7f5994075b40 2024-04-02T17:23:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.575+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59941a67d0 con 0x7f5994075b40 2024-04-02T17:23:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.579+0000 7f599da70740 1 Processor -- start 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.579+0000 7f599da70740 1 -- start start 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 0x564e828662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 0x564e8286dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564e827cbd70 con 0x564e82865ed0 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564e827742a0 con 0x564e828668c0 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f599da70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e827c7d20 con 0x564e8286ba30 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5990c00640 1 -- 172.21.15.59:0/2011436695 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f59800046c0 con 0x7f5994075b40 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50286/0 (socket says 172.21.15.59:50286) 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 -- 172.21.15.59:0/1719803327 learned_addr learned my addr 172.21.15.59:0/1719803327 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993400640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 0x564e8286dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5998e00640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 0x564e828662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 -- 172.21.15.59:0/1719803327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 msgr2=0x564e8286dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 0x564e8286dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 -- 172.21.15.59:0/1719803327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 msgr2=0x564e828662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 0x564e828662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993e00640 1 -- 172.21.15.59:0/1719803327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e82788840 con 0x564e828668c0 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5993400640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 0x564e8286dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.580+0000 7f5998e00640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 0x564e828662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:23:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.581+0000 7f5993e00640 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f598402cfd0 tx=0x7f598402c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a7b05649baa0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.581+0000 7f5989600640 1 -- 172.21.15.59:0/1719803327 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984042070 con 0x564e828668c0 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.581+0000 7f5989600640 1 -- 172.21.15.59:0/1719803327 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598402cbd0 con 0x564e828668c0 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.581+0000 7f5989600640 1 -- 172.21.15.59:0/1719803327 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984005190 con 0x564e828668c0 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.581+0000 7f5990c00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 0x7f59780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f5990c00640 1 -- 172.21.15.59:0/2011436695 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f59800d07e0 con 0x7f5994075b40 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f5998e00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 0x7f59780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 -- 172.21.15.59:0/1719803327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 msgr2=0x564e8286b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f598402cfd0 tx=0x7f598402c5f0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 -- 172.21.15.59:0/1719803327 shutdown_connections 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e8286ba30 0x564e8286dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e828668c0 0x564e8286b3c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 --2- 172.21.15.59:0/1719803327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e82865ed0 0x564e828662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 -- 172.21.15.59:0/1719803327 >> 172.21.15.59:0/1719803327 conn(0x564e82861440 msgr2=0x564e82861840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f599da70740 1 -- 172.21.15.59:0/1719803327 shutdown_connections 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.582+0000 7f5998e00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 0x7f59780a6190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x564e82867120 tx=0x7f598c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.583+0000 7f599da70740 1 -- 172.21.15.59:0/1719803327 wait complete. 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.583+0000 7f599da70740 1 -- start start 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.583+0000 7f599da70740 1 Processor -- start 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.583+0000 7f599da70740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e828668c0 0x564e82870920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e8286ba30 0x564e82877960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564e8286e980 con 0x564e8286ba30 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564e82787660 con 0x564e82865ed0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e8286eb00 con 0x564e828668c0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f599da70740 10 client.0 _ll_register_callbacks cb 0x564e82787050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e828668c0 0x564e82870920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50288/0 (socket says 172.21.15.59:50288) 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 -- 172.21.15.59:0/1595814698 learned_addr learned my addr 172.21.15.59:0/1595814698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5998e00640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e8286ba30 0x564e82877960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 -- 172.21.15.59:0/1595814698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e828668c0 msgr2=0x564e82870920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e828668c0 0x564e82870920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 -- 172.21.15.59:0/1595814698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e8286ba30 msgr2=0x564e82877960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e8286ba30 0x564e82877960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 -- 172.21.15.59:0/1595814698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e82796e80 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.584+0000 7f5993e00640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59840096c0 tx=0x7f5984004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e77909b5d90b689 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f597e000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984042070 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5984004f10 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598402cbd0 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5984005110 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f599da70740 10 client.26747 fetch_fsmap learned FSMap version 34 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564e824e0210 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.585+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598402cbd0 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f5984047460 con 0x564e82865ed0 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 10 client.26747 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 20 client.26747 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8287be50 0x564e8287e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 4 client.26747 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:23:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e8277bcd0 con 0x564e8287be50 2024-04-02T17:23:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.586+0000 7f5993400640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8287be50 0x564e8287e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.587+0000 7f5993400640 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8287be50 0x564e8287e240 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be9e5e12240f119 server_cookie=ad786114be7c4c8d in_seq=0 out_seq=0 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.587+0000 7f597e000640 10 client.26747 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.587+0000 7f597e000640 1 -- 172.21.15.59:0/1595814698 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e8277bcd0 con 0x564e8287be50 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.587+0000 7f597e000640 10 client.26747 handle_command_reply: tid=0 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.587+0000 7f599da70740 1 client.26747 shutdown 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 msgr2=0x564e828703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59840096c0 tx=0x7f5984004250 comp rx=0 tx=0).stop 2024-04-02T17:23:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 shutdown_connections 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8287be50 0x564e8287e240 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e8286ba30 0x564e82877960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e828668c0 0x564e82870920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 --2- 172.21.15.59:0/1595814698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82865ed0 0x564e828703e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f597e000640 0 client.26747 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 >> 172.21.15.59:0/1595814698 conn(0x564e82861440 msgr2=0x564e82860640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 shutdown_connections 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 1 -- 172.21.15.59:0/1595814698 wait complete. 2024-04-02T17:23:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.588+0000 7f599da70740 20 client.26747 trim_cache size 0 max 16384 2024-04-02T17:23:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 Processor -- start 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 -- start start 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e8286fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82870480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59941af580 con 0x564e828668c0 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b10f0 con 0x564e82861440 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f599da70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59941afc50 con 0x564e82865ed0 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.606+0000 7f5993400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55890/0 (socket says 172.21.15.59:55890) 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 -- 172.21.15.59:0/3471308123 learned_addr learned my addr 172.21.15.59:0/3471308123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993e00640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e8286fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5998e00640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82870480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 -- 172.21.15.59:0/3471308123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 msgr2=0x564e8286fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e8286fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 -- 172.21.15.59:0/3471308123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 msgr2=0x564e82870480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82870480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 -- 172.21.15.59:0/3471308123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e825ac620 con 0x564e82865ed0 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993e00640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e8286fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.607+0000 7f5993400640 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5980098ec0 tx=0x7f5980004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e549eb4b1e4964 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.608+0000 7f5988c00640 1 -- 172.21.15.59:0/3471308123 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598009d160 con 0x564e82865ed0 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.608+0000 7f5988c00640 1 -- 172.21.15.59:0/3471308123 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598009ca60 con 0x564e82865ed0 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.608+0000 7f5988c00640 1 -- 172.21.15.59:0/3471308123 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800a5240 con 0x564e82865ed0 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.608+0000 7f599da70740 1 -- 172.21.15.59:0/3471308123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 msgr2=0x564e8286ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.608+0000 7f599da70740 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5980098ec0 tx=0x7f5980004a40 comp rx=0 tx=0).stop 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 -- 172.21.15.59:0/3471308123 shutdown_connections 2024-04-02T17:23:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82870480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e8286ff40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 --2- 172.21.15.59:0/3471308123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e8286fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 -- 172.21.15.59:0/3471308123 >> 172.21.15.59:0/3471308123 conn(0x564e8287e5b0 msgr2=0x564e82864f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 -- 172.21.15.59:0/3471308123 shutdown_connections 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 -- 172.21.15.59:0/3471308123 wait complete. 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 -- start start 2024-04-02T17:23:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.609+0000 7f599da70740 1 Processor -- start 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e82874940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82875950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b0a20 con 0x564e828668c0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b0320 con 0x564e82861440 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f59941ad560 con 0x564e82865ed0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f599da70740 10 client.0 _ll_register_callbacks cb 0x7f5984004d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50304/0 (socket says 172.21.15.59:50304) 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 -- 172.21.15.59:0/1348546454 learned_addr learned my addr 172.21.15.59:0/1348546454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993400640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e82874940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5998e00640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82875950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 -- 172.21.15.59:0/1348546454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 msgr2=0x564e82874940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e82874940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 -- 172.21.15.59:0/1348546454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 msgr2=0x564e82875950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82875950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.610+0000 7f5993e00640 1 -- 172.21.15.59:0/1348546454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e82875f40 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f5993400640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e82874940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f5993e00640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5984001eb0 tx=0x7f598402c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85fc00f42f577782 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f597e000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984042040 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598009ca60 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5984047460 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5984063050 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f599da70740 10 client.26753 fetch_fsmap learned FSMap version 34 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.611+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564e828e2260 con 0x564e82861440 2024-04-02T17:23:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598009ca60 con 0x564e82861440 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f598405b020 con 0x564e82861440 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 10 client.26753 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 20 client.26753 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8286ba30 0x564e828e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 4 client.26753 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:23:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e827d92b0 con 0x564e8286ba30 2024-04-02T17:23:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.612+0000 7f5993400640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8286ba30 0x564e828e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.613+0000 7f5993400640 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8286ba30 0x564e828e4a30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d066eed4ebcc1a server_cookie=87dd78ee1ae915b4 in_seq=0 out_seq=0 2024-04-02T17:23:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.613+0000 7f597e000640 10 client.26753 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f597e000640 1 -- 172.21.15.59:0/1348546454 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e827d92b0 con 0x564e8286ba30 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f597e000640 10 client.26753 handle_command_reply: tid=0 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 client.26753 shutdown 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 msgr2=0x564e82874400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5984001eb0 tx=0x7f598402c5f0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 shutdown_connections 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564e8286ba30 0x564e828e4a30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e828668c0 0x564e82875950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e82865ed0 0x564e82874940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f597e000640 0 client.26753 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 --2- 172.21.15.59:0/1348546454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564e82861440 0x564e82874400 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.649+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 >> 172.21.15.59:0/1348546454 conn(0x564e8287e5b0 msgr2=0x564e8285fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 shutdown_connections 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599da70740 1 -- 172.21.15.59:0/1348546454 wait complete. 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599da70740 20 client.26753 trim_cache size 0 max 16384 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:23:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 msgr2=0x7f59780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 0x7f59780a6190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x564e82867120 tx=0x7f598c009600 comp rx=0 tx=0).stop 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 msgr2=0x7f599419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.650+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 shutdown_connections 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f599410e100 0x7f59941a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5994075b40 0x7f599419e8b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f59780a3cd0 0x7f59780a6190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 --2- 172.21.15.59:0/2011436695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5994074560 0x7f599419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 >> 172.21.15.59:0/2011436695 conn(0x7f59940fd0c0 msgr2=0x7f59941014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 shutdown_connections 2024-04-02T17:23:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:12.651+0000 7f599ae00640 1 -- 172.21.15.59:0/2011436695 wait complete. 2024-04-02T17:23:13.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 354860 3.96 MB/sec execute 400 sec latency 27.319 ms 2024-04-02T17:23:13.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:13 smithi059 ceph-mon[31664]: pgmap v692: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-02T17:23:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:13 smithi164 ceph-mon[36499]: pgmap v692: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-02T17:23:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:13 smithi154 ceph-mon[36479]: pgmap v692: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-02T17:23:14.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 355158 3.96 MB/sec execute 401 sec latency 40.318 ms 2024-04-02T17:23:14.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:14 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:23:14.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:14 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:23:14.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:14 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:23:15.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 355794 3.95 MB/sec execute 402 sec latency 24.294 ms 2024-04-02T17:23:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:15 smithi164 ceph-mon[36499]: pgmap v693: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-02T17:23:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:15 smithi154 ceph-mon[36479]: pgmap v693: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-02T17:23:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:15 smithi059 ceph-mon[31664]: pgmap v693: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-02T17:23:16.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 357574 3.97 MB/sec execute 403 sec latency 17.816 ms 2024-04-02T17:23:17.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 358517 3.97 MB/sec execute 404 sec latency 26.782 ms 2024-04-02T17:23:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:17 smithi164 ceph-mon[36499]: pgmap v694: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:23:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:17 smithi154 ceph-mon[36479]: pgmap v694: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:23:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:17 smithi059 ceph-mon[31664]: pgmap v694: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-02T17:23:18.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 358960 3.97 MB/sec execute 405 sec latency 22.125 ms 2024-04-02T17:23:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:18 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:23:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:18 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:18 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:18 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:23:18.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:18 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:18.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:18 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:18 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:23:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:18 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:18.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:18 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 360651 3.97 MB/sec execute 406 sec latency 19.557 ms 2024-04-02T17:23:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:19 smithi164 ceph-mon[36499]: pgmap v695: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-04-02T17:23:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:19 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:19 smithi154 ceph-mon[36479]: pgmap v695: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-04-02T17:23:19.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:19 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:19 smithi059 ceph-mon[31664]: pgmap v695: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-04-02T17:23:19.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:19.683 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:19 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:20.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 361956 3.99 MB/sec execute 407 sec latency 19.323 ms 2024-04-02T17:23:21.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 362514 3.98 MB/sec execute 408 sec latency 22.839 ms 2024-04-02T17:23:21.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:21 smithi154 ceph-mon[36479]: pgmap v696: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T17:23:21.770 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:21.770 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:21 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:21 smithi059 ceph-mon[31664]: pgmap v696: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T17:23:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:21.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:21 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:21 smithi164 ceph-mon[36499]: pgmap v696: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T17:23:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:21 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:22.097 INFO:tasks.workunit.client.0.smithi059.stdout: 1 364209 3.99 MB/sec execute 409 sec latency 19.212 ms 2024-04-02T17:23:23.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 365535 4.01 MB/sec execute 410 sec latency 25.388 ms 2024-04-02T17:23:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:23 smithi154 ceph-mon[36479]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-02T17:23:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:23:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:23:23.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:23 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:23 smithi059 ceph-mon[31664]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-02T17:23:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:23:23.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:23:23.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:23 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:23 smithi164 ceph-mon[36499]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-02T17:23:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:23:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:23:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:23 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:23:24.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 365786 4.00 MB/sec execute 411 sec latency 32.856 ms 2024-04-02T17:23:25.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 366332 3.99 MB/sec execute 412 sec latency 25.244 ms 2024-04-02T17:23:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:25 smithi154 ceph-mon[36479]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:23:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:25 smithi059 ceph-mon[31664]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 -- 172.21.15.59:0/3239693634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28102b60 msgr2=0x7f8f28102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3239693634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28102b60 0x7f8f28102fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009900 tx=0x7f8f1402bc30 comp rx=0 tx=0).stop 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 -- 172.21.15.59:0/3239693634 shutdown_connections 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3239693634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28103520 0x7f8f28109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3239693634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28102b60 0x7f8f28102fe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3239693634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28101970 0x7f8f28101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 -- 172.21.15.59:0/3239693634 >> 172.21.15.59:0/3239693634 conn(0x7f8f280fcee0 msgr2=0x7f8f280ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 -- 172.21.15.59:0/3239693634 shutdown_connections 2024-04-02T17:23:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.801+0000 7f8f2da00640 1 -- 172.21.15.59:0/3239693634 wait complete. 2024-04-02T17:23:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.802+0000 7f8f2da00640 1 Processor -- start 2024-04-02T17:23:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.802+0000 7f8f2da00640 1 -- start start 2024-04-02T17:23:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 0x7f8f28197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28102b60 0x7f8f281981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d5f0 con 0x7f8f28102b60 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d470 con 0x7f8f28103520 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810d770 con 0x7f8f28101970 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f27400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28102b60 0x7f8f281981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 0x7f8f28197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42098/0 (socket says 172.21.15.59:42098) 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 0x7f8f28197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33844/0 (socket says 172.21.15.59:33844) 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.803+0000 7f8f2d000640 1 -- 172.21.15.59:0/3383821834 learned_addr learned my addr 172.21.15.59:0/3383821834 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 -- 172.21.15.59:0/3383821834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 msgr2=0x7f8f28197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 0x7f8f28197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 -- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28102b60 msgr2=0x7f8f281981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28102b60 0x7f8f281981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 -- 172.21.15.59:0/3383821834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2819f890 con 0x7f8f28103520 2024-04-02T17:23:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2d000640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000eaa0 tx=0x7f8f2000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f24c00640 1 -- 172.21.15.59:0/3383821834 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000cdb0 con 0x7f8f28103520 2024-04-02T17:23:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f24c00640 1 -- 172.21.15.59:0/3383821834 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20005020 con 0x7f8f28103520 2024-04-02T17:23:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2819fb80 con 0x7f8f28103520 2024-04-02T17:23:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f24c00640 1 -- 172.21.15.59:0/3383821834 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200116b0 con 0x7f8f28103520 2024-04-02T17:23:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.804+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281a00d0 con 0x7f8f28103520 2024-04-02T17:23:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.808+0000 7f8f31e35740 1 Processor -- start 2024-04-02T17:23:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.808+0000 7f8f24c00640 1 -- 172.21.15.59:0/3383821834 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f20005560 con 0x7f8f28103520 2024-04-02T17:23:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.808+0000 7f8f31e35740 1 -- start start 2024-04-02T17:23:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.808+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.808+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610099a4c60 0x5610099a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f31e35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561009904d70 con 0x56100999f100 2024-04-02T17:23:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f31e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5610098ad2a0 con 0x56100999faf0 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f31e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561009900d20 con 0x5610099a4c60 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58524/0 (socket says 172.21.15.59:58524) 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 -- 172.21.15.59:0/1299110766 learned_addr learned my addr 172.21.15.59:0/1299110766 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27e00640 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610099a4c60 0x5610099a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 -- 172.21.15.59:0/1299110766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610099a4c60 msgr2=0x5610099a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610099a4c60 0x5610099a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 -- 172.21.15.59:0/1299110766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 msgr2=0x5610099a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.809+0000 7f8f27400640 1 -- 172.21.15.59:0/1299110766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610098c1840 con 0x56100999f100 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f27400640 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f1402c710 tx=0x7f8f14005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be0f1f3aad73160 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f1d600640 1 -- 172.21.15.59:0/1299110766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14004230 con 0x56100999f100 2024-04-02T17:23:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f1d600640 1 -- 172.21.15.59:0/1299110766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f140043d0 con 0x56100999f100 2024-04-02T17:23:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f1d600640 1 -- 172.21.15.59:0/1299110766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14004c80 con 0x56100999f100 2024-04-02T17:23:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f24c00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 0x7f8f0c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f24c00640 1 -- 172.21.15.59:0/3383821834 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8f200d06e0 con 0x7f8f28103520 2024-04-02T17:23:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.810+0000 7f8f27e00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 0x7f8f0c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 -- 172.21.15.59:0/1299110766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 msgr2=0x56100999f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f1402c710 tx=0x7f8f14005b50 comp rx=0 tx=0).stop 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 -- 172.21.15.59:0/1299110766 shutdown_connections 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610099a4c60 0x5610099a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 --2- 172.21.15.59:0/1299110766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x56100999f520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 -- 172.21.15.59:0/1299110766 >> 172.21.15.59:0/1299110766 conn(0x56100999a670 msgr2=0x56100999aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 -- 172.21.15.59:0/1299110766 shutdown_connections 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f27e00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 0x7f8f0c0a6060 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56100999fef0 tx=0x7f8f18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.811+0000 7f8f31e35740 1 -- 172.21.15.59:0/1299110766 wait complete. 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 -- start start 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 Processor -- start 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x5610099b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.812+0000 7f8f31e35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5610099a7bb0 con 0x5610099a4c60 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f31e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5610098c0660 con 0x56100999faf0 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f31e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5610099a7d30 con 0x56100999f100 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f31e35740 10 client.0 _ll_register_callbacks cb 0x5610098c0050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33862/0 (socket says 172.21.15.59:33862) 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 -- 172.21.15.59:0/935198411 learned_addr learned my addr 172.21.15.59:0/935198411 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f2d000640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x5610099b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27400640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 -- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 msgr2=0x5610099a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 -- 172.21.15.59:0/935198411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 msgr2=0x5610099b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x5610099b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.813+0000 7f8f27e00640 1 -- 172.21.15.59:0/935198411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610098cfe80 con 0x56100999f100 2024-04-02T17:23:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f27e00640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8f18013010 tx=0x7f8f18004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f200ad46cce135e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f12000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1802e060 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8f18018030 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f140043d0 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f18009e60 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f31e35740 10 client.26779 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.814+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561009619210 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f140043d0 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8f18032030 con 0x56100999f100 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 10 client.26779 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 10 client.26779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 20 client.26779 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5610099b5540 0x5610099b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 4 client.26779 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:23:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610098b4cd0 con 0x5610099b5540 2024-04-02T17:23:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.815+0000 7f8f27400640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5610099b5540 0x5610099b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.816+0000 7f8f27400640 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5610099b5540 0x5610099b79b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7daaa4b0259bb5ab server_cookie=634957833afd02f4 in_seq=0 out_seq=0 2024-04-02T17:23:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.816+0000 7f8f12000640 10 client.26779 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f12000640 1 -- 172.21.15.59:0/935198411 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610098b4cd0 con 0x5610099b5540 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f12000640 10 client.26779 handle_command_reply: tid=0 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 client.26779 shutdown 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 msgr2=0x5610099a9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8f18013010 tx=0x7f8f18004730 comp rx=0 tx=0).stop 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 shutdown_connections 2024-04-02T17:23:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5610099b5540 0x5610099b79b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x5610099b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999faf0 0x5610099a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 --2- 172.21.15.59:0/935198411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999f100 0x5610099a9610 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f12000640 0 client.26779 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 >> 172.21.15.59:0/935198411 conn(0x56100999a670 msgr2=0x561009999870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 shutdown_connections 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.817+0000 7f8f31e35740 1 -- 172.21.15.59:0/935198411 wait complete. 2024-04-02T17:23:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.818+0000 7f8f31e35740 20 client.26779 trim_cache size 0 max 16384 2024-04-02T17:23:25.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:25 smithi164 ceph-mon[36499]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-02T17:23:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.835+0000 7f8f31e35740 1 Processor -- start 2024-04-02T17:23:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.835+0000 7f8f31e35740 1 -- start start 2024-04-02T17:23:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x56100999dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a8f60 con 0x5610099a4c60 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281aaaa0 con 0x56100999f100 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f31e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a9630 con 0x56100999faf0 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33870/0 (socket says 172.21.15.59:33870) 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42142/0 (socket says 172.21.15.59:42142) 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27400640 1 -- 172.21.15.59:0/3945075462 learned_addr learned my addr 172.21.15.59:0/3945075462 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f2d000640 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x56100999dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 -- 172.21.15.59:0/3945075462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 msgr2=0x5610099b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 -- 172.21.15.59:0/3945075462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 msgr2=0x56100999dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x56100999dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.836+0000 7f8f27e00640 1 -- 172.21.15.59:0/3945075462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610096e5620 con 0x56100999f100 2024-04-02T17:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.837+0000 7f8f2d000640 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x56100999dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.837+0000 7f8f27e00640 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800f7b0 tx=0x7f8f18009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7b9ef01c11c770 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.837+0000 7f8f1cc00640 1 -- 172.21.15.59:0/3945075462 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800f480 con 0x56100999f100 2024-04-02T17:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.837+0000 7f8f1cc00640 1 -- 172.21.15.59:0/3945075462 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1800f9a0 con 0x56100999f100 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.837+0000 7f8f1cc00640 1 -- 172.21.15.59:0/3945075462 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f180333f0 con 0x56100999f100 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 -- 172.21.15.59:0/3945075462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 msgr2=0x5610099a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800f7b0 tx=0x7f8f18009c20 comp rx=0 tx=0).stop 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 -- 172.21.15.59:0/3945075462 shutdown_connections 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610099a4c60 0x56100999dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 --2- 172.21.15.59:0/3945075462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56100999f100 0x5610099a9ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 -- 172.21.15.59:0/3945075462 >> 172.21.15.59:0/3945075462 conn(0x56100999a670 msgr2=0x56100999d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 -- 172.21.15.59:0/3945075462 shutdown_connections 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.838+0000 7f8f31e35740 1 -- 172.21.15.59:0/3945075462 wait complete. 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 -- start start 2024-04-02T17:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 Processor -- start 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x5610099ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 0x5610099afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281aa3d0 con 0x56100999f100 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a9d00 con 0x5610099a4c60 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.839+0000 7f8f31e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281a6f40 con 0x56100999faf0 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f31e35740 10 client.0 _ll_register_callbacks cb 0x7f8f1801e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x5610099ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f2d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 0x5610099afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33882/0 (socket says 172.21.15.59:33882) 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 -- 172.21.15.59:0/2751814757 learned_addr learned my addr 172.21.15.59:0/2751814757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 -- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 msgr2=0x5610099afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 0x5610099afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 -- 172.21.15.59:0/2751814757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 msgr2=0x5610099ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x5610099ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f27400640 1 -- 172.21.15.59:0/2751814757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610099b0150 con 0x56100999faf0 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.840+0000 7f8f2d000640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 0x5610099afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f27400640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5610099a0350 tx=0x7f8f1402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89af9474a5badff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:23:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14054020 con 0x56100999faf0 2024-04-02T17:23:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8f14004130 con 0x56100999faf0 2024-04-02T17:23:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1800f9a0 con 0x56100999faf0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f14004330 con 0x56100999faf0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f31e35740 10 client.26785 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561009a24660 con 0x56100999faf0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1800f9a0 con 0x56100999faf0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.841+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8f14058030 con 0x56100999faf0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 10 client.26785 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 20 client.26785 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561009a24b70 0x561009a26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 4 client.26785 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610099122b0 con 0x561009a24b70 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f27e00640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561009a24b70 0x561009a26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f27e00640 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561009a24b70 0x561009a26f60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7d44bb946328d5 server_cookie=9b6cdcb7226c1e84 in_seq=0 out_seq=0 2024-04-02T17:23:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.842+0000 7f8f12000640 10 client.26785 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f12000640 1 -- 172.21.15.59:0/2751814757 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610099122b0 con 0x561009a24b70 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f12000640 10 client.26785 handle_command_reply: tid=0 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 client.26785 shutdown 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 msgr2=0x5610099aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5610099a0350 tx=0x7f8f1402bc30 comp rx=0 tx=0).stop 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 shutdown_connections 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561009a24b70 0x561009a26f60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610099a4c60 0x5610099afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56100999faf0 0x5610099aeb50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 --2- 172.21.15.59:0/2751814757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100999f100 0x5610099ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f12000640 0 client.26785 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 >> 172.21.15.59:0/2751814757 conn(0x56100999a670 msgr2=0x5610099a98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 shutdown_connections 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 1 -- 172.21.15.59:0/2751814757 wait complete. 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.881+0000 7f8f31e35740 20 client.26785 trim_cache size 0 max 16384 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:23:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 msgr2=0x7f8f0c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 0x7f8f0c0a6060 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56100999fef0 tx=0x7f8f18005f90 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 msgr2=0x7f8f2819f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000eaa0 tx=0x7f8f2000ef60 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 shutdown_connections 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8f0c0a3ba0 0x7f8f0c0a6060 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f28102b60 0x7f8f281981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f28103520 0x7f8f2819f1f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 --2- 172.21.15.59:0/3383821834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f28101970 0x7f8f28197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 >> 172.21.15.59:0/3383821834 conn(0x7f8f280fcee0 msgr2=0x7f8f28105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 shutdown_connections 2024-04-02T17:23:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:25.882+0000 7f8f2da00640 1 -- 172.21.15.59:0/3383821834 wait complete. 2024-04-02T17:23:26.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 367830 4.00 MB/sec execute 413 sec latency 67.816 ms 2024-04-02T17:23:27.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 369042 4.01 MB/sec execute 414 sec latency 34.101 ms 2024-04-02T17:23:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:27 smithi154 ceph-mon[36479]: pgmap v699: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:27 smithi059 ceph-mon[31664]: pgmap v699: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:27 smithi164 ceph-mon[36499]: pgmap v699: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:28.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 369313 4.01 MB/sec execute 415 sec latency 42.287 ms 2024-04-02T17:23:29.098 INFO:tasks.workunit.client.0.smithi059.stdout: 1 369827 4.00 MB/sec execute 416 sec latency 29.092 ms 2024-04-02T17:23:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:29 smithi154 ceph-mon[36479]: pgmap v700: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-02T17:23:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:29 smithi059 ceph-mon[31664]: pgmap v700: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-02T17:23:29.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:29 smithi164 ceph-mon[36499]: pgmap v700: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-02T17:23:30.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 371352 4.01 MB/sec execute 417 sec latency 24.021 ms 2024-04-02T17:23:31.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 372579 4.02 MB/sec execute 418 sec latency 16.422 ms 2024-04-02T17:23:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:31 smithi154 ceph-mon[36479]: pgmap v701: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:23:31.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:31 smithi059 ceph-mon[31664]: pgmap v701: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:23:31.826 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:31 smithi164 ceph-mon[36499]: pgmap v701: 97 pgs: 97 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:23:32.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 372871 4.02 MB/sec execute 419 sec latency 34.591 ms 2024-04-02T17:23:33.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 373610 4.01 MB/sec execute 420 sec latency 29.474 ms 2024-04-02T17:23:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:33 smithi154 ceph-mon[36479]: pgmap v702: 97 pgs: 97 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:23:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:33 smithi059 ceph-mon[31664]: pgmap v702: 97 pgs: 97 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:23:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:33 smithi164 ceph-mon[36499]: pgmap v702: 97 pgs: 97 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-04-02T17:23:34.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 375206 4.02 MB/sec execute 421 sec latency 19.265 ms 2024-04-02T17:23:35.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 376219 4.03 MB/sec execute 422 sec latency 26.546 ms 2024-04-02T17:23:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:35 smithi154 ceph-mon[36479]: pgmap v703: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-02T17:23:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:35 smithi059 ceph-mon[31664]: pgmap v703: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-02T17:23:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:35 smithi164 ceph-mon[36499]: pgmap v703: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-02T17:23:36.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 376446 4.02 MB/sec execute 423 sec latency 27.777 ms 2024-04-02T17:23:37.099 INFO:tasks.workunit.client.0.smithi059.stdout: 1 377154 4.02 MB/sec execute 424 sec latency 23.908 ms 2024-04-02T17:23:37.624 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:37 smithi164 ceph-mon[36499]: pgmap v704: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:23:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:37 smithi154 ceph-mon[36479]: pgmap v704: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:23:37.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:37 smithi059 ceph-mon[31664]: pgmap v704: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:23:38.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 378794 4.03 MB/sec execute 425 sec latency 50.571 ms 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 -- 172.21.15.59:0/2413281032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0104d00 msgr2=0x7f6cd01070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 --2- 172.21.15.59:0/2413281032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0104d00 0x7f6cd01070f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02bc30 comp rx=0 tx=0).stop 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 -- 172.21.15.59:0/2413281032 shutdown_connections 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 --2- 172.21.15.59:0/2413281032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd0107630 0x7f6cd0109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 --2- 172.21.15.59:0/2413281032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0104d00 0x7f6cd01070f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 --2- 172.21.15.59:0/2413281032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd01023d0 0x7f6cd01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.031+0000 7f6ccf200640 1 -- 172.21.15.59:0/2413281032 >> 172.21.15.59:0/2413281032 conn(0x7f6cd00fbde0 msgr2=0x7f6cd00fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.032+0000 7f6ccf200640 1 -- 172.21.15.59:0/2413281032 shutdown_connections 2024-04-02T17:23:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.032+0000 7f6ccf200640 1 -- 172.21.15.59:0/2413281032 wait complete. 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 Processor -- start 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 -- start start 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd01023d0 0x7f6cd0197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 0x7f6cd01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010d610 con 0x7f6cd01023d0 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010d490 con 0x7f6cd0104d00 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010d790 con 0x7f6cd0107630 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6cce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 0x7f6cd01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6cce800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49402/0 (socket says 172.21.15.59:49402) 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6cce800640 1 -- 172.21.15.59:0/126889280 learned_addr learned my addr 172.21.15.59:0/126889280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.033+0000 7f6ccde00640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd01023d0 0x7f6cd0197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 -- 172.21.15.59:0/126889280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 msgr2=0x7f6cd019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 -- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd01023d0 msgr2=0x7f6cd0197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd01023d0 0x7f6cd0197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 -- 172.21.15.59:0/126889280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd019f8c0 con 0x7f6cd0104d00 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6cce800640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.034+0000 7f6ccd400640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 0x7f6cd01981c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009230 tx=0x7f6cbc02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.035+0000 7f6cc6a00640 1 -- 172.21.15.59:0/126889280 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc043020 con 0x7f6cd0104d00 2024-04-02T17:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.035+0000 7f6cc6a00640 1 -- 172.21.15.59:0/126889280 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cbc0054e0 con 0x7f6cd0104d00 2024-04-02T17:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.035+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd019fb50 con 0x7f6cd0104d00 2024-04-02T17:23:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.035+0000 7f6cc6a00640 1 -- 172.21.15.59:0/126889280 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc003b60 con 0x7f6cd0104d00 2024-04-02T17:23:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.035+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01a00c0 con 0x7f6cd0104d00 2024-04-02T17:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.038+0000 7f6cd7dc2740 1 Processor -- start 2024-04-02T17:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.038+0000 7f6cc6a00640 1 -- 172.21.15.59:0/126889280 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6cbc046020 con 0x7f6cd0104d00 2024-04-02T17:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.038+0000 7f6cd7dc2740 1 -- start start 2024-04-02T17:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842b1c60 0x55a6842b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a684211d70 con 0x55a6842acaf0 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a6841ba2a0 con 0x55a6842ac100 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a68420dd20 con 0x55a6842b1c60 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cce800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cce800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57348/0 (socket says 172.21.15.59:57348) 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6cce800640 1 -- 172.21.15.59:0/2480713629 learned_addr learned my addr 172.21.15.59:0/2480713629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6ccde00640 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.039+0000 7f6ccd400640 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842b1c60 0x55a6842b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 -- 172.21.15.59:0/2480713629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842b1c60 msgr2=0x55a6842b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842b1c60 0x55a6842b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 -- 172.21.15.59:0/2480713629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 msgr2=0x55a6842b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 -- 172.21.15.59:0/2480713629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6841ce840 con 0x55a6842ac100 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.040+0000 7f6cce800640 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80118b0 tx=0x7f6cc8011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a7efcec0423333b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6cc6a00640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 0x7f6cb40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6cbb400640 1 -- 172.21.15.59:0/2480713629 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc801e070 con 0x55a6842ac100 2024-04-02T17:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6cbb400640 1 -- 172.21.15.59:0/2480713629 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8011310 con 0x55a6842ac100 2024-04-02T17:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6cbb400640 1 -- 172.21.15.59:0/2480713629 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8019e60 con 0x55a6842ac100 2024-04-02T17:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6cc6a00640 1 -- 172.21.15.59:0/126889280 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6cbc0f23f0 con 0x7f6cd0104d00 2024-04-02T17:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.041+0000 7f6ccde00640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 0x7f6cb40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2480713629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 msgr2=0x55a6842ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80118b0 tx=0x7f6cc8011d70 comp rx=0 tx=0).stop 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2480713629 shutdown_connections 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842b1c60 0x55a6842b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2480713629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ac520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2480713629 >> 172.21.15.59:0/2480713629 conn(0x55a6842a7670 msgr2=0x55a6842a7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6ccde00640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 0x7f6cb40a60e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55a6842acef0 tx=0x7f6cc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2480713629 shutdown_connections 2024-04-02T17:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.042+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2480713629 wait complete. 2024-04-02T17:23:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.043+0000 7f6cd7dc2740 1 -- start start 2024-04-02T17:23:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.043+0000 7f6cd7dc2740 1 Processor -- start 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.043+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.043+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.043+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6842b4c10 con 0x55a6842ac100 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a6841cd660 con 0x55a6842b1c60 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a6842b4d90 con 0x55a6842acaf0 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6cd7dc2740 10 client.0 _ll_register_callbacks cb 0x55a6841cd050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56798/0 (socket says 172.21.15.59:56798) 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccde00640 1 -- 172.21.15.59:0/1390150262 learned_addr learned my addr 172.21.15.59:0/1390150262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6cce800640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 -- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 msgr2=0x55a6842bdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 -- 172.21.15.59:0/1390150262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 msgr2=0x55a6842b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccd400640 1 -- 172.21.15.59:0/1390150262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6841dce80 con 0x55a6842acaf0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.044+0000 7f6ccde00640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6ccd400640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842b6b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0f5fd0 tx=0x7f6cbc0042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1101640ea0c749a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0ba8b0 con 0x55a6842acaf0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6cbc003d00 con 0x55a6842acaf0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc8011310 con 0x55a6842acaf0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6cbc0bdf70 con 0x55a6842acaf0 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cd7dc2740 10 client.26797 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a683f26210 con 0x55a6842acaf0 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8011310 con 0x55a6842acaf0 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f6cbc0f1060 con 0x55a6842acaf0 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cd7dc2740 10 client.26797 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.045+0000 7f6cd7dc2740 10 client.26797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6cd7dc2740 20 client.26797 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a6842c2410 0x55a6842c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6cd7dc2740 4 client.26797 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6841c1cd0 con 0x55a6842c2410 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6ccde00640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a6842c2410 0x55a6842c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6ccde00640 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a6842c2410 0x55a6842c4860 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97bcac2a68a5eaf9 server_cookie=2b26b9514a09c7d in_seq=0 out_seq=0 2024-04-02T17:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.046+0000 7f6cafe00640 10 client.26797 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.047+0000 7f6cafe00640 1 -- 172.21.15.59:0/1390150262 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6841c1cd0 con 0x55a6842c2410 2024-04-02T17:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.047+0000 7f6cafe00640 10 client.26797 handle_command_reply: tid=0 2024-04-02T17:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.047+0000 7f6cd7dc2740 1 client.26797 shutdown 2024-04-02T17:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.047+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 msgr2=0x55a6842b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.047+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842b6b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0f5fd0 tx=0x7f6cbc0042c0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 shutdown_connections 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a6842c2410 0x55a6842c4860 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842b6b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/1390150262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842ac100 0x55a6842b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cafe00640 0 client.26797 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 >> 172.21.15.59:0/1390150262 conn(0x55a6842a7670 msgr2=0x55a6842a6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 shutdown_connections 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/1390150262 wait complete. 2024-04-02T17:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.048+0000 7f6cd7dc2740 20 client.26797 trim_cache size 0 max 16384 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.067+0000 7f6cd7dc2740 1 Processor -- start 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.067+0000 7f6cd7dc2740 1 -- start start 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842b1c60 0x55a6842ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a6e10 con 0x55a6842b1c60 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a8950 con 0x55a6842ac100 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a74e0 con 0x55a6842acaf0 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57378/0 (socket says 172.21.15.59:57378) 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccde00640 1 -- 172.21.15.59:0/2280555986 learned_addr learned my addr 172.21.15.59:0/2280555986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6cce800640 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842b1c60 0x55a6842ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 -- 172.21.15.59:0/2280555986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 msgr2=0x55a6842ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 -- 172.21.15.59:0/2280555986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842b1c60 msgr2=0x55a6842ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842b1c60 0x55a6842ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccd400640 1 -- 172.21.15.59:0/2280555986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a683ff2620 con 0x55a6842acaf0 2024-04-02T17:23:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.068+0000 7f6ccde00640 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.069+0000 7f6ccd400640 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842adf50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0be5a0 tx=0x7f6cbc0f5ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfeb2826cf886253 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.069+0000 7f6cbaa00640 1 -- 172.21.15.59:0/2280555986 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0be150 con 0x55a6842acaf0 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.069+0000 7f6cbaa00640 1 -- 172.21.15.59:0/2280555986 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cbc0ba040 con 0x55a6842acaf0 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.069+0000 7f6cbaa00640 1 -- 172.21.15.59:0/2280555986 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0be3e0 con 0x55a6842acaf0 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2280555986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 msgr2=0x55a6842adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842adf50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0be5a0 tx=0x7f6cbc0f5ea0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2280555986 shutdown_connections 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842b1c60 0x55a6842ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842acaf0 0x55a6842adf50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/2280555986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842ac100 0x55a6842ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2280555986 >> 172.21.15.59:0/2280555986 conn(0x55a6842a7670 msgr2=0x55a6842a7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2280555986 shutdown_connections 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/2280555986 wait complete. 2024-04-02T17:23:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.070+0000 7f6cd7dc2740 1 -- start start 2024-04-02T17:23:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 Processor -- start 2024-04-02T17:23:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a8280 con 0x55a6842acaf0 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a7bb0 con 0x55a6842b1c60 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd01a4df0 con 0x55a6842ac100 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.071+0000 7f6cd7dc2740 10 client.0 _ll_register_callbacks cb 0x55a6841cd050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49416/0 (socket says 172.21.15.59:49416) 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 -- 172.21.15.59:0/111045862 learned_addr learned my addr 172.21.15.59:0/111045862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6cce800640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 -- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 msgr2=0x55a6842bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 -- 172.21.15.59:0/111045862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 msgr2=0x55a6842bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 -- 172.21.15.59:0/111045862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6842bc270 con 0x55a6842ac100 2024-04-02T17:23:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6ccde00640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc002b5b0 tx=0x7f6cc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72668e888c96ca8c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.072+0000 7f6cafe00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0003820 con 0x55a6842ac100 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f6cc00039c0 con 0x55a6842ac100 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cbc0ba040 con 0x55a6842ac100 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6cc0003bc0 con 0x55a6842ac100 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cd7dc2740 10 client.26803 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.073+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a684328a20 con 0x55a6842ac100 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cbc0ba040 con 0x55a6842ac100 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc0004ba0 con 0x55a6842ac100 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 10 client.26803 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 20 client.26803 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a684329400 0x55a68432b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 4 client.26803 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a68421f2b0 con 0x55a684329400 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.074+0000 7f6ccd400640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a684329400 0x55a68432b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.075+0000 7f6cafe00640 10 client.26803 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.075+0000 7f6ccd400640 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a684329400 0x55a68432b820 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef3b76c69515554 server_cookie=d943ae22efafae77 in_seq=0 out_seq=0 2024-04-02T17:23:39.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 379803 4.04 MB/sec execute 426 sec latency 42.378 ms 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.107+0000 7f6cafe00640 1 -- 172.21.15.59:0/111045862 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a68421f2b0 con 0x55a684329400 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.107+0000 7f6cafe00640 10 client.26803 handle_command_reply: tid=0 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.107+0000 7f6cd7dc2740 1 client.26803 shutdown 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 msgr2=0x55a6842ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc002b5b0 tx=0x7f6cc00046c0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 shutdown_connections 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a684329400 0x55a68432b820 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6842acaf0 0x55a6842bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a6842b1c60 0x55a6842bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 --2- 172.21.15.59:0/111045862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a6842ac100 0x55a6842ba730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cafe00640 0 client.26803 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 >> 172.21.15.59:0/111045862 conn(0x55a6842a7670 msgr2=0x55a6842adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 shutdown_connections 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 1 -- 172.21.15.59:0/111045862 wait complete. 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.108+0000 7f6cd7dc2740 20 client.26803 trim_cache size 0 max 16384 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:23:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:23:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:23:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 msgr2=0x7f6cb40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 0x7f6cb40a60e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55a6842acef0 tx=0x7f6cc0005f90 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 msgr2=0x7f6cd01981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 0x7f6cd01981c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009230 tx=0x7f6cbc02c5d0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 shutdown_connections 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cd0107630 0x7f6cd019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cd0104d00 0x7f6cd01981c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f6cb40a3c20 0x7f6cb40a60e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 --2- 172.21.15.59:0/126889280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd01023d0 0x7f6cd0197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 >> 172.21.15.59:0/126889280 conn(0x7f6cd00fbde0 msgr2=0x7f6cd0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 shutdown_connections 2024-04-02T17:23:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:39.109+0000 7f6ccf200640 1 -- 172.21.15.59:0/126889280 wait complete. 2024-04-02T17:23:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:39 smithi154 ceph-mon[36479]: pgmap v705: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-02T17:23:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:39 smithi059 ceph-mon[31664]: pgmap v705: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-02T17:23:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:39 smithi164 ceph-mon[36499]: pgmap v705: 97 pgs: 97 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-02T17:23:40.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 380134 4.03 MB/sec execute 427 sec latency 29.781 ms 2024-04-02T17:23:41.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 381015 4.03 MB/sec execute 428 sec latency 25.535 ms 2024-04-02T17:23:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:41 smithi154 ceph-mon[36479]: pgmap v706: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:23:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:41 smithi059 ceph-mon[31664]: pgmap v706: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:23:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:41 smithi164 ceph-mon[36499]: pgmap v706: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-02T17:23:42.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 382575 4.04 MB/sec execute 429 sec latency 22.217 ms 2024-04-02T17:23:43.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 383415 4.05 MB/sec execute 430 sec latency 35.220 ms 2024-04-02T17:23:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:43 smithi154 ceph-mon[36479]: pgmap v707: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:23:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:43 smithi059 ceph-mon[31664]: pgmap v707: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:23:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:43 smithi164 ceph-mon[36499]: pgmap v707: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-04-02T17:23:44.100 INFO:tasks.workunit.client.0.smithi059.stdout: 1 383701 4.04 MB/sec execute 431 sec latency 26.251 ms 2024-04-02T17:23:45.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 384557 4.04 MB/sec execute 432 sec latency 26.699 ms 2024-04-02T17:23:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:45 smithi154 ceph-mon[36479]: pgmap v708: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-02T17:23:45.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:45 smithi059 ceph-mon[31664]: pgmap v708: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-02T17:23:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:45 smithi164 ceph-mon[36499]: pgmap v708: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-02T17:23:46.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 386131 4.05 MB/sec execute 433 sec latency 21.654 ms 2024-04-02T17:23:47.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 386969 4.05 MB/sec execute 434 sec latency 35.541 ms 2024-04-02T17:23:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:47 smithi154 ceph-mon[36479]: pgmap v709: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:23:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:47 smithi059 ceph-mon[31664]: pgmap v709: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:23:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:47 smithi164 ceph-mon[36499]: pgmap v709: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:23:48.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 387457 4.04 MB/sec execute 435 sec latency 49.973 ms 2024-04-02T17:23:49.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 388860 4.05 MB/sec execute 436 sec latency 28.396 ms 2024-04-02T17:23:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:49 smithi154 ceph-mon[36479]: pgmap v710: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:23:49.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:49 smithi059 ceph-mon[31664]: pgmap v710: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:23:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:49 smithi164 ceph-mon[36499]: pgmap v710: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T17:23:50.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 390164 4.06 MB/sec execute 437 sec latency 18.068 ms 2024-04-02T17:23:51.101 INFO:tasks.workunit.client.0.smithi059.stdout: 1 390601 4.06 MB/sec execute 438 sec latency 33.270 ms 2024-04-02T17:23:51.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:51 smithi059 ceph-mon[31664]: pgmap v711: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:51.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:51 smithi164 ceph-mon[36499]: pgmap v711: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:51.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:51 smithi154 ceph-mon[36479]: pgmap v711: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-02T17:23:52.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 391020 4.05 MB/sec execute 439 sec latency 26.949 ms 2024-04-02T17:23:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 -- 172.21.15.59:0/1224049197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1641036e0 msgr2=0x7fe164109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 --2- 172.21.15.59:0/1224049197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1641036e0 0x7fe164109f70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 -- 172.21.15.59:0/1224049197 shutdown_connections 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 --2- 172.21.15.59:0/1224049197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe164102d20 0x7fe1641031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 --2- 172.21.15.59:0/1224049197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1641036e0 0x7fe164109f70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 --2- 172.21.15.59:0/1224049197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 0x7fe164101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 -- 172.21.15.59:0/1224049197 >> 172.21.15.59:0/1224049197 conn(0x7fe1640fd0c0 msgr2=0x7fe1640ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 -- 172.21.15.59:0/1224049197 shutdown_connections 2024-04-02T17:23:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.280+0000 7fe16b400640 1 -- 172.21.15.59:0/1224049197 wait complete. 2024-04-02T17:23:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.281+0000 7fe16b400640 1 Processor -- start 2024-04-02T17:23:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 -- start start 2024-04-02T17:23:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 0x7fe164199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1641036e0 0x7fe1641a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410d7e0 con 0x7fe1641036e0 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410d660 con 0x7fe164102d20 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe16b400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410d960 con 0x7fe164101b30 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe15be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe15be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58940/0 (socket says 172.21.15.59:58940) 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe15be00640 1 -- 172.21.15.59:0/3323610773 learned_addr learned my addr 172.21.15.59:0/3323610773 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.282+0000 7fe168a00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 0x7fe164199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe169400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1641036e0 0x7fe1641a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 -- 172.21.15.59:0/3323610773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 msgr2=0x7fe164199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 0x7fe164199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 -- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1641036e0 msgr2=0x7fe1641a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1641036e0 0x7fe1641a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 -- 172.21.15.59:0/3323610773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1641a1b10 con 0x7fe164102d20 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe15be00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e9f0 tx=0x7fe14c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe162000640 1 -- 172.21.15.59:0/3323610773 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00cd80 con 0x7fe164102d20 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe162000640 1 -- 172.21.15.59:0/3323610773 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c010910 con 0x7fe164102d20 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.283+0000 7fe162000640 1 -- 172.21.15.59:0/3323610773 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c018990 con 0x7fe164102d20 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.284+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641a1e00 con 0x7fe164102d20 2024-04-02T17:23:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.284+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641a2270 con 0x7fe164102d20 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.288+0000 7fe16e193740 1 Processor -- start 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe162000640 1 -- 172.21.15.59:0/3323610773 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe14c010ac0 con 0x7fe164102d20 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 -- start start 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfdd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfe4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56494bf42d70 con 0x56494bfdd8c0 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56494beeb2a0 con 0x56494bfe2a30 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe16e193740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56494bf3ed20 con 0x56494bfdced0 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe168a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe169400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfdd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe168a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59094/0 (socket says 172.21.15.59:59094) 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.289+0000 7fe168a00640 1 -- 172.21.15.59:0/1563433467 learned_addr learned my addr 172.21.15.59:0/1563433467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe15be00640 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfe4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe169400640 1 -- 172.21.15.59:0/1563433467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 msgr2=0x56494bfe4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe169400640 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfe4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe169400640 1 -- 172.21.15.59:0/1563433467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 msgr2=0x56494bfe23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe169400640 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe169400640 1 -- 172.21.15.59:0/1563433467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494beff840 con 0x56494bfdced0 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.290+0000 7fe168a00640 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe169400640 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfdd2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe15c02c170 tx=0x7fe15c02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88873375a66c580d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe162000640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 0x7fe1480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe159c00640 1 -- 172.21.15.59:0/1563433467 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c044020 con 0x56494bfdced0 2024-04-02T17:23:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe162000640 1 -- 172.21.15.59:0/3323610773 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe14c0d0750 con 0x7fe164102d20 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe159c00640 1 -- 172.21.15.59:0/1563433467 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c02cb80 con 0x56494bfdced0 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe159c00640 1 -- 172.21.15.59:0/1563433467 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0048c0 con 0x56494bfdced0 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.291+0000 7fe168a00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 0x7fe1480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 -- 172.21.15.59:0/1563433467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 msgr2=0x56494bfdd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfdd2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe15c02c170 tx=0x7fe15c02c680 comp rx=0 tx=0).stop 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 -- 172.21.15.59:0/1563433467 shutdown_connections 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdd8c0 0x56494bfe23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfe4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 --2- 172.21.15.59:0/1563433467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfdd2f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 -- 172.21.15.59:0/1563433467 >> 172.21.15.59:0/1563433467 conn(0x56494bfd8440 msgr2=0x56494bfd8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe168a00640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 0x7fe1480a6190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56494bfddcc0 tx=0x7fe154009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 -- 172.21.15.59:0/1563433467 shutdown_connections 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.292+0000 7fe16e193740 1 -- 172.21.15.59:0/1563433467 wait complete. 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.293+0000 7fe16e193740 1 -- start start 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.293+0000 7fe16e193740 1 Processor -- start 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfe73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 0x56494bfe7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56494bfe5980 con 0x56494bfe2a30 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56494befe660 con 0x56494bfdd8c0 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56494bfe5b00 con 0x56494bfdced0 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe16e193740 10 client.0 _ll_register_callbacks cb 0x56494befe050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe15be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 0x56494bfe7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe168a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfe73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe168a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfe73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41444/0 (socket says 172.21.15.59:41444) 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe168a00640 1 -- 172.21.15.59:0/63137987 learned_addr learned my addr 172.21.15.59:0/63137987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.294+0000 7fe169400640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 -- 172.21.15.59:0/63137987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 msgr2=0x56494bfe73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfe73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 -- 172.21.15.59:0/63137987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 msgr2=0x56494bfee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 -- 172.21.15.59:0/63137987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494bf0de80 con 0x56494bfdd8c0 2024-04-02T17:23:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe15be00640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 0x56494bfe7920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0ce240 tx=0x7fe14c098c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701dc8f42672adc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe152000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c09c330 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe14c0d0dd0 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.295+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c02cb80 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe14c09d0d0 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 10 client.26801 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56494bc57210 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c02cb80 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe14c0bbcb0 con 0x56494bfdd8c0 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 10 client.26801 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 10 client.26801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 20 client.26801 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494bff31d0 0x56494bff5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 4 client.26801 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.296+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56494bef2cd0 con 0x56494bff31d0 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.297+0000 7fe168a00640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494bff31d0 0x56494bff5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.298+0000 7fe168a00640 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494bff31d0 0x56494bff5620 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7289987c84cf8738 server_cookie=462406ab35d4bcf9 in_seq=0 out_seq=0 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.298+0000 7fe152000640 10 client.26801 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.298+0000 7fe152000640 1 -- 172.21.15.59:0/63137987 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56494bef2cd0 con 0x56494bff31d0 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.298+0000 7fe152000640 10 client.26801 handle_command_reply: tid=0 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.298+0000 7fe16e193740 1 client.26801 shutdown 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 msgr2=0x56494bfe7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 0x56494bfe7920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0ce240 tx=0x7fe14c098c70 comp rx=0 tx=0).stop 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 shutdown_connections 2024-04-02T17:23:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494bff31d0 0x56494bff5620 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdd8c0 0x56494bfe7920 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 --2- 172.21.15.59:0/63137987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfe73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 >> 172.21.15.59:0/63137987 conn(0x56494bfd8440 msgr2=0x56494bfd7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe152000640 0 client.26801 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 shutdown_connections 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 1 -- 172.21.15.59:0/63137987 wait complete. 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.299+0000 7fe16e193740 20 client.26801 trim_cache size 0 max 16384 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.319+0000 7fe16e193740 1 Processor -- start 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 -- start start 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfd6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ab3d0 con 0x56494bfe2a30 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641acf10 con 0x56494bfdf4d0 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe16e193740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641abaa0 con 0x56494bfdced0 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe169400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe15be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe169400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58966/0 (socket says 172.21.15.59:58966) 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe15be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59110/0 (socket says 172.21.15.59:59110) 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe169400640 1 -- 172.21.15.59:0/1070886052 learned_addr learned my addr 172.21.15.59:0/1070886052 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.320+0000 7fe168a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfd6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 -- 172.21.15.59:0/1070886052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 msgr2=0x56494bfd6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfd6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 -- 172.21.15.59:0/1070886052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 msgr2=0x56494bfee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 -- 172.21.15.59:0/1070886052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494bd23620 con 0x56494bfdf4d0 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe168a00640 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfd6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe169400640 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a30237bb413125c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe159200640 1 -- 172.21.15.59:0/1070886052 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c04d070 con 0x56494bfdf4d0 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe159200640 1 -- 172.21.15.59:0/1070886052 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c005520 con 0x56494bfdf4d0 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.321+0000 7fe159200640 1 -- 172.21.15.59:0/1070886052 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c044040 con 0x56494bfdf4d0 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 -- 172.21.15.59:0/1070886052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 msgr2=0x56494bfd72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c005e50 comp rx=0 tx=0).stop 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 -- 172.21.15.59:0/1070886052 shutdown_connections 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfe2a30 0x56494bfee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfdf4d0 0x56494bfd72f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 --2- 172.21.15.59:0/1070886052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfd6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.322+0000 7fe16e193740 1 -- 172.21.15.59:0/1070886052 >> 172.21.15.59:0/1070886052 conn(0x56494bfd8440 msgr2=0x56494bfdedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.323+0000 7fe16e193740 1 -- 172.21.15.59:0/1070886052 shutdown_connections 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.323+0000 7fe16e193740 1 -- 172.21.15.59:0/1070886052 wait complete. 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.323+0000 7fe16e193740 1 -- start start 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.323+0000 7fe16e193740 1 Processor -- start 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdf4d0 0x56494bfeca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfeda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ac840 con 0x56494bfdf4d0 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ac170 con 0x56494bfe2a30 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641a93b0 con 0x56494bfdced0 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe16e193740 10 client.0 _ll_register_callbacks cb 0x7fe14c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41466/0 (socket says 172.21.15.59:41466) 2024-04-02T17:23:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 -- 172.21.15.59:0/3912704855 learned_addr learned my addr 172.21.15.59:0/3912704855 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe15be00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdf4d0 0x56494bfeca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe169400640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfeda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 -- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 msgr2=0x56494bfeda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfeda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 -- 172.21.15.59:0/3912704855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdf4d0 msgr2=0x56494bfeca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.324+0000 7fe168a00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdf4d0 0x56494bfeca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe168a00640 1 -- 172.21.15.59:0/3912704855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494bfee080 con 0x56494bfdced0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe168a00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56494bff5ea0 tx=0x7fe154004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab488e4a23cbbc46 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe152000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540294f0 con 0x56494bfdced0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c005520 con 0x56494bfdced0 2024-04-02T17:23:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe154029690 con 0x56494bfdced0 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.325+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe154039050 con 0x56494bfdced0 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 10 client.26821 fetch_fsmap learned FSMap version 35 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56494c059db0 con 0x56494bfdced0 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c005520 con 0x56494bfdced0 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe154031530 con 0x56494bfdced0 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 10 client.26821 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 20 client.26821 populate_metadata read hostname 'smithi059' 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494c05a790 0x56494c05cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 4 client.26821 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:23:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56494bf502b0 con 0x56494c05a790 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.326+0000 7fe15be00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494c05a790 0x56494c05cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.327+0000 7fe15be00640 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494c05a790 0x56494c05cbb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd55ef3a547d1f4 server_cookie=83b08fdbeaa93a92 in_seq=0 out_seq=0 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.327+0000 7fe152000640 10 client.26821 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe152000640 1 -- 172.21.15.59:0/3912704855 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56494bf502b0 con 0x56494c05a790 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe152000640 10 client.26821 handle_command_reply: tid=0 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe16e193740 1 client.26821 shutdown 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 msgr2=0x56494bfec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56494bff5ea0 tx=0x7fe154004a40 comp rx=0 tx=0).stop 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 shutdown_connections 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.358+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56494c05a790 0x56494c05cbb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56494bfdf4d0 0x56494bfeca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56494bfe2a30 0x56494bfeda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 --2- 172.21.15.59:0/3912704855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56494bfdced0 0x56494bfec540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe152000640 0 client.26821 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 >> 172.21.15.59:0/3912704855 conn(0x56494bfd8440 msgr2=0x56494bfdb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 shutdown_connections 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 1 -- 172.21.15.59:0/3912704855 wait complete. 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.359+0000 7fe16e193740 20 client.26821 trim_cache size 0 max 16384 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:23:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:23:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:23:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 msgr2=0x7fe1480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 0x7fe1480a6190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56494bfddcc0 tx=0x7fe154009500 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 msgr2=0x7fe16419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e9f0 tx=0x7fe14c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 shutdown_connections 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1480a3cd0 0x7fe1480a6190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1641036e0 0x7fe1641a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe164102d20 0x7fe16419a430 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 --2- 172.21.15.59:0/3323610773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe164101b30 0x7fe164199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 >> 172.21.15.59:0/3323610773 conn(0x7fe1640fd0c0 msgr2=0x7fe164073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 shutdown_connections 2024-04-02T17:23:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:23:52.360+0000 7fe16b400640 1 -- 172.21.15.59:0/3323610773 wait complete. 2024-04-02T17:23:53.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 392094 4.05 MB/sec execute 440 sec latency 220.583 ms 2024-04-02T17:23:53.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:53 smithi059 ceph-mon[31664]: pgmap v712: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-02T17:23:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:53 smithi164 ceph-mon[36499]: pgmap v712: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-02T17:23:53.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:53 smithi154 ceph-mon[36479]: pgmap v712: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-04-02T17:23:54.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 393428 4.06 MB/sec execute 441 sec latency 22.450 ms 2024-04-02T17:23:55.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 394085 4.07 MB/sec execute 442 sec latency 32.523 ms 2024-04-02T17:23:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:55 smithi059 ceph-mon[31664]: pgmap v713: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-02T17:23:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:55 smithi164 ceph-mon[36499]: pgmap v713: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-02T17:23:55.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:55 smithi154 ceph-mon[36479]: pgmap v713: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-04-02T17:23:56.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 394443 4.06 MB/sec execute 443 sec latency 24.470 ms 2024-04-02T17:23:57.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 395407 4.06 MB/sec execute 444 sec latency 31.296 ms 2024-04-02T17:23:57.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:57 smithi059 ceph-mon[31664]: pgmap v714: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:23:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:57 smithi164 ceph-mon[36499]: pgmap v714: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:23:57.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:57 smithi154 ceph-mon[36479]: pgmap v714: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-04-02T17:23:58.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 396941 4.07 MB/sec execute 445 sec latency 19.971 ms 2024-04-02T17:23:59.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 397621 4.07 MB/sec execute 446 sec latency 27.113 ms 2024-04-02T17:23:59.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:23:59 smithi059 ceph-mon[31664]: pgmap v715: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-02T17:23:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:23:59 smithi164 ceph-mon[36499]: pgmap v715: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-02T17:23:59.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:23:59 smithi154 ceph-mon[36479]: pgmap v715: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-02T17:24:00.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 397976 4.06 MB/sec execute 447 sec latency 24.926 ms 2024-04-02T17:24:01.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 398931 4.07 MB/sec execute 448 sec latency 28.223 ms 2024-04-02T17:24:01.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:01 smithi059 ceph-mon[31664]: pgmap v716: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:24:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:01 smithi164 ceph-mon[36499]: pgmap v716: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:24:01.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:01 smithi154 ceph-mon[36479]: pgmap v716: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-02T17:24:02.102 INFO:tasks.workunit.client.0.smithi059.stdout: 1 400476 4.07 MB/sec execute 449 sec latency 20.171 ms 2024-04-02T17:24:03.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401166 4.08 MB/sec execute 450 sec latency 36.296 ms 2024-04-02T17:24:03.701 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:03 smithi154 ceph-mon[36479]: pgmap v717: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:24:03.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:03 smithi059 ceph-mon[31664]: pgmap v717: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:24:03.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:03 smithi164 ceph-mon[36499]: pgmap v717: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-04-02T17:24:04.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401580 4.07 MB/sec execute 451 sec latency 46.312 ms 2024-04-02T17:24:04.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:04 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:04.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:04 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:04.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:04 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:05.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401580 4.06 MB/sec execute 452 sec latency 1046.440 ms 2024-04-02T17:24:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 -- 172.21.15.59:0/4133068177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 msgr2=0x7fb8cc078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/4133068177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc078940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ca0 tx=0x7fb8c402c100 comp rx=0 tx=0).stop 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 -- 172.21.15.59:0/4133068177 shutdown_connections 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/4133068177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc078940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/4133068177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/4133068177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 0x7fb8cc0768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 -- 172.21.15.59:0/4133068177 >> 172.21.15.59:0/4133068177 conn(0x7fb8cc0ff2c0 msgr2=0x7fb8cc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 -- 172.21.15.59:0/4133068177 shutdown_connections 2024-04-02T17:24:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.507+0000 7fb8d2c00640 1 -- 172.21.15.59:0/4133068177 wait complete. 2024-04-02T17:24:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.508+0000 7fb8d2c00640 1 Processor -- start 2024-04-02T17:24:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.508+0000 7fb8d2c00640 1 -- start start 2024-04-02T17:24:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.508+0000 7fb8d2c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 0x7fb8cc19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.508+0000 7fb8d2c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.508+0000 7fb8d2c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc1a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8d2c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113de0 con 0x7fb8cc0780a0 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8d2c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113c60 con 0x7fb8cc0764c0 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8d2c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113f60 con 0x7fb8cc077700 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8d0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc1a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48810/0 (socket says 172.21.15.59:48810) 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 -- 172.21.15.59:0/3106202957 learned_addr learned my addr 172.21.15.59:0/3106202957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 0x7fb8cc19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 -- 172.21.15.59:0/3106202957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 msgr2=0x7fb8cc19e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 0x7fb8cc19e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 -- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 msgr2=0x7fb8cc1a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc1a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8cb400640 1 -- 172.21.15.59:0/3106202957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc1a5f60 con 0x7fb8cc077700 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.509+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8c8c00640 1 -- 172.21.15.59:0/3106202957 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800cd80 con 0x7fb8cc077700 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8c8c00640 1 -- 172.21.15.59:0/3106202957 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b8005500 con 0x7fb8cc077700 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8c8c00640 1 -- 172.21.15.59:0/3106202957 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8010690 con 0x7fb8cc077700 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc1a6250 con 0x7fb8cc077700 2024-04-02T17:24:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.510+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a6750 con 0x7fb8cc077700 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 Processor -- start 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 -- start start 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 0x561effe7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe82a30 0x561effe84ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561effde2d70 con 0x561effe7d8c0 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561effd8b2a0 con 0x561effe7ced0 2024-04-02T17:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d59df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561effdded20 con 0x561effe82a30 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8c8c00640 1 -- 172.21.15.59:0/3106202957 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8b80046c0 con 0x7fb8cc077700 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.514+0000 7fb8d0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 0x561effe7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49070/0 (socket says 172.21.15.59:49070) 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8d0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 0x561effe7d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:47014/0 (socket says 172.21.15.59:47014) 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3434351234 learned_addr learned my addr 172.21.15.59:0/3434351234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe82a30 0x561effe84ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3434351234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe82a30 msgr2=0x561effe84ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe82a30 0x561effe84ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3434351234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 msgr2=0x561effe7d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 0x561effe7d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3434351234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561effd9f840 con 0x561effe7d8c0 2024-04-02T17:24:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.515+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009cf0 tx=0x7fb8bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220a0c887b521f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8c1200640 1 -- 172.21.15.59:0/3434351234 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc01e070 con 0x561effe7d8c0 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8c1200640 1 -- 172.21.15.59:0/3434351234 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8bc019b10 con 0x561effe7d8c0 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8c1200640 1 -- 172.21.15.59:0/3434351234 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00ea20 con 0x561effe7d8c0 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8c8c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 0x7fb8b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8d59df740 1 -- 172.21.15.59:0/3434351234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 msgr2=0x561effe823c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009cf0 tx=0x7fb8bc011810 comp rx=0 tx=0).stop 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.516+0000 7fb8c8c00640 1 -- 172.21.15.59:0/3106202957 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb8b80d0840 con 0x7fb8cc077700 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 0x7fb8b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 -- 172.21.15.59:0/3434351234 shutdown_connections 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe823c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe82a30 0x561effe84ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3434351234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7ced0 0x561effe7d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 -- 172.21.15.59:0/3434351234 >> 172.21.15.59:0/3434351234 conn(0x561effe78440 msgr2=0x561effe78840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 -- 172.21.15.59:0/3434351234 shutdown_connections 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 -- 172.21.15.59:0/3434351234 wait complete. 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d59df740 1 -- start start 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.517+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 0x7fb8b00a6190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005e30 tx=0x7fb8c4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 Processor -- start 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effe8e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561effe859e0 con 0x561effe7d8c0 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561effd9e660 con 0x561effe82a30 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561effe85b60 con 0x561effe7ced0 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8d59df740 10 client.0 _ll_register_callbacks cb 0x561effd9e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49074/0 (socket says 172.21.15.59:49074) 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.518+0000 7fb8cb400640 1 -- 172.21.15.59:0/1821168103 learned_addr learned my addr 172.21.15.59:0/1821168103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effe8e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 -- 172.21.15.59:0/1821168103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 msgr2=0x561effe87410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe87410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 -- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 msgr2=0x561effe8e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effe8e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 -- 172.21.15.59:0/1821168103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561effdade80 con 0x561effe7d8c0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8cb400640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8098ec0 tx=0x7fb8b809c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee5a355d0f79063 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8b5600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:24:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a5440 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.519+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb8b809beb0 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8bc019b10 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8b809ca60 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 10 client.16722 fetch_fsmap learned FSMap version 36 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561effaf7210 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8bc019b10 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb8b809bb20 con 0x561effe7d8c0 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 10 client.16722 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 20 client.16722 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effe932e0 0x561effe95710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 4 client.16722 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:24:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.520+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561effd92cd0 con 0x561effe932e0 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.521+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effe932e0 0x561effe95710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.522+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effe932e0 0x561effe95710 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f846efb6113292e server_cookie=a2596b0b2fa245e6 in_seq=0 out_seq=0 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.522+0000 7fb8b5600640 10 client.16722 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8b5600640 1 -- 172.21.15.59:0/1821168103 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561effd92cd0 con 0x561effe932e0 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8b5600640 10 client.16722 handle_command_reply: tid=0 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 client.16722 shutdown 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 msgr2=0x561effe87950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8098ec0 tx=0x7fb8b809c3c0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 shutdown_connections 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effe932e0 0x561effe95710 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe87950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effe8e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 --2- 172.21.15.59:0/1821168103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8b5600640 0 client.16722 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 >> 172.21.15.59:0/1821168103 conn(0x561effe78440 msgr2=0x561effe77640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 shutdown_connections 2024-04-02T17:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 1 -- 172.21.15.59:0/1821168103 wait complete. 2024-04-02T17:24:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.523+0000 7fb8d59df740 20 client.16722 trim_cache size 0 max 16384 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.539+0000 7fb8d59df740 1 Processor -- start 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 -- start start 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 0x561effe7f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 0x561effe8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1af760 con 0x561effe82a30 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b12d0 con 0x561effe7d8c0 2024-04-02T17:24:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d59df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1afe30 con 0x561effe7ced0 2024-04-02T17:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48832/0 (socket says 172.21.15.59:48832) 2024-04-02T17:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8cbe00640 1 -- 172.21.15.59:0/2038810680 learned_addr learned my addr 172.21.15.59:0/2038810680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 0x561effe8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.540+0000 7fb8cb400640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 0x561effe7f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 -- 172.21.15.59:0/2038810680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 msgr2=0x561effe7f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 0x561effe7f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 -- 172.21.15.59:0/2038810680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 msgr2=0x561effe8e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 0x561effe8e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 -- 172.21.15.59:0/2038810680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561effbc3620 con 0x561effe7ced0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cb400640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 0x561effe7f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 0x561effe8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009c40 tx=0x7fb8bc009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e776d3b0b6b04214 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8b7e00640 1 -- 172.21.15.59:0/2038810680 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc01e040 con 0x561effe7ced0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8b7e00640 1 -- 172.21.15.59:0/2038810680 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8bc02f030 con 0x561effe7ced0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.541+0000 7fb8b7e00640 1 -- 172.21.15.59:0/2038810680 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc003bf0 con 0x561effe7ced0 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 -- 172.21.15.59:0/2038810680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 msgr2=0x561effe7edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009c40 tx=0x7fb8bc009ea0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 -- 172.21.15.59:0/2038810680 shutdown_connections 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe82a30 0x561effe8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe7d8c0 0x561effe7f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 --2- 172.21.15.59:0/2038810680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe7edc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 -- 172.21.15.59:0/2038810680 >> 172.21.15.59:0/2038810680 conn(0x561effe78440 msgr2=0x561effe78820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 -- 172.21.15.59:0/2038810680 shutdown_connections 2024-04-02T17:24:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.542+0000 7fb8d59df740 1 -- 172.21.15.59:0/2038810680 wait complete. 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 -- start start 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 Processor -- start 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effef7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b0c00 con 0x561effe7d8c0 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b0500 con 0x561effe82a30 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1ad740 con 0x561effe7ced0 2024-04-02T17:24:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.543+0000 7fb8d59df740 10 client.0 _ll_register_callbacks cb 0x7fb8b809c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48840/0 (socket says 172.21.15.59:48840) 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3150009614 learned_addr learned my addr 172.21.15.59:0/3150009614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49098/0 (socket says 172.21.15.59:49098) 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8d0c00640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effef7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 msgr2=0x561effef7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effef7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3150009614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 msgr2=0x561effe8bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cbe00640 1 -- 172.21.15.59:0/3150009614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561effef7d90 con 0x561effe7ced0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.544+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8cbe00640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009cf0 tx=0x7fb8bc005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5610c5af7603521c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8b5600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc037020 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fb8bc0039d0 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8bc02f030 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8bc042050 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8d59df740 10 client.26839 fetch_fsmap learned FSMap version 36 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.545+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561effef8fa0 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8bc02f030 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb8bc00e660 con 0x561effe7ced0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 10 client.26839 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 20 client.26839 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effef9770 0x561effefbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 4 client.26839 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561effdf02b0 con 0x561effef9770 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.546+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effef9770 0x561effefbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.547+0000 7fb8cb400640 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effef9770 0x561effefbb90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d7021816399d76 server_cookie=941edb111b076964 in_seq=0 out_seq=0 2024-04-02T17:24:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.547+0000 7fb8b5600640 10 client.26839 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8b5600640 1 -- 172.21.15.59:0/3150009614 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561effdf02b0 con 0x561effef9770 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8b5600640 10 client.26839 handle_command_reply: tid=0 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 client.26839 shutdown 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 msgr2=0x561effe8b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009cf0 tx=0x7fb8bc005da0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 shutdown_connections 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561effef9770 0x561effefbb90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561effe7d8c0 0x561effe8bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561effe82a30 0x561effef7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 --2- 172.21.15.59:0/3150009614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561effe7ced0 0x561effe8b870 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 >> 172.21.15.59:0/3150009614 conn(0x561effe78440 msgr2=0x561effe873d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8b5600640 0 client.26839 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 shutdown_connections 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 1 -- 172.21.15.59:0/3150009614 wait complete. 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.583+0000 7fb8d59df740 20 client.26839 trim_cache size 0 max 16384 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:24:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 msgr2=0x7fb8b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 0x7fb8b00a6190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005e30 tx=0x7fb8c4032590 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 msgr2=0x7fb8cc19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800edf0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 shutdown_connections 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8b00a3cd0 0x7fb8b00a6190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8cc0780a0 0x7fb8cc1a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8cc077700 0x7fb8cc19e880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 --2- 172.21.15.59:0/3106202957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8cc0764c0 0x7fb8cc19e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 >> 172.21.15.59:0/3106202957 conn(0x7fb8cc0ff2c0 msgr2=0x7fb8cc100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 shutdown_connections 2024-04-02T17:24:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:05.584+0000 7fb8d2c00640 1 -- 172.21.15.59:0/3106202957 wait complete. 2024-04-02T17:24:05.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:05 smithi059 ceph-mon[31664]: pgmap v718: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:24:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:05 smithi164 ceph-mon[36499]: pgmap v718: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:24:05.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:05 smithi154 ceph-mon[36479]: pgmap v718: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-04-02T17:24:06.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401580 4.05 MB/sec execute 453 sec latency 2046.553 ms 2024-04-02T17:24:07.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401580 4.04 MB/sec execute 454 sec latency 3046.700 ms 2024-04-02T17:24:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:07 smithi059 ceph-mon[31664]: pgmap v719: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T17:24:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:07 smithi164 ceph-mon[36499]: pgmap v719: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T17:24:07.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:07 smithi154 ceph-mon[36479]: pgmap v719: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T17:24:08.103 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401580 4.03 MB/sec execute 455 sec latency 4046.863 ms 2024-04-02T17:24:09.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401649 4.03 MB/sec execute 456 sec latency 4667.460 ms 2024-04-02T17:24:09.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:09 smithi059 ceph-mon[31664]: pgmap v720: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-04-02T17:24:09.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:09 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:09.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:09 smithi164 ceph-mon[36499]: pgmap v720: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-04-02T17:24:09.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:09 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:09.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:09 smithi154 ceph-mon[36479]: pgmap v720: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-04-02T17:24:09.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:09 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:24:10.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401649 4.02 MB/sec execute 457 sec latency 1265.262 ms 2024-04-02T17:24:11.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401649 4.01 MB/sec execute 458 sec latency 2265.375 ms 2024-04-02T17:24:11.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:11 smithi059 ceph-mon[31664]: pgmap v721: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-02T17:24:11.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:11 smithi164 ceph-mon[36499]: pgmap v721: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-02T17:24:11.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:11 smithi154 ceph-mon[36479]: pgmap v721: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-04-02T17:24:12.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401649 4.00 MB/sec execute 459 sec latency 3265.466 ms 2024-04-02T17:24:13.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401649 3.99 MB/sec execute 460 sec latency 4265.592 ms 2024-04-02T17:24:13.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:13 smithi059 ceph-mon[31664]: pgmap v722: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-04-02T17:24:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:13 smithi164 ceph-mon[36499]: pgmap v722: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-04-02T17:24:13.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:13 smithi154 ceph-mon[36479]: pgmap v722: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-04-02T17:24:14.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 401994 3.99 MB/sec execute 461 sec latency 4878.476 ms 2024-04-02T17:24:15.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 403640 4.00 MB/sec execute 462 sec latency 23.969 ms 2024-04-02T17:24:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:15 smithi059 ceph-mon[31664]: pgmap v723: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-04-02T17:24:15.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:15 smithi164 ceph-mon[36499]: pgmap v723: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-04-02T17:24:15.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:15 smithi154 ceph-mon[36479]: pgmap v723: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-04-02T17:24:16.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 404641 4.00 MB/sec execute 463 sec latency 25.367 ms 2024-04-02T17:24:16.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:16 smithi059 ceph-mon[31664]: pgmap v724: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1009 KiB/s wr, 95 op/s 2024-04-02T17:24:16.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:16 smithi164 ceph-mon[36499]: pgmap v724: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1009 KiB/s wr, 95 op/s 2024-04-02T17:24:16.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:16 smithi154 ceph-mon[36479]: pgmap v724: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1009 KiB/s wr, 95 op/s 2024-04-02T17:24:17.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 404857 4.00 MB/sec execute 464 sec latency 29.163 ms 2024-04-02T17:24:18.104 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405274 3.99 MB/sec execute 465 sec latency 310.952 ms 2024-04-02T17:24:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 -- 172.21.15.59:0/3861634290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37980764c0 msgr2=0x7f37980768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 --2- 172.21.15.59:0/3861634290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37980764c0 0x7f37980768e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c02bc40 comp rx=0 tx=0).stop 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 -- 172.21.15.59:0/3861634290 shutdown_connections 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 --2- 172.21.15.59:0/3861634290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798077700 0x7f3798077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 --2- 172.21.15.59:0/3861634290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 0x7f3798078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 --2- 172.21.15.59:0/3861634290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37980764c0 0x7f37980768e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 -- 172.21.15.59:0/3861634290 >> 172.21.15.59:0/3861634290 conn(0x7f37980ff110 msgr2=0x7f3798101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 -- 172.21.15.59:0/3861634290 shutdown_connections 2024-04-02T17:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.730+0000 7f379ec00640 1 -- 172.21.15.59:0/3861634290 wait complete. 2024-04-02T17:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.731+0000 7f379ec00640 1 Processor -- start 2024-04-02T17:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.731+0000 7f379ec00640 1 -- start start 2024-04-02T17:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37980764c0 0x7f379819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 0x7f37981a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3798113c10 con 0x7f37980764c0 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3798113a90 con 0x7f3798077700 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3798113d90 con 0x7f37980780a0 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38270/0 (socket says 172.21.15.59:38270) 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 -- 172.21.15.59:0/1300518517 learned_addr learned my addr 172.21.15.59:0/1300518517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379e200640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 0x7f37981a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379d800640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37980764c0 0x7f379819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 -- 172.21.15.59:0/1300518517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 msgr2=0x7f37981a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 0x7f37981a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 -- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37980764c0 msgr2=0x7f379819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37980764c0 0x7f379819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.732+0000 7f379ce00640 1 -- 172.21.15.59:0/1300518517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37981a5e40 con 0x7f3798077700 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f379ce00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f378400ea60 tx=0x7f378400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f3796000640 1 -- 172.21.15.59:0/1300518517 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378400cde0 con 0x7f3798077700 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37981a6130 con 0x7f3798077700 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f3796000640 1 -- 172.21.15.59:0/1300518517 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37840035f0 con 0x7f3798077700 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f3796000640 1 -- 172.21.15.59:0/1300518517 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784010670 con 0x7f3798077700 2024-04-02T17:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.733+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a66d0 con 0x7f3798077700 2024-04-02T17:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.736+0000 7f37a2f7f740 1 Processor -- start 2024-04-02T17:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 -- start start 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641abbc2d70 con 0x5641abc5daf0 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641abb6b2a0 con 0x5641abc5d100 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5641abbbed20 con 0x5641abc62c60 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f379d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f379e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f379e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38272/0 (socket says 172.21.15.59:38272) 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f379e200640 1 -- 172.21.15.59:0/2901224165 learned_addr learned my addr 172.21.15.59:0/2901224165 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.737+0000 7f379d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49682/0 (socket says 172.21.15.59:49682) 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 -- 172.21.15.59:0/2901224165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 msgr2=0x5641abc650f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc650f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 -- 172.21.15.59:0/2901224165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 msgr2=0x5641abc625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 -- 172.21.15.59:0/2901224165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641abb7f840 con 0x5641abc5d100 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f3796000640 1 -- 172.21.15.59:0/1300518517 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f37840037a0 con 0x7f3798077700 2024-04-02T17:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f379e200640 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3790009fd0 tx=0x7f37900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970140054a21b887 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f378aa00640 1 -- 172.21.15.59:0/2901224165 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004d20 con 0x5641abc5d100 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.738+0000 7f378aa00640 1 -- 172.21.15.59:0/2901224165 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3790004050 con 0x5641abc5d100 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.739+0000 7f378aa00640 1 -- 172.21.15.59:0/2901224165 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790005260 con 0x5641abc5d100 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.739+0000 7f37a2f7f740 1 -- 172.21.15.59:0/2901224165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 msgr2=0x5641abc5d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.739+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3790009fd0 tx=0x7f37900118d0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.739+0000 7f37a2f7f740 1 -- 172.21.15.59:0/2901224165 shutdown_connections 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.739+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/2901224165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5d100 0x5641abc5d520 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f3796000640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 0x7f37800a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 -- 172.21.15.59:0/2901224165 >> 172.21.15.59:0/2901224165 conn(0x5641abc58670 msgr2=0x5641abc58a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 -- 172.21.15.59:0/2901224165 shutdown_connections 2024-04-02T17:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f3796000640 1 -- 172.21.15.59:0/1300518517 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f37840d0700 con 0x7f3798077700 2024-04-02T17:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 -- 172.21.15.59:0/2901224165 wait complete. 2024-04-02T17:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f379e200640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 0x7f37800a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.740+0000 7f37a2f7f740 1 -- start start 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f379e200640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 0x7f37800a60e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3790009de0 tx=0x7f3790000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 Processor -- start 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abc6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641abc65c10 con 0x5641abc5daf0 2024-04-02T17:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641abb7e660 con 0x5641abc62c60 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5641abc65d90 con 0x5641abc5d100 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.741+0000 7f37a2f7f740 10 client.0 _ll_register_callbacks cb 0x5641abb7e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52768/0 (socket says 172.21.15.59:52768) 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 -- 172.21.15.59:0/3625006677 learned_addr learned my addr 172.21.15.59:0/3625006677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379e200640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abc6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379ce00640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 -- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 msgr2=0x5641abc6ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abc6ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 -- 172.21.15.59:0/3625006677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 msgr2=0x5641abc67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379d800640 1 -- 172.21.15.59:0/3625006677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641abb8de80 con 0x5641abc5d100 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.742+0000 7f379ce00640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f379d800640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f378c02c7c0 tx=0x7f378c004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85253c718ab3e16d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f377f400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c005d50 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f378c0048c0 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3790004050 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f378c004ac0 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f37a2f7f740 10 client.26848 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641ab8d7210 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.743+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3790004050 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f378c050020 con 0x5641abc5d100 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 10 client.26848 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 10 client.26848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 20 client.26848 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abc73450 0x5641abc758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 4 client.26848 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:24:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641abb72cd0 con 0x5641abc73450 2024-04-02T17:24:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.744+0000 7f379ce00640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abc73450 0x5641abc758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.745+0000 7f379ce00640 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abc73450 0x5641abc758a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd7d45de611488d server_cookie=c4ecaacc07d31d74 in_seq=0 out_seq=0 2024-04-02T17:24:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.745+0000 7f377f400640 10 client.26848 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.745+0000 7f377f400640 1 -- 172.21.15.59:0/3625006677 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641abb72cd0 con 0x5641abc73450 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.745+0000 7f377f400640 10 client.26848 handle_command_reply: tid=0 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 client.26848 shutdown 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 msgr2=0x5641abc67640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f378c02c7c0 tx=0x7f378c004250 comp rx=0 tx=0).stop 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 shutdown_connections 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abc73450 0x5641abc758a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abc6ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/3625006677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc67640 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 >> 172.21.15.59:0/3625006677 conn(0x5641abc58670 msgr2=0x5641abc57870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f377f400640 0 client.26848 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 shutdown_connections 2024-04-02T17:24:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 1 -- 172.21.15.59:0/3625006677 wait complete. 2024-04-02T17:24:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.746+0000 7f37a2f7f740 20 client.26848 trim_cache size 0 max 16384 2024-04-02T17:24:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 Processor -- start 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 -- start start 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5d100 0x5641abc56fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc57520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37981af650 con 0x5641abc5d100 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f37981b11f0 con 0x5641abc5daf0 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37981afd20 con 0x5641abc62c60 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f379ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f379ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38294/0 (socket says 172.21.15.59:38294) 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f379ce00640 1 -- 172.21.15.59:0/1861687463 learned_addr learned my addr 172.21.15.59:0/1861687463 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.767+0000 7f379e200640 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc57520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 -- 172.21.15.59:0/1861687463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 msgr2=0x5641abc57520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc57520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 -- 172.21.15.59:0/1861687463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5d100 msgr2=0x5641abc56fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379d800640 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5d100 0x5641abc56fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5d100 0x5641abc56fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 -- 172.21.15.59:0/1861687463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ab9a3620 con 0x5641abc5daf0 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f379ce00640 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3784003ee0 tx=0x7f3784087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb92779cab9290c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f378a000640 1 -- 172.21.15.59:0/1861687463 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378409c920 con 0x5641abc5daf0 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f378a000640 1 -- 172.21.15.59:0/1861687463 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37840a0cd0 con 0x5641abc5daf0 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.768+0000 7f378a000640 1 -- 172.21.15.59:0/1861687463 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840870b0 con 0x5641abc5daf0 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1861687463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 msgr2=0x5641abc6eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3784003ee0 tx=0x7f3784087040 comp rx=0 tx=0).stop 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1861687463 shutdown_connections 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc62c60 0x5641abc57520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc5daf0 0x5641abc6eb80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1861687463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5d100 0x5641abc56fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1861687463 >> 172.21.15.59:0/1861687463 conn(0x5641abc58670 msgr2=0x5641abc58a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1861687463 shutdown_connections 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.769+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1861687463 wait complete. 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 -- start start 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 Processor -- start 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abcd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37981b0b20 con 0x5641abc5daf0 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f37981b03f0 con 0x5641abc62c60 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37981ad630 con 0x5641abc5d100 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.770+0000 7f37a2f7f740 10 client.0 _ll_register_callbacks cb 0x7f378c02ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52792/0 (socket says 172.21.15.59:52792) 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 -- 172.21.15.59:0/1659869725 learned_addr learned my addr 172.21.15.59:0/1659869725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379e200640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abcd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379ce00640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 -- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 msgr2=0x5641abcd7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abcd7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 -- 172.21.15.59:0/1659869725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 msgr2=0x5641abc6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 -- 172.21.15.59:0/1659869725 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641abc6ed50 con 0x5641abc5d100 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379e200640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abcd7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.771+0000 7f379d800640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7823eda9204f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f377f400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c005d50 con 0x5641abc5d100 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f378c0035f0 con 0x5641abc5d100 2024-04-02T17:24:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37840a0cd0 con 0x5641abc5d100 2024-04-02T17:24:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f378c003e80 con 0x5641abc5d100 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f37a2f7f740 10 client.26854 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641abcd8f60 con 0x5641abc5d100 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.772+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37840a0cd0 con 0x5641abc5d100 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f378c050040 con 0x5641abc5d100 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 10 client.26854 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 10 client.26854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 20 client.26854 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abcd9920 0x5641abcdbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 4 client.26854 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641abbd02b0 con 0x5641abcd9920 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f379ce00640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abcd9920 0x5641abcdbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f379ce00640 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abcd9920 0x5641abcdbd40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e59830713ef24a4 server_cookie=8d355cc91fea097d in_seq=0 out_seq=0 2024-04-02T17:24:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.773+0000 7f377f400640 10 client.26854 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.802+0000 7f377f400640 1 -- 172.21.15.59:0/1659869725 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641abbd02b0 con 0x5641abcd9920 2024-04-02T17:24:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f377f400640 10 client.26854 handle_command_reply: tid=0 2024-04-02T17:24:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 client.26854 shutdown 2024-04-02T17:24:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 msgr2=0x5641abc6b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f378c009900 tx=0x7f378c005b50 comp rx=0 tx=0).stop 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 shutdown_connections 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5641abcd9920 0x5641abcdbd40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641abc5daf0 0x5641abc6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641abc62c60 0x5641abcd7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 --2- 172.21.15.59:0/1659869725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641abc5d100 0x5641abc6b790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f377f400640 0 client.26854 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 >> 172.21.15.59:0/1659869725 conn(0x5641abc58670 msgr2=0x5641abc5b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 shutdown_connections 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 1 -- 172.21.15.59:0/1659869725 wait complete. 2024-04-02T17:24:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.803+0000 7f37a2f7f740 20 client.26854 trim_cache size 0 max 16384 2024-04-02T17:24:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:24:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:24:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:24:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:24:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 msgr2=0x7f37800a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 0x7f37800a60e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3790009de0 tx=0x7f3790000f40 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 msgr2=0x7f379819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f378400ea60 tx=0x7f378400ef20 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 shutdown_connections 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37980780a0 0x7f37981a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3798077700 0x7f379819e760 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f37800a3c20 0x7f37800a60e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 --2- 172.21.15.59:0/1300518517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37980764c0 0x7f379819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 >> 172.21.15.59:0/1300518517 conn(0x7f37980ff110 msgr2=0x7f3798100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 shutdown_connections 2024-04-02T17:24:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:18.804+0000 7f379ec00640 1 -- 172.21.15.59:0/1300518517 wait complete. 2024-04-02T17:24:19.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405380 3.98 MB/sec execute 466 sec latency 924.091 ms 2024-04-02T17:24:19.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:18 smithi059 ceph-mon[31664]: pgmap v725: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 906 KiB/s wr, 89 op/s 2024-04-02T17:24:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:18 smithi164 ceph-mon[36499]: pgmap v725: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 906 KiB/s wr, 89 op/s 2024-04-02T17:24:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:18 smithi154 ceph-mon[36479]: pgmap v725: 97 pgs: 97 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 906 KiB/s wr, 89 op/s 2024-04-02T17:24:20.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405380 3.97 MB/sec execute 467 sec latency 1278.581 ms 2024-04-02T17:24:21.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405380 3.96 MB/sec execute 468 sec latency 2278.737 ms 2024-04-02T17:24:21.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:20 smithi059 ceph-mon[31664]: pgmap v726: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:24:21.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:20 smithi164 ceph-mon[36499]: pgmap v726: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:24:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:20 smithi154 ceph-mon[36479]: pgmap v726: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-04-02T17:24:22.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405380 3.96 MB/sec execute 469 sec latency 3278.886 ms 2024-04-02T17:24:23.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405380 3.95 MB/sec execute 470 sec latency 4279.003 ms 2024-04-02T17:24:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:22 smithi059 ceph-mon[31664]: pgmap v727: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 933 KiB/s wr, 91 op/s 2024-04-02T17:24:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:22 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:24:23.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:22 smithi164 ceph-mon[36499]: pgmap v727: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 933 KiB/s wr, 91 op/s 2024-04-02T17:24:23.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:22 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:24:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:22 smithi154 ceph-mon[36479]: pgmap v727: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 933 KiB/s wr, 91 op/s 2024-04-02T17:24:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:22 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:24:24.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 405909 3.95 MB/sec execute 471 sec latency 4891.495 ms 2024-04-02T17:24:24.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:24:24.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:24 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:24:24.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:24:24.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:24 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:24:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:24:24.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:24 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:24:25.105 INFO:tasks.workunit.client.0.smithi059.stdout: 1 407421 3.96 MB/sec execute 472 sec latency 18.465 ms 2024-04-02T17:24:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:25 smithi059 ceph-mon[31664]: pgmap v728: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-02T17:24:25.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:25 smithi164 ceph-mon[36499]: pgmap v728: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-02T17:24:25.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:25 smithi154 ceph-mon[36479]: pgmap v728: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-02T17:24:26.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408248 3.96 MB/sec execute 473 sec latency 27.899 ms 2024-04-02T17:24:27.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408540 3.95 MB/sec execute 474 sec latency 24.349 ms 2024-04-02T17:24:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:27 smithi059 ceph-mon[31664]: pgmap v729: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 880 KiB/s wr, 103 op/s 2024-04-02T17:24:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:27 smithi164 ceph-mon[36499]: pgmap v729: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 880 KiB/s wr, 103 op/s 2024-04-02T17:24:27.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:27 smithi154 ceph-mon[36479]: pgmap v729: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 880 KiB/s wr, 103 op/s 2024-04-02T17:24:28.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408615 3.94 MB/sec execute 475 sec latency 930.032 ms 2024-04-02T17:24:29.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408671 3.94 MB/sec execute 476 sec latency 1541.940 ms 2024-04-02T17:24:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:29 smithi059 ceph-mon[31664]: pgmap v730: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 913 KiB/s wr, 103 op/s 2024-04-02T17:24:29.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:29 smithi164 ceph-mon[36499]: pgmap v730: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 913 KiB/s wr, 103 op/s 2024-04-02T17:24:29.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:29 smithi154 ceph-mon[36479]: pgmap v730: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 913 KiB/s wr, 103 op/s 2024-04-02T17:24:30.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408671 3.93 MB/sec execute 477 sec latency 1318.761 ms 2024-04-02T17:24:31.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408671 3.92 MB/sec execute 478 sec latency 2318.903 ms 2024-04-02T17:24:31.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:31 smithi059 ceph-mon[31664]: pgmap v731: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-04-02T17:24:31.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:31 smithi164 ceph-mon[36499]: pgmap v731: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-04-02T17:24:31.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:31 smithi154 ceph-mon[36479]: pgmap v731: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 -- 172.21.15.59:0/3455083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c105910 msgr2=0x7ff57c10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 --2- 172.21.15.59:0/3455083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c105910 0x7ff57c10c1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff574009ca0 tx=0x7ff57402c100 comp rx=0 tx=0).stop 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 -- 172.21.15.59:0/3455083926 shutdown_connections 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 --2- 172.21.15.59:0/3455083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c105910 0x7ff57c10c1a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 --2- 172.21.15.59:0/3455083926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c104f50 0x7ff57c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 --2- 172.21.15.59:0/3455083926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c103d60 0x7ff57c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 -- 172.21.15.59:0/3455083926 >> 172.21.15.59:0/3455083926 conn(0x7ff57c0ff2f0 msgr2=0x7ff57c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 -- 172.21.15.59:0/3455083926 shutdown_connections 2024-04-02T17:24:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.949+0000 7ff583e00640 1 -- 172.21.15.59:0/3455083926 wait complete. 2024-04-02T17:24:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.950+0000 7ff583e00640 1 Processor -- start 2024-04-02T17:24:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.950+0000 7ff583e00640 1 -- start start 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 0x7ff57c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 0x7ff57c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10f9e0 con 0x7ff57c104f50 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10f860 con 0x7ff57c103d60 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff583e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10fb60 con 0x7ff57c105910 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff580a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff581400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 0x7ff57c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff580a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58978/0 (socket says 172.21.15.59:58978) 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff581e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 0x7ff57c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.951+0000 7ff580a00640 1 -- 172.21.15.59:0/3971329441 learned_addr learned my addr 172.21.15.59:0/3971329441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 -- 172.21.15.59:0/3971329441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 msgr2=0x7ff57c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 0x7ff57c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 -- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 msgr2=0x7ff57c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 -- 172.21.15.59:0/3971329441 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c1a1b80 con 0x7ff57c103d60 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 0x7ff57c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff580a00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff581400640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 0x7ff57c199f60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff57c104db0 tx=0x7ff57000dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff57a000640 1 -- 172.21.15.59:0/3971329441 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570017070 con 0x7ff57c103d60 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff57a000640 1 -- 172.21.15.59:0/3971329441 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff570010460 con 0x7ff57c103d60 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c1a1e70 con 0x7ff57c103d60 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff57a000640 1 -- 172.21.15.59:0/3971329441 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000f620 con 0x7ff57c103d60 2024-04-02T17:24:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.952+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c1a22d0 con 0x7ff57c103d60 2024-04-02T17:24:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.957+0000 7ff586bc8740 1 Processor -- start 2024-04-02T17:24:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.957+0000 7ff586bc8740 1 -- start start 2024-04-02T17:24:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.957+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a44004b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.957+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a44004b8c0 0x55a4400503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 0x55a440052ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff586bc8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a43ffb0d70 con 0x55a44004b8c0 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff586bc8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a43ff592a0 con 0x55a440050a30 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff586bc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a43ffacd20 con 0x55a44004aed0 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff57a000640 1 -- 172.21.15.59:0/3971329441 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff57001a070 con 0x7ff57c103d60 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff581e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a44004b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 0x55a440052ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff581e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a44004b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47230/0 (socket says 172.21.15.59:47230) 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff581e00640 1 -- 172.21.15.59:0/239565434 learned_addr learned my addr 172.21.15.59:0/239565434 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff581400640 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a44004b8c0 0x55a4400503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 -- 172.21.15.59:0/239565434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 msgr2=0x55a44004b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a44004b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 -- 172.21.15.59:0/239565434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a44004b8c0 msgr2=0x55a4400503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a44004b8c0 0x55a4400503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.958+0000 7ff580a00640 1 -- 172.21.15.59:0/239565434 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43ff6d840 con 0x55a440050a30 2024-04-02T17:24:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff580a00640 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 0x55a440052ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff5680114c0 tx=0x7ff568011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d95de86463140a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff56e600640 1 -- 172.21.15.59:0/239565434 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568004030 con 0x55a440050a30 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff56e600640 1 -- 172.21.15.59:0/239565434 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5680041d0 con 0x55a440050a30 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff56e600640 1 -- 172.21.15.59:0/239565434 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56800f960 con 0x55a440050a30 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff57a000640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 0x7ff5640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.959+0000 7ff57a000640 1 -- 172.21.15.59:0/3971329441 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff5700cfcf0 con 0x7ff57c103d60 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- 172.21.15.59:0/239565434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 msgr2=0x55a440052ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 0x55a440052ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff5680114c0 tx=0x7ff568011980 comp rx=0 tx=0).stop 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff581e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 0x7ff5640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- 172.21.15.59:0/239565434 shutdown_connections 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a44004b8c0 0x55a4400503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a440050a30 0x55a440052ec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 --2- 172.21.15.59:0/239565434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a44004b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- 172.21.15.59:0/239565434 >> 172.21.15.59:0/239565434 conn(0x55a440046440 msgr2=0x55a440046840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- 172.21.15.59:0/239565434 shutdown_connections 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- 172.21.15.59:0/239565434 wait complete. 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff586bc8740 1 -- start start 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.960+0000 7ff581e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 0x7ff5640a6190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff574006010 tx=0x7ff574032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 Processor -- start 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a4400553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a440055920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4400539e0 con 0x55a440050a30 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a43ff6c660 con 0x55a44004b8c0 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a440053b60 con 0x55a44004aed0 2024-04-02T17:24:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff586bc8740 10 client.0 _ll_register_callbacks cb 0x55a43ff6c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff581e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff580a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a440055920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff581e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59000/0 (socket says 172.21.15.59:59000) 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.961+0000 7ff581e00640 1 -- 172.21.15.59:0/405353963 learned_addr learned my addr 172.21.15.59:0/405353963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff581400640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a4400553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff580a00640 1 -- 172.21.15.59:0/405353963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 msgr2=0x55a4400553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff580a00640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a4400553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff580a00640 1 -- 172.21.15.59:0/405353963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 msgr2=0x55a44005c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff580a00640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.962+0000 7ff580a00640 1 -- 172.21.15.59:0/405353963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43ff7be80 con 0x55a44004b8c0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff580a00640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a440055920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff56801b040 tx=0x7ff568003fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2347d94d76f82064 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff562a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568027070 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff5680035f0 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5680041d0 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff568033050 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff586bc8740 10 client.26855 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.963+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a43fcc5210 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5680041d0 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff56800f6a0 con 0x55a44004b8c0 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 10 client.26855 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 10 client.26855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 20 client.26855 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a440060fd0 0x55a4400633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 4 client.26855 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.964+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43ff60cd0 con 0x55a440060fd0 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.965+0000 7ff581400640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a440060fd0 0x55a4400633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.965+0000 7ff581400640 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a440060fd0 0x55a4400633c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13ba72341c94458 server_cookie=bacf017fed9dbbcf in_seq=0 out_seq=0 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.965+0000 7ff562a00640 10 client.26855 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff562a00640 1 -- 172.21.15.59:0/405353963 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a43ff60cd0 con 0x55a440060fd0 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff562a00640 10 client.26855 handle_command_reply: tid=0 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 client.26855 shutdown 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 msgr2=0x55a440055920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a440055920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff56801b040 tx=0x7ff568003fe0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 shutdown_connections 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a440060fd0 0x55a4400633c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff562a00640 0 client.26855 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a440055920 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 --2- 172.21.15.59:0/405353963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a4400553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.966+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 >> 172.21.15.59:0/405353963 conn(0x55a440046440 msgr2=0x55a440045640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.967+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 shutdown_connections 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.967+0000 7ff586bc8740 1 -- 172.21.15.59:0/405353963 wait complete. 2024-04-02T17:24:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.967+0000 7ff586bc8740 20 client.26855 trim_cache size 0 max 16384 2024-04-02T17:24:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.983+0000 7ff586bc8740 1 Processor -- start 2024-04-02T17:24:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.983+0000 7ff586bc8740 1 -- start start 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.983+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440044db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.983+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.983+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a4400452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff586bc8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1ab320 con 0x55a440050a30 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff586bc8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1acec0 con 0x55a44004b8c0 2024-04-02T17:24:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff586bc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1ab9f0 con 0x55a44004aed0 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff581e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a4400452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff581400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440044db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:52240/0 (socket says 172.21.15.59:52240) 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 -- 172.21.15.59:0/605626142 learned_addr learned my addr 172.21.15.59:0/605626142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 -- 172.21.15.59:0/605626142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 msgr2=0x55a440044db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440044db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 -- 172.21.15.59:0/605626142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 msgr2=0x55a4400452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a4400452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff580a00640 1 -- 172.21.15.59:0/605626142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43fd91620 con 0x55a44004b8c0 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff581400640 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440044db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.984+0000 7ff581e00640 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a4400452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.985+0000 7ff580a00640 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fac0 tx=0x7ff568012f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bf398799b7b112 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.985+0000 7ff56dc00640 1 -- 172.21.15.59:0/605626142 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568027070 con 0x55a44004b8c0 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.985+0000 7ff56dc00640 1 -- 172.21.15.59:0/605626142 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568030020 con 0x55a44004b8c0 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.985+0000 7ff56dc00640 1 -- 172.21.15.59:0/605626142 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5680035f0 con 0x55a44004b8c0 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- 172.21.15.59:0/605626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 msgr2=0x55a44005c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fac0 tx=0x7ff568012f40 comp rx=0 tx=0).stop 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- 172.21.15.59:0/605626142 shutdown_connections 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a4400452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005c920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 --2- 172.21.15.59:0/605626142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440044db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- 172.21.15.59:0/605626142 >> 172.21.15.59:0/605626142 conn(0x55a440046440 msgr2=0x55a4400553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- 172.21.15.59:0/605626142 shutdown_connections 2024-04-02T17:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- 172.21.15.59:0/605626142 wait complete. 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.986+0000 7ff586bc8740 1 -- start start 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 Processor -- start 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440059c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1ac7f0 con 0x55a440050a30 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1ac0c0 con 0x55a44004b8c0 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c1a9300 con 0x55a44004aed0 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff586bc8740 10 client.0 _ll_register_callbacks cb 0x7ff568005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff581e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff580a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff581e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59006/0 (socket says 172.21.15.59:59006) 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.987+0000 7ff581e00640 1 -- 172.21.15.59:0/1287613429 learned_addr learned my addr 172.21.15.59:0/1287613429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 -- 172.21.15.59:0/1287613429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 msgr2=0x55a440059c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff581400640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440059c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440059c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 -- 172.21.15.59:0/1287613429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 msgr2=0x55a44005b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 -- 172.21.15.59:0/1287613429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44005b6f0 con 0x55a44004b8c0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff581400640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440059c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff581e00640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff580a00640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005a140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fbb0 tx=0x7ff56800fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f22d753d444a90f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff562a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:24:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.988+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568027040 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff568030020 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff568004790 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff56800f840 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff586bc8740 10 client.26861 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a44005ca00 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568030020 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff56802c440 con 0x55a44004b8c0 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.989+0000 7ff586bc8740 10 client.26861 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff586bc8740 10 client.26861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff586bc8740 20 client.26861 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a4400c79d0 0x55a4400c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff586bc8740 4 client.26861 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:24:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a43ffbe2b0 con 0x55a4400c79d0 2024-04-02T17:24:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff581400640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a4400c79d0 0x55a4400c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff581400640 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a4400c79d0 0x55a4400c9df0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebb79b6d17a3513 server_cookie=91004a3a9013c4b0 in_seq=0 out_seq=0 2024-04-02T17:24:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:31.990+0000 7ff562a00640 10 client.26861 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff562a00640 1 -- 172.21.15.59:0/1287613429 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a43ffbe2b0 con 0x55a4400c79d0 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff562a00640 10 client.26861 handle_command_reply: tid=0 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 client.26861 shutdown 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 msgr2=0x55a44005a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005a140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff56800fbb0 tx=0x7ff56800fcb0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 shutdown_connections 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a4400c79d0 0x55a4400c9df0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a440050a30 0x55a44005b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a44004b8c0 0x55a44005a140 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff562a00640 0 client.26861 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 --2- 172.21.15.59:0/1287613429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44004aed0 0x55a440059c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.019+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 >> 172.21.15.59:0/1287613429 conn(0x55a440046440 msgr2=0x55a4400496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.020+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 shutdown_connections 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.020+0000 7ff586bc8740 1 -- 172.21.15.59:0/1287613429 wait complete. 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.020+0000 7ff586bc8740 20 client.26861 trim_cache size 0 max 16384 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 msgr2=0x7ff5640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 0x7ff5640a6190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff574006010 tx=0x7ff574032590 comp rx=0 tx=0).stop 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 msgr2=0x7ff57c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 0x7ff57c199f60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff57c104db0 tx=0x7ff57000dbb0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 shutdown_connections 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff5640a3cd0 0x7ff5640a6190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff57c104f50 0x7ff57c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff57c105910 0x7ff57c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 --2- 172.21.15.59:0/3971329441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff57c103d60 0x7ff57c199f60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 >> 172.21.15.59:0/3971329441 conn(0x7ff57c0ff2f0 msgr2=0x7ff57c108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 shutdown_connections 2024-04-02T17:24:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:32.021+0000 7ff583e00640 1 -- 172.21.15.59:0/3971329441 wait complete. 2024-04-02T17:24:32.106 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408671 3.91 MB/sec execute 479 sec latency 3319.056 ms 2024-04-02T17:24:33.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408671 3.90 MB/sec execute 480 sec latency 4319.186 ms 2024-04-02T17:24:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:33 smithi059 ceph-mon[31664]: pgmap v732: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 939 KiB/s wr, 93 op/s 2024-04-02T17:24:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:33 smithi164 ceph-mon[36499]: pgmap v732: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 939 KiB/s wr, 93 op/s 2024-04-02T17:24:33.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:33 smithi154 ceph-mon[36479]: pgmap v732: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 939 KiB/s wr, 93 op/s 2024-04-02T17:24:34.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408739 3.90 MB/sec execute 481 sec latency 4933.289 ms 2024-04-02T17:24:35.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408739 3.89 MB/sec execute 482 sec latency 1293.029 ms 2024-04-02T17:24:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:35 smithi059 ceph-mon[31664]: pgmap v733: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T17:24:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:35 smithi164 ceph-mon[36499]: pgmap v733: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T17:24:35.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:35 smithi154 ceph-mon[36479]: pgmap v733: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T17:24:36.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408739 3.88 MB/sec execute 483 sec latency 2293.195 ms 2024-04-02T17:24:37.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408739 3.87 MB/sec execute 484 sec latency 3293.296 ms 2024-04-02T17:24:37.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:37 smithi059 ceph-mon[31664]: pgmap v734: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-04-02T17:24:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:37 smithi164 ceph-mon[36499]: pgmap v734: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-04-02T17:24:37.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:37 smithi154 ceph-mon[36479]: pgmap v734: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-04-02T17:24:38.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 408739 3.86 MB/sec execute 485 sec latency 4293.450 ms 2024-04-02T17:24:38.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:38 smithi059 ceph-mon[31664]: pgmap v735: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T17:24:38.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:38 smithi164 ceph-mon[36499]: pgmap v735: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T17:24:38.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:38 smithi154 ceph-mon[36479]: pgmap v735: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T17:24:39.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 409085 3.86 MB/sec execute 486 sec latency 4907.297 ms 2024-04-02T17:24:40.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 410723 3.87 MB/sec execute 487 sec latency 18.641 ms 2024-04-02T17:24:41.107 INFO:tasks.workunit.client.0.smithi059.stdout: 1 411702 3.88 MB/sec execute 488 sec latency 147.698 ms 2024-04-02T17:24:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:40 smithi059 ceph-mon[31664]: pgmap v736: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 874 KiB/s wr, 70 op/s 2024-04-02T17:24:41.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:40 smithi164 ceph-mon[36499]: pgmap v736: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 874 KiB/s wr, 70 op/s 2024-04-02T17:24:41.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:40 smithi154 ceph-mon[36479]: pgmap v736: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 874 KiB/s wr, 70 op/s 2024-04-02T17:24:42.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 411917 3.87 MB/sec execute 489 sec latency 233.966 ms 2024-04-02T17:24:43.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412214 3.86 MB/sec execute 490 sec latency 373.978 ms 2024-04-02T17:24:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:42 smithi059 ceph-mon[31664]: pgmap v737: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 645 KiB/s wr, 65 op/s 2024-04-02T17:24:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:42 smithi164 ceph-mon[36499]: pgmap v737: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 645 KiB/s wr, 65 op/s 2024-04-02T17:24:43.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:42 smithi154 ceph-mon[36479]: pgmap v737: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 645 KiB/s wr, 65 op/s 2024-04-02T17:24:44.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412282 3.86 MB/sec execute 491 sec latency 987.573 ms 2024-04-02T17:24:45.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412282 3.85 MB/sec execute 492 sec latency 1277.999 ms 2024-04-02T17:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 -- 172.21.15.59:0/196898843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 msgr2=0x7f6960105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 --2- 172.21.15.59:0/196898843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f6960105210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 -- 172.21.15.59:0/196898843 shutdown_connections 2024-04-02T17:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 --2- 172.21.15.59:0/196898843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 0x7f696010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 --2- 172.21.15.59:0/196898843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f6960105210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 --2- 172.21.15.59:0/196898843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 -- 172.21.15.59:0/196898843 >> 172.21.15.59:0/196898843 conn(0x7f69600ff110 msgr2=0x7f6960101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 -- 172.21.15.59:0/196898843 shutdown_connections 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.166+0000 7f6966400640 1 -- 172.21.15.59:0/196898843 wait complete. 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.167+0000 7f6966400640 1 Processor -- start 2024-04-02T17:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.167+0000 7f6966400640 1 -- start start 2024-04-02T17:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.167+0000 7f6966400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.167+0000 7f6966400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f696019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.167+0000 7f6966400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 0x7f69601a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6966400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f696010f820 con 0x7f6960105750 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6966400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f696010f6a0 con 0x7f6960103ba0 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6966400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f696010f9a0 con 0x7f6960104d90 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58840/0 (socket says 172.21.15.59:58840) 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 -- 172.21.15.59:0/1942176407 learned_addr learned my addr 172.21.15.59:0/1942176407 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f695fe00640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f696019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965a00640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 0x7f69601a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 -- 172.21.15.59:0/1942176407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 msgr2=0x7f696019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f696019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 -- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 msgr2=0x7f69601a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 0x7f69601a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.168+0000 7f6965000640 1 -- 172.21.15.59:0/1942176407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69601a1ad0 con 0x7f6960103ba0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f6965000640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9c0 tx=0x7f695400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f695d600640 1 -- 172.21.15.59:0/1942176407 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695400cd80 con 0x7f6960103ba0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f695d600640 1 -- 172.21.15.59:0/1942176407 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69540035f0 con 0x7f6960103ba0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69601a1dc0 con 0x7f6960103ba0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f695d600640 1 -- 172.21.15.59:0/1942176407 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954010690 con 0x7f6960103ba0 2024-04-02T17:24:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.169+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a21d0 con 0x7f6960103ba0 2024-04-02T17:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.172+0000 7f695d600640 1 -- 172.21.15.59:0/1942176407 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f69540037a0 con 0x7f6960103ba0 2024-04-02T17:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.172+0000 7f696a7f7740 1 Processor -- start 2024-04-02T17:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.172+0000 7f696a7f7740 1 -- start start 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eeda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eedd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da1ee3ad70 con 0x55da1eed5100 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da1ede32a0 con 0x55da1eed5af0 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f696a7f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da1ee36d20 con 0x55da1eedac60 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42874/0 (socket says 172.21.15.59:42874) 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f6965a00640 1 -- 172.21.15.59:0/87032772 learned_addr learned my addr 172.21.15.59:0/87032772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f6965000640 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eeda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.173+0000 7f695fe00640 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eedd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 -- 172.21.15.59:0/87032772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 msgr2=0x55da1eedd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eedd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 -- 172.21.15.59:0/87032772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 msgr2=0x55da1eeda5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eeda5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 -- 172.21.15.59:0/87032772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1edf7840 con 0x55da1eed5100 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6965a00640 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6958009fd0 tx=0x7f69580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36beef9d07cf55d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6952000640 1 -- 172.21.15.59:0/87032772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004d20 con 0x55da1eed5100 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6952000640 1 -- 172.21.15.59:0/87032772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958004050 con 0x55da1eed5100 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f6952000640 1 -- 172.21.15.59:0/87032772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958005260 con 0x55da1eed5100 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.174+0000 7f695d600640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 0x7f69480a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f695d600640 1 -- 172.21.15.59:0/1942176407 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f69540d0840 con 0x7f6960103ba0 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f6965000640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 0x7f69480a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 -- 172.21.15.59:0/87032772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 msgr2=0x55da1eed5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6958009fd0 tx=0x7f69580118d0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 -- 172.21.15.59:0/87032772 shutdown_connections 2024-04-02T17:24:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eedd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eeda5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 --2- 172.21.15.59:0/87032772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eed5520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 -- 172.21.15.59:0/87032772 >> 172.21.15.59:0/87032772 conn(0x55da1eed0670 msgr2=0x55da1eed0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 -- 172.21.15.59:0/87032772 shutdown_connections 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f696a7f7740 1 -- 172.21.15.59:0/87032772 wait complete. 2024-04-02T17:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.175+0000 7f6965000640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 0x7f69480a6060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f69540d0ae0 tx=0x7f69540d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.176+0000 7f696a7f7740 1 -- start start 2024-04-02T17:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.176+0000 7f696a7f7740 1 Processor -- start 2024-04-02T17:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eee6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da1eeddc10 con 0x55da1eed5100 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da1edf6660 con 0x55da1eed5af0 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da1eeddd90 con 0x55da1eedac60 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f696a7f7740 10 client.0 _ll_register_callbacks cb 0x55da1edf6050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42880/0 (socket says 172.21.15.59:42880) 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f6965a00640 1 -- 172.21.15.59:0/2089461962 learned_addr learned my addr 172.21.15.59:0/2089461962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f6965a00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eee6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 -- 172.21.15.59:0/2089461962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 msgr2=0x55da1eee6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eee6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 -- 172.21.15.59:0/2089461962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 msgr2=0x55da1eedf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f695fe00640 1 -- 172.21.15.59:0/2089461962 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1ee05e80 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.177+0000 7f6965a00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f695fe00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f694c009230 tx=0x7f694c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dde857b00a2f408 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f6946a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c005d50 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f694c005230 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6958004050 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.178+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f694c005430 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 10 client.26873 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da1eb4f210 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958004050 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f694c003990 con 0x55da1eed5af0 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 10 client.26873 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 20 client.26873 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1eeeb390 0x55da1eeed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 4 client.26873 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.179+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1edeacd0 con 0x55da1eeeb390 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.180+0000 7f6965a00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1eeeb390 0x55da1eeed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.180+0000 7f6965a00640 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1eeeb390 0x55da1eeed7e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4328137deb902642 server_cookie=5b02e072c7b11e32 in_seq=0 out_seq=0 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.180+0000 7f6946a00640 10 client.26873 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.181+0000 7f6946a00640 1 -- 172.21.15.59:0/2089461962 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da1edeacd0 con 0x55da1eeeb390 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.181+0000 7f6946a00640 10 client.26873 handle_command_reply: tid=0 2024-04-02T17:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.181+0000 7f696a7f7740 1 client.26873 shutdown 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 msgr2=0x55da1eedfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f694c009230 tx=0x7f694c004270 comp rx=0 tx=0).stop 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 shutdown_connections 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1eeeb390 0x55da1eeed7e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eedac60 0x55da1eee6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 --2- 172.21.15.59:0/2089461962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f6946a00640 0 client.26873 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 >> 172.21.15.59:0/2089461962 conn(0x55da1eed0670 msgr2=0x55da1eecf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 shutdown_connections 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 1 -- 172.21.15.59:0/2089461962 wait complete. 2024-04-02T17:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.182+0000 7f696a7f7740 20 client.26873 trim_cache size 0 max 16384 2024-04-02T17:24:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 Processor -- start 2024-04-02T17:24:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 -- start start 2024-04-02T17:24:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ab240 con 0x55da1eed5100 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69601acde0 con 0x55da1eedac60 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f696a7f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ab910 con 0x55da1eed5af0 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42892/0 (socket says 172.21.15.59:42892) 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.199+0000 7f6965a00640 1 -- 172.21.15.59:0/1550566991 learned_addr learned my addr 172.21.15.59:0/1550566991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f695fe00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 -- 172.21.15.59:0/1550566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 msgr2=0x55da1eedfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 -- 172.21.15.59:0/1550566991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 msgr2=0x55da1eee6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 -- 172.21.15.59:0/1550566991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1ec1b620 con 0x55da1eed5100 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f695fe00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6965a00640 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69580096c0 tx=0x7f6958000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e176c0a6f0f5d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6951600640 1 -- 172.21.15.59:0/1550566991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004110 con 0x55da1eed5100 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6951600640 1 -- 172.21.15.59:0/1550566991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958005530 con 0x55da1eed5100 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.200+0000 7f6951600640 1 -- 172.21.15.59:0/1550566991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800fa10 con 0x55da1eed5100 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 -- 172.21.15.59:0/1550566991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 msgr2=0x55da1eedf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69580096c0 tx=0x7f6958000f40 comp rx=0 tx=0).stop 2024-04-02T17:24:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 -- 172.21.15.59:0/1550566991 shutdown_connections 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eedfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1550566991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eedf5e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 -- 172.21.15.59:0/1550566991 >> 172.21.15.59:0/1550566991 conn(0x55da1eed0670 msgr2=0x55da1eed0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 -- 172.21.15.59:0/1550566991 shutdown_connections 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.201+0000 7f696a7f7740 1 -- 172.21.15.59:0/1550566991 wait complete. 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.202+0000 7f696a7f7740 1 -- start start 2024-04-02T17:24:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.202+0000 7f696a7f7740 1 Processor -- start 2024-04-02T17:24:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eee3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ac710 con 0x55da1eed5100 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69601abfe0 con 0x55da1eedac60 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69601a9220 con 0x55da1eed5af0 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f696a7f7740 10 client.0 _ll_register_callbacks cb 0x7f694c02c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f6965a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eee3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48410/0 (socket says 172.21.15.59:48410) 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 -- 172.21.15.59:0/1048399797 learned_addr learned my addr 172.21.15.59:0/1048399797 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f6965a00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 -- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 msgr2=0x55da1eee52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 -- 172.21.15.59:0/1048399797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 msgr2=0x55da1eee3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eee3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f695fe00640 1 -- 172.21.15.59:0/1048399797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1eee58b0 con 0x55da1eed5af0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.203+0000 7f6965a00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eee3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f695fe00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f694c02c7b0 tx=0x7f694c0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1d4b8f1e23ab11 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f6946a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c005d50 con 0x55da1eed5af0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6958005530 con 0x55da1eed5af0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f694c0037f0 con 0x55da1eed5af0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f694c009520 con 0x55da1eed5af0 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f696a7f7740 10 client.26884 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.204+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da1eee6bd0 con 0x55da1eed5af0 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958005530 con 0x55da1eed5af0 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f694c004e40 con 0x55da1eed5af0 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 10 client.26884 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 10 client.26884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 20 client.26884 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1ef51e90 0x55da1ef542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 4 client.26884 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.205+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da1ee482b0 con 0x55da1ef51e90 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.206+0000 7f6965a00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1ef51e90 0x55da1ef542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.206+0000 7f6965a00640 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1ef51e90 0x55da1ef542b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59957a1ae2b27f4b server_cookie=6ca6a0852c502d23 in_seq=0 out_seq=0 2024-04-02T17:24:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.206+0000 7f6946a00640 10 client.26884 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f6946a00640 1 -- 172.21.15.59:0/1048399797 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6958034040 con 0x55da1ef51e90 2024-04-02T17:24:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f6946a00640 10 client.26884 handle_command_reply: tid=0 2024-04-02T17:24:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 client.26884 shutdown 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 msgr2=0x55da1eee4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f694c02c7b0 tx=0x7f694c0098c0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 shutdown_connections 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1ef51e90 0x55da1ef542b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1eedac60 0x55da1eee52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1eed5af0 0x55da1eee4350 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 --2- 172.21.15.59:0/1048399797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1eed5100 0x55da1eee3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f6946a00640 0 client.26884 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 >> 172.21.15.59:0/1048399797 conn(0x55da1eed0670 msgr2=0x55da1eed6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 shutdown_connections 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 1 -- 172.21.15.59:0/1048399797 wait complete. 2024-04-02T17:24:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.235+0000 7f696a7f7740 20 client.26884 trim_cache size 0 max 16384 2024-04-02T17:24:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:24:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:24:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:24:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 msgr2=0x7f69480a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 0x7f69480a6060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f69540d0ae0 tx=0x7f69540d6530 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 msgr2=0x7f6960199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9c0 tx=0x7f695400ee80 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 shutdown_connections 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f69480a3ba0 0x7f69480a6060 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6960105750 0x7f69601a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6960104d90 0x7f696019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 --2- 172.21.15.59:0/1942176407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6960103ba0 0x7f6960199eb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.236+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 >> 172.21.15.59:0/1942176407 conn(0x7f69600ff110 msgr2=0x7f6960108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.237+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 shutdown_connections 2024-04-02T17:24:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:45.237+0000 7f6966400640 1 -- 172.21.15.59:0/1942176407 wait complete. 2024-04-02T17:24:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:44 smithi059 ceph-mon[31664]: pgmap v738: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 663 KiB/s wr, 78 op/s 2024-04-02T17:24:45.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:44 smithi164 ceph-mon[36499]: pgmap v738: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 663 KiB/s wr, 78 op/s 2024-04-02T17:24:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:44 smithi154 ceph-mon[36479]: pgmap v738: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 663 KiB/s wr, 78 op/s 2024-04-02T17:24:46.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412282 3.84 MB/sec execute 493 sec latency 2278.101 ms 2024-04-02T17:24:47.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412282 3.83 MB/sec execute 494 sec latency 3278.267 ms 2024-04-02T17:24:47.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:46 smithi059 ceph-mon[31664]: pgmap v739: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 481 KiB/s wr, 70 op/s 2024-04-02T17:24:47.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:46 smithi164 ceph-mon[36499]: pgmap v739: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 481 KiB/s wr, 70 op/s 2024-04-02T17:24:47.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:46 smithi154 ceph-mon[36479]: pgmap v739: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 481 KiB/s wr, 70 op/s 2024-04-02T17:24:48.108 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412282 3.82 MB/sec execute 495 sec latency 4278.409 ms 2024-04-02T17:24:49.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 412646 3.82 MB/sec execute 496 sec latency 4887.561 ms 2024-04-02T17:24:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:48 smithi059 ceph-mon[31664]: pgmap v740: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 290 KiB/s wr, 65 op/s 2024-04-02T17:24:49.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:48 smithi164 ceph-mon[36499]: pgmap v740: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 290 KiB/s wr, 65 op/s 2024-04-02T17:24:49.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:48 smithi154 ceph-mon[36479]: pgmap v740: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 290 KiB/s wr, 65 op/s 2024-04-02T17:24:50.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 414261 3.83 MB/sec execute 497 sec latency 15.310 ms 2024-04-02T17:24:51.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 415278 3.84 MB/sec execute 498 sec latency 32.572 ms 2024-04-02T17:24:51.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:50 smithi059 ceph-mon[31664]: pgmap v741: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 341 KiB/s wr, 69 op/s 2024-04-02T17:24:51.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:50 smithi164 ceph-mon[36499]: pgmap v741: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 341 KiB/s wr, 69 op/s 2024-04-02T17:24:51.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:50 smithi154 ceph-mon[36479]: pgmap v741: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 341 KiB/s wr, 69 op/s 2024-04-02T17:24:52.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 415503 3.83 MB/sec execute 499 sec latency 24.378 ms 2024-04-02T17:24:53.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 415707 3.82 MB/sec execute 500 sec latency 642.505 ms 2024-04-02T17:24:53.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:52 smithi059 ceph-mon[31664]: pgmap v742: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-04-02T17:24:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:52 smithi164 ceph-mon[36499]: pgmap v742: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-04-02T17:24:53.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:52 smithi154 ceph-mon[36479]: pgmap v742: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 413 KiB/s wr, 74 op/s 2024-04-02T17:24:54.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 415984 3.82 MB/sec execute 501 sec latency 1254.789 ms 2024-04-02T17:24:55.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 417537 3.82 MB/sec execute 502 sec latency 30.522 ms 2024-04-02T17:24:55.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:54 smithi059 ceph-mon[31664]: pgmap v743: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 529 KiB/s wr, 99 op/s 2024-04-02T17:24:55.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:54 smithi164 ceph-mon[36499]: pgmap v743: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 529 KiB/s wr, 99 op/s 2024-04-02T17:24:55.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:54 smithi154 ceph-mon[36479]: pgmap v743: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 529 KiB/s wr, 99 op/s 2024-04-02T17:24:56.109 INFO:tasks.workunit.client.0.smithi059.stdout: 1 418743 3.84 MB/sec execute 503 sec latency 17.356 ms 2024-04-02T17:24:57.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 418986 3.83 MB/sec execute 504 sec latency 27.555 ms 2024-04-02T17:24:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:57 smithi059 ceph-mon[31664]: pgmap v744: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 492 KiB/s wr, 81 op/s 2024-04-02T17:24:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:57 smithi164 ceph-mon[36499]: pgmap v744: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 492 KiB/s wr, 81 op/s 2024-04-02T17:24:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:57 smithi154 ceph-mon[36479]: pgmap v744: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 492 KiB/s wr, 81 op/s 2024-04-02T17:24:58.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419306 3.82 MB/sec execute 505 sec latency 315.610 ms 2024-04-02T17:24:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 -- 172.21.15.59:0/1022244491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800104ed0 msgr2=0x7fa8001072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 --2- 172.21.15.59:0/1022244491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800104ed0 0x7fa8001072c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009860 tx=0x7fa7e802bb70 comp rx=0 tx=0).stop 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 -- 172.21.15.59:0/1022244491 shutdown_connections 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 --2- 172.21.15.59:0/1022244491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800107800 0x7fa800109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 --2- 172.21.15.59:0/1022244491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800104ed0 0x7fa8001072c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 --2- 172.21.15.59:0/1022244491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 -- 172.21.15.59:0/1022244491 >> 172.21.15.59:0/1022244491 conn(0x7fa8000fbf90 msgr2=0x7fa8000fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 -- 172.21.15.59:0/1022244491 shutdown_connections 2024-04-02T17:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.382+0000 7fa805000640 1 -- 172.21.15.59:0/1022244491 wait complete. 2024-04-02T17:24:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.383+0000 7fa805000640 1 Processor -- start 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.383+0000 7fa805000640 1 -- start start 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 0x7fa8001a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010d790 con 0x7fa800104ed0 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010d610 con 0x7fa8001025a0 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa805000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa80010d910 con 0x7fa800107800 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51690/0 (socket says 172.21.15.59:51690) 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fe800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 0x7fa8001a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46032/0 (socket says 172.21.15.59:46032) 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 -- 172.21.15.59:0/3163119087 learned_addr learned my addr 172.21.15.59:0/3163119087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 -- 172.21.15.59:0/3163119087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 msgr2=0x7fa8001a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 0x7fa8001a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 -- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 msgr2=0x7fa80019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fde00640 1 -- 172.21.15.59:0/3163119087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001a1ba0 con 0x7fa8001025a0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.384+0000 7fa7fe800640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 0x7fa8001a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e9f0 tx=0x7fa7f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa7f6a00640 1 -- 172.21.15.59:0/3163119087 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000cd80 con 0x7fa8001025a0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa7f6a00640 1 -- 172.21.15.59:0/3163119087 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f0010910 con 0x7fa8001025a0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa7f6a00640 1 -- 172.21.15.59:0/3163119087 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0018990 con 0x7fa8001025a0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001a1e90 con 0x7fa8001025a0 2024-04-02T17:24:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.385+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001a2300 con 0x7fa8001025a0 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.388+0000 7fa807dc3740 1 Processor -- start 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.388+0000 7fa7f6a00640 1 -- 172.21.15.59:0/3163119087 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa7f0010ac0 con 0x7fa8001025a0 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.388+0000 7fa807dc3740 1 -- start start 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba0722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba0773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddb9fd7d70 con 0x55ddba071ed0 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ddb9f802a0 con 0x55ddba077a30 2024-04-02T17:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa807dc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ddb9fd3d20 con 0x55ddba0728c0 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51696/0 (socket says 172.21.15.59:51696) 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa7f7e00640 1 -- 172.21.15.59:0/2044506860 learned_addr learned my addr 172.21.15.59:0/2044506860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa7fe800640 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.389+0000 7fa7fde00640 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba0773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 -- 172.21.15.59:0/2044506860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 msgr2=0x55ddba0773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba0773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 -- 172.21.15.59:0/2044506860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 msgr2=0x55ddba0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba0722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 -- 172.21.15.59:0/2044506860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb9f94840 con 0x55ddba077a30 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009830 tx=0x7fa7e8034ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24632c9fc4812d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f6a00640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7ef400640 1 -- 172.21.15.59:0/2044506860 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8044020 con 0x55ddba077a30 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7ef400640 1 -- 172.21.15.59:0/2044506860 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e8034640 con 0x55ddba077a30 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7ef400640 1 -- 172.21.15.59:0/2044506860 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8039040 con 0x55ddba077a30 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.390+0000 7fa7f6a00640 1 -- 172.21.15.59:0/3163119087 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7f00d0730 con 0x7fa8001025a0 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa7fe800640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 -- 172.21.15.59:0/2044506860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 msgr2=0x55ddba079ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009830 tx=0x7fa7e8034ab0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 -- 172.21.15.59:0/2044506860 shutdown_connections 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba079ec0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba0773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 --2- 172.21.15.59:0/2044506860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba0722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 -- 172.21.15.59:0/2044506860 >> 172.21.15.59:0/2044506860 conn(0x55ddba06d440 msgr2=0x55ddba06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 -- 172.21.15.59:0/2044506860 shutdown_connections 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa7fe800640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55ddba073120 tx=0x7fa7f8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.391+0000 7fa807dc3740 1 -- 172.21.15.59:0/2044506860 wait complete. 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.392+0000 7fa807dc3740 1 -- start start 2024-04-02T17:24:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.392+0000 7fa807dc3740 1 Processor -- start 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.392+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba071ed0 0x55ddba07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.392+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.392+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba083940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa807dc3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddba07a980 con 0x55ddba0728c0 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa807dc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ddb9f93660 con 0x55ddba077a30 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa807dc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ddba07ab00 con 0x55ddba071ed0 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa807dc3740 10 client.0 _ll_register_callbacks cb 0x55ddb9f93050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7fe800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba083940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46052/0 (socket says 172.21.15.59:46052) 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 -- 172.21.15.59:0/839705701 learned_addr learned my addr 172.21.15.59:0/839705701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7fde00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba071ed0 0x55ddba07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 -- 172.21.15.59:0/839705701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba071ed0 msgr2=0x55ddba07c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba071ed0 0x55ddba07c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 -- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 msgr2=0x55ddba083940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba083940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7f7e00640 1 -- 172.21.15.59:0/839705701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb9fa2e80 con 0x55ddba0728c0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.393+0000 7fa7fe800640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba083940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8037040 tx=0x7fa7e8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83362efaa7b6160b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa7e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e804f070 con 0x55ddba0728c0 2024-04-02T17:24:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7e8032640 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e8034640 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa7e805b050 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa807dc3740 10 client.16791 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.394+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ddb9cec210 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e8034640 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa7e8049a50 con 0x55ddba0728c0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 10 client.16791 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 20 client.16791 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba087e30 0x55ddba08a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 4 client.16791 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb9f87cd0 con 0x55ddba087e30 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.395+0000 7fa7fde00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba087e30 0x55ddba08a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.396+0000 7fa7fde00640 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba087e30 0x55ddba08a220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd46bc3e88fd540 server_cookie=300e7e31cec4c50c in_seq=0 out_seq=0 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.396+0000 7fa7e3e00640 10 client.16791 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa7e3e00640 1 -- 172.21.15.59:0/839705701 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddb9f87cd0 con 0x55ddba087e30 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa7e3e00640 10 client.16791 handle_command_reply: tid=0 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 client.16791 shutdown 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 msgr2=0x55ddba07c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8037040 tx=0x7fa7e8005900 comp rx=0 tx=0).stop 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 shutdown_connections 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba087e30 0x55ddba08a220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba0728c0 0x55ddba07c900 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba083940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 --2- 172.21.15.59:0/839705701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba071ed0 0x55ddba07c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa7e3e00640 0 client.16791 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.397+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 >> 172.21.15.59:0/839705701 conn(0x55ddba06d440 msgr2=0x55ddba06c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.398+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 shutdown_connections 2024-04-02T17:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.398+0000 7fa807dc3740 1 -- 172.21.15.59:0/839705701 wait complete. 2024-04-02T17:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.398+0000 7fa807dc3740 20 client.16791 trim_cache size 0 max 16384 2024-04-02T17:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 Processor -- start 2024-04-02T17:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 -- start start 2024-04-02T17:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba083900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ab290 con 0x55ddba071ed0 2024-04-02T17:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ace30 con 0x55ddba077a30 2024-04-02T17:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.414+0000 7fa807dc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ab960 con 0x55ddba0728c0 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46064/0 (socket says 172.21.15.59:46064) 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 -- 172.21.15.59:0/4074169034 learned_addr learned my addr 172.21.15.59:0/4074169034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fe800640 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba083900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 -- 172.21.15.59:0/4074169034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 msgr2=0x55ddba083900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba083900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 -- 172.21.15.59:0/4074169034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 msgr2=0x55ddba06c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba06c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.415+0000 7fa7fde00640 1 -- 172.21.15.59:0/4074169034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb9db8620 con 0x55ddba071ed0 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa7fde00640 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000cfd0 tx=0x7fa7f00ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e673edf3f04fc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa7eea00640 1 -- 172.21.15.59:0/4074169034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a55c0 con 0x55ddba071ed0 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa7eea00640 1 -- 172.21.15.59:0/4074169034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f009ad00 con 0x55ddba071ed0 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa7eea00640 1 -- 172.21.15.59:0/4074169034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0018b30 con 0x55ddba071ed0 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 -- 172.21.15.59:0/4074169034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 msgr2=0x55ddba06bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000cfd0 tx=0x7fa7f00ce040 comp rx=0 tx=0).stop 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 -- 172.21.15.59:0/4074169034 shutdown_connections 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba077a30 0x55ddba06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba083900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 --2- 172.21.15.59:0/4074169034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba071ed0 0x55ddba06bdb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.416+0000 7fa807dc3740 1 -- 172.21.15.59:0/4074169034 >> 172.21.15.59:0/4074169034 conn(0x55ddba06d440 msgr2=0x55ddba073e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 -- 172.21.15.59:0/4074169034 shutdown_connections 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 -- 172.21.15.59:0/4074169034 wait complete. 2024-04-02T17:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 -- start start 2024-04-02T17:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 Processor -- start 2024-04-02T17:24:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba071ed0 0x55ddba0807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 0x55ddba0f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.417+0000 7fa807dc3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ac760 con 0x55ddba077a30 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa807dc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ac030 con 0x55ddba071ed0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa807dc3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001a9270 con 0x55ddba0728c0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa807dc3740 10 client.0 _ll_register_callbacks cb 0x7fa7e80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57758/0 (socket says 172.21.15.59:57758) 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 -- 172.21.15.59:0/3366727600 learned_addr learned my addr 172.21.15.59:0/3366727600 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7fe800640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 0x55ddba0f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba071ed0 0x55ddba0807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 -- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba071ed0 msgr2=0x55ddba0807a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba071ed0 0x55ddba0807a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 -- 172.21.15.59:0/3366727600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 msgr2=0x55ddba0f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 0x55ddba0f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7f7e00640 1 -- 172.21.15.59:0/3366727600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddba0f4fb0 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.418+0000 7fa7fe800640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 0x55ddba0f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7f7e00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802c0b0 tx=0x7fa7e8032380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520a9f5aa69eba69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e804f070 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f009ad00 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fa7e8034640 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa7e8066050 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa807dc3740 10 client.26902 fetch_fsmap learned FSMap version 37 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ddba0f6180 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.419+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f009ad00 con 0x55ddba0728c0 2024-04-02T17:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa7e8049780 con 0x55ddba0728c0 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 10 client.26902 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 20 client.26902 populate_metadata read hostname 'smithi059' 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba0f6aa0 0x55ddba0f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 4 client.26902 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddb9fe52b0 con 0x55ddba0f6aa0 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.420+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba0f6aa0 0x55ddba0f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.421+0000 7fa7fde00640 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba0f6aa0 0x55ddba0f8ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c986a8ce295194c0 server_cookie=d0efe85ca943e70b in_seq=0 out_seq=0 2024-04-02T17:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.421+0000 7fa7e3e00640 10 client.26902 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa7e3e00640 1 -- 172.21.15.59:0/3366727600 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddb9fe52b0 con 0x55ddba0f6aa0 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa7e3e00640 10 client.26902 handle_command_reply: tid=0 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 client.26902 shutdown 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 msgr2=0x55ddba080ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802c0b0 tx=0x7fa7e8032380 comp rx=0 tx=0).stop 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 shutdown_connections 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ddba0f6aa0 0x55ddba0f8ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddba077a30 0x55ddba0f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ddba0728c0 0x55ddba080ce0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 --2- 172.21.15.59:0/3366727600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ddba071ed0 0x55ddba0807a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 >> 172.21.15.59:0/3366727600 conn(0x55ddba06d440 msgr2=0x55ddba0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.453+0000 7fa7e3e00640 0 client.26902 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.454+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 shutdown_connections 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.454+0000 7fa807dc3740 1 -- 172.21.15.59:0/3366727600 wait complete. 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.454+0000 7fa807dc3740 20 client.26902 trim_cache size 0 max 16384 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:24:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 msgr2=0x7fa7e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55ddba073120 tx=0x7fa7f8010040 comp rx=0 tx=0).stop 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 msgr2=0x7fa800199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e9f0 tx=0x7fa7f000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 shutdown_connections 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7e40a3cd0 0x7fa7e40a6190 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa800104ed0 0x7fa80019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa800107800 0x7fa8001a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 --2- 172.21.15.59:0/3163119087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8001025a0 0x7fa800199f80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 >> 172.21.15.59:0/3163119087 conn(0x7fa8000fbf90 msgr2=0x7fa800101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 shutdown_connections 2024-04-02T17:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:24:58.455+0000 7fa805000640 1 -- 172.21.15.59:0/3163119087 wait complete. 2024-04-02T17:24:59.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419374 3.82 MB/sec execute 506 sec latency 921.964 ms 2024-04-02T17:24:59.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:24:59 smithi059 ceph-mon[31664]: pgmap v745: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 559 KiB/s wr, 83 op/s 2024-04-02T17:24:59.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:24:59 smithi164 ceph-mon[36499]: pgmap v745: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 559 KiB/s wr, 83 op/s 2024-04-02T17:24:59.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:24:59 smithi154 ceph-mon[36479]: pgmap v745: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 559 KiB/s wr, 83 op/s 2024-04-02T17:25:00.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419374 3.81 MB/sec execute 507 sec latency 1314.503 ms 2024-04-02T17:25:01.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419374 3.80 MB/sec execute 508 sec latency 2314.647 ms 2024-04-02T17:25:01.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:01 smithi059 ceph-mon[31664]: pgmap v746: 97 pgs: 97 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 760 KiB/s wr, 109 op/s 2024-04-02T17:25:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:01 smithi164 ceph-mon[36499]: pgmap v746: 97 pgs: 97 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 760 KiB/s wr, 109 op/s 2024-04-02T17:25:01.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:01 smithi154 ceph-mon[36479]: pgmap v746: 97 pgs: 97 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 760 KiB/s wr, 109 op/s 2024-04-02T17:25:02.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419374 3.79 MB/sec execute 509 sec latency 3314.787 ms 2024-04-02T17:25:03.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419374 3.79 MB/sec execute 510 sec latency 4314.916 ms 2024-04-02T17:25:03.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:03 smithi059 ceph-mon[31664]: pgmap v747: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 660 KiB/s wr, 105 op/s 2024-04-02T17:25:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:03 smithi164 ceph-mon[36499]: pgmap v747: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 660 KiB/s wr, 105 op/s 2024-04-02T17:25:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:03 smithi154 ceph-mon[36479]: pgmap v747: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 660 KiB/s wr, 105 op/s 2024-04-02T17:25:04.110 INFO:tasks.workunit.client.0.smithi059.stdout: 1 419738 3.78 MB/sec execute 511 sec latency 4926.635 ms 2024-04-02T17:25:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 421349 3.79 MB/sec execute 512 sec latency 31.998 ms 2024-04-02T17:25:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:05 smithi059 ceph-mon[31664]: pgmap v748: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 731 KiB/s wr, 114 op/s 2024-04-02T17:25:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:05 smithi164 ceph-mon[36499]: pgmap v748: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 731 KiB/s wr, 114 op/s 2024-04-02T17:25:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:05 smithi154 ceph-mon[36479]: pgmap v748: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 731 KiB/s wr, 114 op/s 2024-04-02T17:25:06.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422363 3.80 MB/sec execute 513 sec latency 26.555 ms 2024-04-02T17:25:07.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422593 3.79 MB/sec execute 514 sec latency 25.165 ms 2024-04-02T17:25:07.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:07 smithi059 ceph-mon[31664]: pgmap v749: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 572 KiB/s wr, 82 op/s 2024-04-02T17:25:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:07 smithi164 ceph-mon[36499]: pgmap v749: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 572 KiB/s wr, 82 op/s 2024-04-02T17:25:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:07 smithi154 ceph-mon[36479]: pgmap v749: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 572 KiB/s wr, 82 op/s 2024-04-02T17:25:08.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422798 3.79 MB/sec execute 515 sec latency 697.990 ms 2024-04-02T17:25:09.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422922 3.78 MB/sec execute 516 sec latency 1308.673 ms 2024-04-02T17:25:09.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:09 smithi059 ceph-mon[31664]: pgmap v750: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 571 KiB/s wr, 88 op/s 2024-04-02T17:25:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:09 smithi164 ceph-mon[36499]: pgmap v750: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 571 KiB/s wr, 88 op/s 2024-04-02T17:25:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:09 smithi154 ceph-mon[36479]: pgmap v750: 97 pgs: 97 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 571 KiB/s wr, 88 op/s 2024-04-02T17:25:10.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422922 3.77 MB/sec execute 517 sec latency 1194.644 ms 2024-04-02T17:25:11.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422922 3.77 MB/sec execute 518 sec latency 2194.777 ms 2024-04-02T17:25:11.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:11 smithi059 ceph-mon[31664]: pgmap v751: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-04-02T17:25:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:11 smithi164 ceph-mon[36499]: pgmap v751: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-04-02T17:25:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:11 smithi154 ceph-mon[36479]: pgmap v751: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-04-02T17:25:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 -- 172.21.15.59:0/72413368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 msgr2=0x7f319c077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 --2- 172.21.15.59:0/72413368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c077b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3184009900 tx=0x7f318402bc30 comp rx=0 tx=0).stop 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 -- 172.21.15.59:0/72413368 shutdown_connections 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 --2- 172.21.15.59:0/72413368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0780a0 0x7f319c078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 --2- 172.21.15.59:0/72413368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c077b60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 --2- 172.21.15.59:0/72413368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0764c0 0x7f319c0768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 -- 172.21.15.59:0/72413368 >> 172.21.15.59:0/72413368 conn(0x7f319c0ff2c0 msgr2=0x7f319c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 -- 172.21.15.59:0/72413368 shutdown_connections 2024-04-02T17:25:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.606+0000 7f31a1000640 1 -- 172.21.15.59:0/72413368 wait complete. 2024-04-02T17:25:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.607+0000 7f31a1000640 1 Processor -- start 2024-04-02T17:25:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 -- start start 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0780a0 0x7f319c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113db0 con 0x7f319c0780a0 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113c30 con 0x7f319c0764c0 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f31a1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319c113f30 con 0x7f319c077700 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f3199e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f3191400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.608+0000 7f3199e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57892/0 (socket says 172.21.15.59:57892) 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3199e00640 1 -- 172.21.15.59:0/936299273 learned_addr learned my addr 172.21.15.59:0/936299273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f319a800640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0780a0 0x7f319c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 -- 172.21.15.59:0/936299273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 msgr2=0x7f319c19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 -- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0780a0 msgr2=0x7f319c1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0780a0 0x7f319c1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 -- 172.21.15.59:0/936299273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a5fe0 con 0x7f319c077700 2024-04-02T17:25:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3199e00640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.609+0000 7f3191400640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c19e900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3184009230 tx=0x7f318402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.610+0000 7f3193400640 1 -- 172.21.15.59:0/936299273 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184041070 con 0x7f319c077700 2024-04-02T17:25:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.610+0000 7f3193400640 1 -- 172.21.15.59:0/936299273 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3184004450 con 0x7f319c077700 2024-04-02T17:25:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.610+0000 7f3193400640 1 -- 172.21.15.59:0/936299273 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184003d80 con 0x7f319c077700 2024-04-02T17:25:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.610+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a6270 con 0x7f319c077700 2024-04-02T17:25:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.610+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a6760 con 0x7f319c077700 2024-04-02T17:25:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.613+0000 7f31a3c74740 1 Processor -- start 2024-04-02T17:25:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.613+0000 7f3193400640 1 -- 172.21.15.59:0/936299273 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3184044070 con 0x7f319c077700 2024-04-02T17:25:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 -- start start 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba345c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba345eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561ba33bcd70 con 0x561ba3456ed0 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561ba33652a0 con 0x561ba34578c0 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f31a3c74740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561ba33b8d20 con 0x561ba345ca30 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f319a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f3199e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba345c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f319a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51692/0 (socket says 172.21.15.59:51692) 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f319a800640 1 -- 172.21.15.59:0/760035213 learned_addr learned my addr 172.21.15.59:0/760035213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.614+0000 7f3191400640 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba345eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 -- 172.21.15.59:0/760035213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 msgr2=0x561ba345eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba345eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 -- 172.21.15.59:0/760035213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 msgr2=0x561ba34572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 -- 172.21.15.59:0/760035213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba3379840 con 0x561ba34578c0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3199e00640 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba345c3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f318c0114f0 tx=0x7f318c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0c8ac072a0f984 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3193400640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 0x7f31800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f318b400640 1 -- 172.21.15.59:0/760035213 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c01e070 con 0x561ba34578c0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f318b400640 1 -- 172.21.15.59:0/760035213 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0035f0 con 0x561ba34578c0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f318b400640 1 -- 172.21.15.59:0/760035213 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00f7c0 con 0x561ba34578c0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.615+0000 7f3193400640 1 -- 172.21.15.59:0/936299273 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f31840f1ea0 con 0x7f319c077700 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f319a800640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 0x7f31800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 -- 172.21.15.59:0/760035213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 msgr2=0x561ba345c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba345c3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f318c0114f0 tx=0x7f318c0119b0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 -- 172.21.15.59:0/760035213 shutdown_connections 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba345eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba345c3c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 --2- 172.21.15.59:0/760035213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 -- 172.21.15.59:0/760035213 >> 172.21.15.59:0/760035213 conn(0x561ba3452440 msgr2=0x561ba3452840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 -- 172.21.15.59:0/760035213 shutdown_connections 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f31a3c74740 1 -- 172.21.15.59:0/760035213 wait complete. 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.616+0000 7f319a800640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 0x7f31800a6190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561ba3458120 tx=0x7f3194009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 -- start start 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 Processor -- start 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba34618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba3468920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561ba345f9e0 con 0x561ba3456ed0 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561ba3378660 con 0x561ba34578c0 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.617+0000 7f31a3c74740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561ba345fb60 con 0x561ba345ca30 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f31a3c74740 10 client.0 _ll_register_callbacks cb 0x561ba3378050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51696/0 (socket says 172.21.15.59:51696) 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 -- 172.21.15.59:0/377026227 learned_addr learned my addr 172.21.15.59:0/377026227 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f319a800640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba3468920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3191400640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba34618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 -- 172.21.15.59:0/377026227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 msgr2=0x561ba3468920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba3468920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 -- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 msgr2=0x561ba34618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba34618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.618+0000 7f3199e00640 1 -- 172.21.15.59:0/377026227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba3387e80 con 0x561ba3456ed0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f3199e00640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2ee43bb05a20dc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f317fe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c01e030 con 0x561ba3456ed0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f318c005260 con 0x561ba3456ed0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c0035f0 con 0x561ba3456ed0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f318c005460 con 0x561ba3456ed0 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f31a3c74740 10 client.16809 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561ba30d1210 con 0x561ba3456ed0 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.619+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0035f0 con 0x561ba3456ed0 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f318c00f650 con 0x561ba3456ed0 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 10 client.16809 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 20 client.16809 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba346cf30 0x561ba346f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 4 client.16809 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ba336ccd0 con 0x561ba346cf30 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.620+0000 7f3191400640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba346cf30 0x561ba346f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.621+0000 7f3191400640 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba346cf30 0x561ba346f320 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfe7c051b081014 server_cookie=170fbc11b7c9ec83 in_seq=0 out_seq=0 2024-04-02T17:25:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.621+0000 7f317fe00640 10 client.16809 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f317fe00640 1 -- 172.21.15.59:0/377026227 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ba336ccd0 con 0x561ba346cf30 2024-04-02T17:25:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f317fe00640 10 client.16809 handle_command_reply: tid=0 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 client.16809 shutdown 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 msgr2=0x561ba34613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c003a50 comp rx=0 tx=0).stop 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 shutdown_connections 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba346cf30 0x561ba346f320 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba345ca30 0x561ba3468920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba34618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 --2- 172.21.15.59:0/377026227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba34613a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f317fe00640 0 client.16809 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 >> 172.21.15.59:0/377026227 conn(0x561ba3452440 msgr2=0x561ba3451640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 shutdown_connections 2024-04-02T17:25:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.622+0000 7f31a3c74740 1 -- 172.21.15.59:0/377026227 wait complete. 2024-04-02T17:25:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.623+0000 7f31a3c74740 20 client.16809 trim_cache size 0 max 16384 2024-04-02T17:25:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 Processor -- start 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 -- start start 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba3456ed0 0x561ba34688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba3455db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1af710 con 0x561ba3452440 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b12b0 con 0x561ba34578c0 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f31a3c74740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1afde0 con 0x561ba3456ed0 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.639+0000 7f3199e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51704/0 (socket says 172.21.15.59:51704) 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 -- 172.21.15.59:0/4236356926 learned_addr learned my addr 172.21.15.59:0/4236356926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f319a800640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba3455db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3191400640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba3456ed0 0x561ba34688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 -- 172.21.15.59:0/4236356926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba3456ed0 msgr2=0x561ba34688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba3456ed0 0x561ba34688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 -- 172.21.15.59:0/4236356926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 msgr2=0x561ba3455db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba3455db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 -- 172.21.15.59:0/4236356926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba319d620 con 0x561ba3452440 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f319a800640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba3455db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.640+0000 7f3199e00640 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f318c02f040 tx=0x7f318c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb30e0b21f3567c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f318aa00640 1 -- 172.21.15.59:0/4236356926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c01e070 con 0x561ba3452440 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f318aa00640 1 -- 172.21.15.59:0/4236356926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c028070 con 0x561ba3452440 2024-04-02T17:25:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f318aa00640 1 -- 172.21.15.59:0/4236356926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00f5c0 con 0x561ba3452440 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 -- 172.21.15.59:0/4236356926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 msgr2=0x561ba34616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f318c02f040 tx=0x7f318c009fa0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 -- 172.21.15.59:0/4236356926 shutdown_connections 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba34578c0 0x561ba3455db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba3456ed0 0x561ba34688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 --2- 172.21.15.59:0/4236356926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3452440 0x561ba34616c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.641+0000 7f31a3c74740 1 -- 172.21.15.59:0/4236356926 >> 172.21.15.59:0/4236356926 conn(0x561ba34586a0 msgr2=0x561ba34556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.642+0000 7f31a3c74740 1 -- 172.21.15.59:0/4236356926 shutdown_connections 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.642+0000 7f31a3c74740 1 -- 172.21.15.59:0/4236356926 wait complete. 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.642+0000 7f31a3c74740 1 -- start start 2024-04-02T17:25:11.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.642+0000 7f31a3c74740 1 Processor -- start 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba3452440 0x561ba3465f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba34578c0 0x561ba34674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b0be0 con 0x561ba3456ed0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b04b0 con 0x561ba3452440 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ad6f0 con 0x561ba34578c0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f31a3c74740 10 client.0 _ll_register_callbacks cb 0x7f318c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51706/0 (socket says 172.21.15.59:51706) 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 -- 172.21.15.59:0/355238540 learned_addr learned my addr 172.21.15.59:0/355238540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f319a800640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba34578c0 0x561ba34674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3199e00640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba3452440 0x561ba3465f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 -- 172.21.15.59:0/355238540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba34578c0 msgr2=0x561ba34674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba34578c0 0x561ba34674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 -- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba3452440 msgr2=0x561ba3465f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba3452440 0x561ba3465f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.643+0000 7f3191400640 1 -- 172.21.15.59:0/355238540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba3467a90 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f3191400640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31840baa90 tx=0x7f3184000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea9ff3b3ac25c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f317fe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318404b040 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f31840ef9f0 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c028070 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f31840bda20 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f31a3c74740 10 client.16815 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.644+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561ba34d34a0 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c028070 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f31840ba730 con 0x561ba3456ed0 2024-04-02T17:25:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 10 client.16815 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 20 client.16815 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba345ca30 0x561ba34d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 4 client.16815 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ba33ca2b0 con 0x561ba345ca30 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.645+0000 7f3199e00640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba345ca30 0x561ba34d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.646+0000 7f3199e00640 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba345ca30 0x561ba34d5e50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8dbf97e839c3 server_cookie=d1593e592eb6f7f4 in_seq=0 out_seq=0 2024-04-02T17:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.646+0000 7f317fe00640 10 client.16815 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f317fe00640 1 -- 172.21.15.59:0/355238540 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ba33ca2b0 con 0x561ba345ca30 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f317fe00640 10 client.16815 handle_command_reply: tid=0 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f31a3c74740 1 client.16815 shutdown 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 msgr2=0x561ba3466490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31840baa90 tx=0x7f3184000bc0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.680+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 shutdown_connections 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561ba345ca30 0x561ba34d5e50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ba3456ed0 0x561ba3466490 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ba34578c0 0x561ba34674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 --2- 172.21.15.59:0/355238540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ba3452440 0x561ba3465f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f317fe00640 0 client.16815 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 >> 172.21.15.59:0/355238540 conn(0x561ba34586a0 msgr2=0x561ba3455f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 shutdown_connections 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 1 -- 172.21.15.59:0/355238540 wait complete. 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.681+0000 7f31a3c74740 20 client.16815 trim_cache size 0 max 16384 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:25:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 msgr2=0x7f31800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 0x7f31800a6190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561ba3458120 tx=0x7f3194009500 comp rx=0 tx=0).stop 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 msgr2=0x7f319c19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c19e900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3184009230 tx=0x7f318402c650 comp rx=0 tx=0).stop 2024-04-02T17:25:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 shutdown_connections 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3cd0 0x7f31800a6190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f319c0780a0 0x7f319c1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c077700 0x7f319c19e900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 --2- 172.21.15.59:0/936299273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319c0764c0 0x7f319c19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 >> 172.21.15.59:0/936299273 conn(0x7f319c0ff2c0 msgr2=0x7f319c100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 shutdown_connections 2024-04-02T17:25:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:11.682+0000 7f31a1000640 1 -- 172.21.15.59:0/936299273 wait complete. 2024-04-02T17:25:12.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422922 3.76 MB/sec execute 519 sec latency 3194.870 ms 2024-04-02T17:25:13.111 INFO:tasks.workunit.client.0.smithi059.stdout: 1 422922 3.75 MB/sec execute 520 sec latency 4195.007 ms 2024-04-02T17:25:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:13 smithi164 ceph-mon[36499]: pgmap v752: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-02T17:25:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:13 smithi154 ceph-mon[36479]: pgmap v752: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-02T17:25:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:13 smithi059 ceph-mon[31664]: pgmap v752: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-02T17:25:14.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 423240 3.75 MB/sec execute 521 sec latency 4804.334 ms 2024-04-02T17:25:15.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 424723 3.75 MB/sec execute 522 sec latency 30.914 ms 2024-04-02T17:25:15.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:15 smithi164 ceph-mon[36499]: pgmap v753: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 374 KiB/s wr, 80 op/s 2024-04-02T17:25:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:15 smithi154 ceph-mon[36479]: pgmap v753: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 374 KiB/s wr, 80 op/s 2024-04-02T17:25:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:15 smithi059 ceph-mon[31664]: pgmap v753: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 374 KiB/s wr, 80 op/s 2024-04-02T17:25:16.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 425885 3.76 MB/sec execute 523 sec latency 18.754 ms 2024-04-02T17:25:17.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 426104 3.76 MB/sec execute 524 sec latency 29.625 ms 2024-04-02T17:25:17.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:17 smithi164 ceph-mon[36499]: pgmap v754: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 247 KiB/s wr, 65 op/s 2024-04-02T17:25:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:17 smithi154 ceph-mon[36479]: pgmap v754: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 247 KiB/s wr, 65 op/s 2024-04-02T17:25:17.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:17 smithi059 ceph-mon[31664]: pgmap v754: 97 pgs: 97 active+clean; 106 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 247 KiB/s wr, 65 op/s 2024-04-02T17:25:18.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 426396 3.75 MB/sec execute 525 sec latency 384.222 ms 2024-04-02T17:25:19.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 426715 3.75 MB/sec execute 526 sec latency 992.461 ms 2024-04-02T17:25:19.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:19 smithi164 ceph-mon[36499]: pgmap v755: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 329 KiB/s wr, 78 op/s 2024-04-02T17:25:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:19 smithi154 ceph-mon[36479]: pgmap v755: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 329 KiB/s wr, 78 op/s 2024-04-02T17:25:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:19 smithi059 ceph-mon[31664]: pgmap v755: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 329 KiB/s wr, 78 op/s 2024-04-02T17:25:20.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 428212 3.75 MB/sec execute 527 sec latency 28.150 ms 2024-04-02T17:25:21.112 INFO:tasks.workunit.client.0.smithi059.stdout: 1 429378 3.76 MB/sec execute 528 sec latency 18.298 ms 2024-04-02T17:25:21.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:21 smithi164 ceph-mon[36499]: pgmap v756: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 541 KiB/s wr, 98 op/s 2024-04-02T17:25:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:21 smithi154 ceph-mon[36479]: pgmap v756: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 541 KiB/s wr, 98 op/s 2024-04-02T17:25:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:21 smithi059 ceph-mon[31664]: pgmap v756: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 541 KiB/s wr, 98 op/s 2024-04-02T17:25:22.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 429638 3.76 MB/sec execute 529 sec latency 33.059 ms 2024-04-02T17:25:23.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 430203 3.75 MB/sec execute 530 sec latency 23.537 ms 2024-04-02T17:25:23.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:23 smithi154 ceph-mon[36479]: pgmap v757: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 455 KiB/s wr, 75 op/s 2024-04-02T17:25:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:23 smithi059 ceph-mon[31664]: pgmap v757: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 455 KiB/s wr, 75 op/s 2024-04-02T17:25:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:23 smithi164 ceph-mon[36499]: pgmap v757: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 455 KiB/s wr, 75 op/s 2024-04-02T17:25:24.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 431707 3.76 MB/sec execute 531 sec latency 18.431 ms 2024-04-02T17:25:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 -- 172.21.15.59:0/1026952017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 msgr2=0x7fe724109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 --2- 172.21.15.59:0/1026952017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe724109a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe720009ca0 tx=0x7fe72002c100 comp rx=0 tx=0).stop 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 -- 172.21.15.59:0/1026952017 shutdown_connections 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 --2- 172.21.15.59:0/1026952017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe724104d00 0x7fe7241070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 --2- 172.21.15.59:0/1026952017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe724109a80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 --2- 172.21.15.59:0/1026952017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7241023d0 0x7fe7241047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 -- 172.21.15.59:0/1026952017 >> 172.21.15.59:0/1026952017 conn(0x7fe7240fbde0 msgr2=0x7fe7240fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 -- 172.21.15.59:0/1026952017 shutdown_connections 2024-04-02T17:25:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.830+0000 7fe72b200640 1 -- 172.21.15.59:0/1026952017 wait complete. 2024-04-02T17:25:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.831+0000 7fe72b200640 1 Processor -- start 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.831+0000 7fe72b200640 1 -- start start 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241023d0 0x7fe724199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe7241a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410f820 con 0x7fe7241023d0 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410f6a0 con 0x7fe724107630 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410f9a0 con 0x7fe724104d00 2024-04-02T17:25:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57904/0 (socket says 172.21.15.59:57904) 2024-04-02T17:25:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 -- 172.21.15.59:0/1836832138 learned_addr learned my addr 172.21.15.59:0/1836832138 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe72a800640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe7241a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729e00640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241023d0 0x7fe724199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 -- 172.21.15.59:0/1836832138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 msgr2=0x7fe7241a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe7241a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 -- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241023d0 msgr2=0x7fe724199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241023d0 0x7fe724199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.832+0000 7fe729400640 1 -- 172.21.15.59:0/1836832138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a19f0 con 0x7fe724104d00 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe729400640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9c0 tx=0x7fe71400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe71ea00640 1 -- 172.21.15.59:0/1836832138 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cd80 con 0x7fe724104d00 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe71ea00640 1 -- 172.21.15.59:0/1836832138 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe714005500 con 0x7fe724104d00 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a1ce0 con 0x7fe724104d00 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe71ea00640 1 -- 172.21.15.59:0/1836832138 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010690 con 0x7fe724104d00 2024-04-02T17:25:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.833+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a2220 con 0x7fe724104d00 2024-04-02T17:25:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 Processor -- start 2024-04-02T17:25:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 -- start start 2024-04-02T17:25:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbc7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbccc60 0x562f5fbcf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f5fb2cd70 con 0x562f5fbc7100 2024-04-02T17:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562f5fad52a0 con 0x562f5fbccc60 2024-04-02T17:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72f433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562f5fb28d20 con 0x562f5fbc7af0 2024-04-02T17:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe71ea00640 1 -- 172.21.15.59:0/1836832138 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe7140046c0 con 0x7fe724104d00 2024-04-02T17:25:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe729e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.837+0000 7fe72a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbc7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57914/0 (socket says 172.21.15.59:57914) 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 -- 172.21.15.59:0/2464117364 learned_addr learned my addr 172.21.15.59:0/2464117364 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729400640 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbccc60 0x562f5fbcf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 -- 172.21.15.59:0/2464117364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbccc60 msgr2=0x562f5fbcf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbccc60 0x562f5fbcf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 -- 172.21.15.59:0/2464117364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 msgr2=0x562f5fbc7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbc7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 -- 172.21.15.59:0/2464117364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5fae9840 con 0x562f5fbc7af0 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.838+0000 7fe729e00640 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe718009d20 tx=0x7fe718011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146d7498ea4e55d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe713000640 1 -- 172.21.15.59:0/2464117364 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71801e070 con 0x562f5fbc7af0 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe713000640 1 -- 172.21.15.59:0/2464117364 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718019b10 con 0x562f5fbc7af0 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe713000640 1 -- 172.21.15.59:0/2464117364 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71800ebf0 con 0x562f5fbc7af0 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe71ea00640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 0x7fe70c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe72f433740 1 -- 172.21.15.59:0/2464117364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 msgr2=0x562f5fbcc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.839+0000 7fe72f433740 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe718009d20 tx=0x7fe718011810 comp rx=0 tx=0).stop 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe71ea00640 1 -- 172.21.15.59:0/1836832138 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe7140d0760 con 0x7fe724104d00 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 -- 172.21.15.59:0/2464117364 shutdown_connections 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbccc60 0x562f5fbcf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbc7af0 0x562f5fbcc5f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 --2- 172.21.15.59:0/2464117364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbc7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 -- 172.21.15.59:0/2464117364 >> 172.21.15.59:0/2464117364 conn(0x562f5fbc2670 msgr2=0x562f5fbc2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72a800640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 0x7fe70c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 -- 172.21.15.59:0/2464117364 shutdown_connections 2024-04-02T17:25:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.840+0000 7fe72f433740 1 -- 172.21.15.59:0/2464117364 wait complete. 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.841+0000 7fe72f433740 1 -- start start 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.841+0000 7fe72f433740 1 Processor -- start 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.841+0000 7fe72a800640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 0x7fe70c0a60e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe72002c640 tx=0x7fe720032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.841+0000 7fe72f433740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbd9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.841+0000 7fe72f433740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72f433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbe13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72f433740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f5fbcfc10 con 0x562f5fbc7100 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72f433740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562f5fae8660 con 0x562f5fbc7af0 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72f433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562f5fbcfd90 con 0x562f5fbccc60 2024-04-02T17:25:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72f433740 10 client.0 _ll_register_callbacks cb 0x562f5fae8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe72a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbe13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbd9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55726/0 (socket says 172.21.15.59:55726) 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 -- 172.21.15.59:0/1754917186 learned_addr learned my addr 172.21.15.59:0/1754917186 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 -- 172.21.15.59:0/1754917186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 msgr2=0x562f5fbe13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbe13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 -- 172.21.15.59:0/1754917186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 msgr2=0x562f5fbd9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbd9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.842+0000 7fe729400640 1 -- 172.21.15.59:0/1754917186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5faf7e80 con 0x562f5fbc7af0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe729400640 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe7140a50c0 tx=0x7fe714098d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbead900fef10b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe707400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140a5360 con 0x562f5fbc7af0 2024-04-02T17:25:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe7140d60d0 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe718019b10 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe71409d080 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe72f433740 10 client.26927 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f5f841210 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.843+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718019b10 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe7140870b0 con 0x562f5fbc7af0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 10 client.26927 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 20 client.26927 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fbe5e50 0x562f5fbe8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 4 client.26927 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5fadccd0 con 0x562f5fbe5e50 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.844+0000 7fe729e00640 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fbe5e50 0x562f5fbe8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.845+0000 7fe729e00640 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fbe5e50 0x562f5fbe8240 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34dfd877d47ab439 server_cookie=ddee5cb8a05cec9d in_seq=0 out_seq=0 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.845+0000 7fe707400640 10 client.26927 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.845+0000 7fe707400640 1 -- 172.21.15.59:0/1754917186 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f5fadccd0 con 0x562f5fbe5e50 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.845+0000 7fe707400640 10 client.26927 handle_command_reply: tid=0 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.845+0000 7fe72f433740 1 client.26927 shutdown 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 msgr2=0x562f5fbda400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe7140a50c0 tx=0x7fe714098d20 comp rx=0 tx=0).stop 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 shutdown_connections 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fbe5e50 0x562f5fbe8240 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbe13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7af0 0x562f5fbda400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 --2- 172.21.15.59:0/1754917186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7100 0x562f5fbd9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe707400640 0 client.26927 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 >> 172.21.15.59:0/1754917186 conn(0x562f5fbc2670 msgr2=0x562f5fbc1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 shutdown_connections 2024-04-02T17:25:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 1 -- 172.21.15.59:0/1754917186 wait complete. 2024-04-02T17:25:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.846+0000 7fe72f433740 20 client.26927 trim_cache size 0 max 16384 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.866+0000 7fe72f433740 1 Processor -- start 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.866+0000 7fe72f433740 1 -- start start 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a6cd0 con 0x562f5fbc7af0 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a8870 con 0x562f5fbc7100 2024-04-02T17:25:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72f433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a73a0 con 0x562f5fbccc60 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe729e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe729400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe729e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55738/0 (socket says 172.21.15.59:55738) 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe729400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55234/0 (socket says 172.21.15.59:55234) 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe729e00640 1 -- 172.21.15.59:0/1317731644 learned_addr learned my addr 172.21.15.59:0/1317731644 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.867+0000 7fe72a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 -- 172.21.15.59:0/1317731644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 msgr2=0x562f5fbec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 -- 172.21.15.59:0/1317731644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 msgr2=0x562f5fbe1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 -- 172.21.15.59:0/1317731644 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5f90d620 con 0x562f5fbc7100 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729400640 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe729e00640 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe718009c40 tx=0x7fe718009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1f4c0150f48fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe712600640 1 -- 172.21.15.59:0/1317731644 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71801e040 con 0x562f5fbc7100 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe712600640 1 -- 172.21.15.59:0/1317731644 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718005520 con 0x562f5fbc7100 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.868+0000 7fe712600640 1 -- 172.21.15.59:0/1317731644 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71800edf0 con 0x562f5fbc7100 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 -- 172.21.15.59:0/1317731644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 msgr2=0x562f5fbc5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe718009c40 tx=0x7fe718009660 comp rx=0 tx=0).stop 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 -- 172.21.15.59:0/1317731644 shutdown_connections 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbe1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 --2- 172.21.15.59:0/1317731644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbc5fe0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 -- 172.21.15.59:0/1317731644 >> 172.21.15.59:0/1317731644 conn(0x562f5fbc2670 msgr2=0x562f5fbc58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 -- 172.21.15.59:0/1317731644 shutdown_connections 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.869+0000 7fe72f433740 1 -- 172.21.15.59:0/1317731644 wait complete. 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 -- start start 2024-04-02T17:25:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 Processor -- start 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbdfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a81a0 con 0x562f5fbc7af0 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a7a70 con 0x562f5fbc7100 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241a4cb0 con 0x562f5fbccc60 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.870+0000 7fe72f433740 10 client.0 _ll_register_callbacks cb 0x7fe7140bd070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55742/0 (socket says 172.21.15.59:55742) 2024-04-02T17:25:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 -- 172.21.15.59:0/208587214 learned_addr learned my addr 172.21.15.59:0/208587214 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe72a800640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbdfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729400640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 -- 172.21.15.59:0/208587214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 msgr2=0x562f5fbdfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbdfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 -- 172.21.15.59:0/208587214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 msgr2=0x562f5fbdec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbdec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.871+0000 7fe729e00640 1 -- 172.21.15.59:0/208587214 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5fbe0260 con 0x562f5fbc7100 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe729e00640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe718009710 tx=0x7fe718009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad21f98bee878540 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe707400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71801e070 con 0x562f5fbc7100 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe718005520 con 0x562f5fbc7100 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe71801ad50 con 0x562f5fbc7100 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe718005090 con 0x562f5fbc7100 2024-04-02T17:25:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe72f433740 10 client.26933 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.872+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f5fbe1570 con 0x562f5fbc7100 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718005520 con 0x562f5fbc7100 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe718003ad0 con 0x562f5fbc7100 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 10 client.26933 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 10 client.26933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 20 client.26933 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fc4c630 0x562f5fc4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 4 client.26933 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f5fb3a2b0 con 0x562f5fc4c630 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.873+0000 7fe729400640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fc4c630 0x562f5fc4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.874+0000 7fe729400640 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fc4c630 0x562f5fc4ea50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b486588eeedd92 server_cookie=f8df810f4faa0e66 in_seq=0 out_seq=0 2024-04-02T17:25:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.874+0000 7fe707400640 10 client.26933 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe707400640 1 -- 172.21.15.59:0/208587214 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f5fb3a2b0 con 0x562f5fc4c630 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe707400640 10 client.26933 handle_command_reply: tid=0 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 client.26933 shutdown 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 msgr2=0x562f5fbde720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe718009710 tx=0x7fe718009f60 comp rx=0 tx=0).stop 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 shutdown_connections 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562f5fc4c630 0x562f5fc4ea50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f5fbc7af0 0x562f5fbdec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562f5fbccc60 0x562f5fbdfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 --2- 172.21.15.59:0/208587214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562f5fbc7100 0x562f5fbde720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe707400640 0 client.26933 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 >> 172.21.15.59:0/208587214 conn(0x562f5fbc2670 msgr2=0x562f5fbc6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 shutdown_connections 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 1 -- 172.21.15.59:0/208587214 wait complete. 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.921+0000 7fe72f433740 20 client.26933 trim_cache size 0 max 16384 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:25:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 msgr2=0x7fe70c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 0x7fe70c0a60e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe72002c640 tx=0x7fe720032590 comp rx=0 tx=0).stop 2024-04-02T17:25:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 msgr2=0x7fe72419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e9c0 tx=0x7fe71400ee80 comp rx=0 tx=0).stop 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 shutdown_connections 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe724107630 0x7fe7241a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe724104d00 0x7fe72419a310 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe70c0a3c20 0x7fe70c0a60e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 --2- 172.21.15.59:0/1836832138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7241023d0 0x7fe724199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 >> 172.21.15.59:0/1836832138 conn(0x7fe7240fbde0 msgr2=0x7fe724102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.922+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 shutdown_connections 2024-04-02T17:25:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:24.923+0000 7fe72b200640 1 -- 172.21.15.59:0/1836832138 wait complete. 2024-04-02T17:25:25.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 432883 3.77 MB/sec execute 532 sec latency 36.824 ms 2024-04-02T17:25:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:25 smithi154 ceph-mon[36479]: pgmap v758: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 604 KiB/s wr, 114 op/s 2024-04-02T17:25:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:25 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:25:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:25 smithi059 ceph-mon[31664]: pgmap v758: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 604 KiB/s wr, 114 op/s 2024-04-02T17:25:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:25 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:25:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:25 smithi164 ceph-mon[36499]: pgmap v758: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 604 KiB/s wr, 114 op/s 2024-04-02T17:25:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:25 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:25:26.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 433236 3.77 MB/sec execute 533 sec latency 29.693 ms 2024-04-02T17:25:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:25:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:25:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:25:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:25:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:25:26.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:25:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:25:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:25:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:25:27.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 433944 3.76 MB/sec execute 534 sec latency 35.226 ms 2024-04-02T17:25:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:27 smithi154 ceph-mon[36479]: pgmap v759: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 642 KiB/s wr, 112 op/s 2024-04-02T17:25:27.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:27 smithi059 ceph-mon[31664]: pgmap v759: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 642 KiB/s wr, 112 op/s 2024-04-02T17:25:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:27 smithi164 ceph-mon[36499]: pgmap v759: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 642 KiB/s wr, 112 op/s 2024-04-02T17:25:28.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 435483 3.77 MB/sec execute 535 sec latency 16.695 ms 2024-04-02T17:25:29.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 436545 3.78 MB/sec execute 536 sec latency 27.232 ms 2024-04-02T17:25:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:29 smithi154 ceph-mon[36479]: pgmap v760: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 891 KiB/s wr, 131 op/s 2024-04-02T17:25:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:29 smithi059 ceph-mon[31664]: pgmap v760: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 891 KiB/s wr, 131 op/s 2024-04-02T17:25:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:29 smithi164 ceph-mon[36499]: pgmap v760: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 891 KiB/s wr, 131 op/s 2024-04-02T17:25:30.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 436760 3.77 MB/sec execute 537 sec latency 29.821 ms 2024-04-02T17:25:31.113 INFO:tasks.workunit.client.0.smithi059.stdout: 1 437449 3.77 MB/sec execute 538 sec latency 44.815 ms 2024-04-02T17:25:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:31 smithi154 ceph-mon[36479]: pgmap v761: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 164 op/s 2024-04-02T17:25:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:31 smithi059 ceph-mon[31664]: pgmap v761: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 164 op/s 2024-04-02T17:25:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:31 smithi164 ceph-mon[36499]: pgmap v761: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 164 op/s 2024-04-02T17:25:32.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 438895 3.77 MB/sec execute 539 sec latency 18.733 ms 2024-04-02T17:25:33.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440070 3.79 MB/sec execute 540 sec latency 27.723 ms 2024-04-02T17:25:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:33 smithi154 ceph-mon[36479]: pgmap v762: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 864 KiB/s wr, 137 op/s 2024-04-02T17:25:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:33 smithi059 ceph-mon[31664]: pgmap v762: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 864 KiB/s wr, 137 op/s 2024-04-02T17:25:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:33 smithi164 ceph-mon[36499]: pgmap v762: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 864 KiB/s wr, 137 op/s 2024-04-02T17:25:34.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440290 3.78 MB/sec execute 541 sec latency 31.103 ms 2024-04-02T17:25:35.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440522 3.77 MB/sec execute 542 sec latency 492.103 ms 2024-04-02T17:25:35.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:35 smithi154 ceph-mon[36479]: pgmap v763: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:25:35.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:35 smithi059 ceph-mon[31664]: pgmap v763: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:25:35.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:35 smithi164 ceph-mon[36499]: pgmap v763: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-04-02T17:25:36.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440522 3.77 MB/sec execute 543 sec latency 1492.233 ms 2024-04-02T17:25:37.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440522 3.76 MB/sec execute 544 sec latency 2492.364 ms 2024-04-02T17:25:37.374 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:37 smithi164 ceph-mon[36499]: pgmap v764: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-02T17:25:37.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:37 smithi154 ceph-mon[36479]: pgmap v764: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-02T17:25:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:37 smithi059 ceph-mon[31664]: pgmap v764: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 -- 172.21.15.59:0/3387462007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34104f40 msgr2=0x7fdd341053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/3387462007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34104f40 0x7fdd341053c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009860 tx=0x7fdd2002bb70 comp rx=0 tx=0).stop 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 -- 172.21.15.59:0/3387462007 shutdown_connections 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/3387462007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34105900 0x7fdd3410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/3387462007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34104f40 0x7fdd341053c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/3387462007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34103d50 0x7fdd34104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 -- 172.21.15.59:0/3387462007 >> 172.21.15.59:0/3387462007 conn(0x7fdd340ff2c0 msgr2=0x7fdd34101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.071+0000 7fdd3ba00640 1 -- 172.21.15.59:0/3387462007 shutdown_connections 2024-04-02T17:25:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.072+0000 7fdd3ba00640 1 -- 172.21.15.59:0/3387462007 wait complete. 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.073+0000 7fdd3ba00640 1 Processor -- start 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.073+0000 7fdd3ba00640 1 -- start start 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.073+0000 7fdd3ba00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 0x7fdd34199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd3ba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34104f40 0x7fdd3419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd3ba00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd3ba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410f9d0 con 0x7fdd34104f40 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd3ba00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410f850 con 0x7fdd34103d50 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd3ba00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3410fb50 con 0x7fdd34105900 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 0x7fdd34199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:60402/0 (socket says 172.21.15.59:60402) 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 -- 172.21.15.59:0/4152012846 learned_addr learned my addr 172.21.15.59:0/4152012846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd2be00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34104f40 0x7fdd3419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 -- 172.21.15.59:0/4152012846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 msgr2=0x7fdd34199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 0x7fdd34199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 -- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34104f40 msgr2=0x7fdd3419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34104f40 0x7fdd3419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39a00640 1 -- 172.21.15.59:0/4152012846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd341a1bd0 con 0x7fdd34105900 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.074+0000 7fdd39000640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 0x7fdd34199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.075+0000 7fdd39a00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00eb10 tx=0x7fdd2c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.075+0000 7fdd32000640 1 -- 172.21.15.59:0/4152012846 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00ce20 con 0x7fdd34105900 2024-04-02T17:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.075+0000 7fdd32000640 1 -- 172.21.15.59:0/4152012846 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c005020 con 0x7fdd34105900 2024-04-02T17:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.075+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341a1ec0 con 0x7fdd34105900 2024-04-02T17:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.075+0000 7fdd32000640 1 -- 172.21.15.59:0/4152012846 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c010690 con 0x7fdd34105900 2024-04-02T17:25:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.076+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a2330 con 0x7fdd34105900 2024-04-02T17:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.079+0000 7fdd3e5ec740 1 Processor -- start 2024-04-02T17:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd32000640 1 -- 172.21.15.59:0/4152012846 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd2c005540 con 0x7fdd34105900 2024-04-02T17:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 -- start start 2024-04-02T17:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 0x55fd65dddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fd65d3bd70 con 0x55fd65dd68c0 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fd65ce42a0 con 0x55fd65dd5ed0 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.080+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fd65d37d20 con 0x55fd65ddba30 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41480/0 (socket says 172.21.15.59:41480) 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 -- 172.21.15.59:0/3469295085 learned_addr learned my addr 172.21.15.59:0/3469295085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd2be00640 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 0x55fd65dddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 -- 172.21.15.59:0/3469295085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 msgr2=0x55fd65dddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 0x55fd65dddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 -- 172.21.15.59:0/3469295085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 msgr2=0x55fd65dd62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd39000640 1 -- 172.21.15.59:0/3469295085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd65cf8840 con 0x55fd65dd68c0 2024-04-02T17:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.081+0000 7fdd2be00640 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 0x55fd65dddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd39000640 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd180114f0 tx=0x7fdd180119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaa7cd03b29447a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd32000640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 0x7fdd100a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd32000640 1 -- 172.21.15.59:0/4152012846 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdd2c0d0a60 con 0x7fdd34105900 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd2a000640 1 -- 172.21.15.59:0/3469295085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801e070 con 0x55fd65dd68c0 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd2be00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 0x7fdd100a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd2a000640 1 -- 172.21.15.59:0/3469295085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1800f680 con 0x55fd65dd68c0 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.082+0000 7fdd2a000640 1 -- 172.21.15.59:0/3469295085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18003850 con 0x55fd65dd68c0 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3469295085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 msgr2=0x55fd65ddb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd180114f0 tx=0x7fdd180119b0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd2be00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 0x7fdd100a6110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002c180 tx=0x7fdd2002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3469295085 shutdown_connections 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65ddb3c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65ddba30 0x55fd65dddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3469295085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3469295085 >> 172.21.15.59:0/3469295085 conn(0x55fd65dd1440 msgr2=0x55fd65dd1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3469295085 shutdown_connections 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.083+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3469295085 wait complete. 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.084+0000 7fdd3e5ec740 1 -- start start 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.084+0000 7fdd3e5ec740 1 Processor -- start 2024-04-02T17:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.084+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65de03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.084+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 0x55fd65de7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fd65dde9e0 con 0x55fd65dd68c0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fd65cf7660 con 0x55fd65ddba30 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fd65ddeb60 con 0x55fd65dd5ed0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd3e5ec740 10 client.0 _ll_register_callbacks cb 0x55fd65cf7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd39000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65de03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41486/0 (socket says 172.21.15.59:41486) 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 -- 172.21.15.59:0/3009451442 learned_addr learned my addr 172.21.15.59:0/3009451442 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd39a00640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 0x55fd65de7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 -- 172.21.15.59:0/3009451442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 msgr2=0x55fd65de03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65de03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 -- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 msgr2=0x55fd65de7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 0x55fd65de7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd2be00640 1 -- 172.21.15.59:0/3009451442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd65d06e80 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.085+0000 7fdd39a00640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 0x55fd65de7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd2be00640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009230 tx=0x7fdd2002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1ea2f5c441ca3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd26e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20039040 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdd2002c920 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd1800f680 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdd2002cb20 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd3e5ec740 10 client.16845 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.086+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fd65a50210 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1800f680 con 0x55fd65dd68c0 2024-04-02T17:25:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdd2003f040 con 0x55fd65dd68c0 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 10 client.16845 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 20 client.16845 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65dec250 0x55fd65dee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 4 client.16845 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd65cebcd0 con 0x55fd65dec250 2024-04-02T17:25:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.087+0000 7fdd39000640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65dec250 0x55fd65dee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.088+0000 7fdd39000640 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65dec250 0x55fd65dee6a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4d2c1997e37dee server_cookie=23e473a8473ca5d in_seq=0 out_seq=0 2024-04-02T17:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.088+0000 7fdd26e00640 10 client.16845 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd26e00640 1 -- 172.21.15.59:0/3009451442 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd65cebcd0 con 0x55fd65dec250 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd26e00640 10 client.16845 handle_command_reply: tid=0 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 client.16845 shutdown 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 msgr2=0x55fd65de08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009230 tx=0x7fdd2002bb70 comp rx=0 tx=0).stop 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 shutdown_connections 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65dec250 0x55fd65dee6a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65dd68c0 0x55fd65de08f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65ddba30 0x55fd65de7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/3009451442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65de03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd26e00640 0 client.16845 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 >> 172.21.15.59:0/3009451442 conn(0x55fd65dd1440 msgr2=0x55fd65dd0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 shutdown_connections 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.089+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/3009451442 wait complete. 2024-04-02T17:25:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.090+0000 7fdd3e5ec740 20 client.16845 trim_cache size 0 max 16384 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 Processor -- start 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 -- start start 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 0x55fd65de78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ab210 con 0x55fd65ddba30 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341acdb0 con 0x55fd65dd68c0 2024-04-02T17:25:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ab8e0 con 0x55fd65dd5ed0 2024-04-02T17:25:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41492/0 (socket says 172.21.15.59:41492) 2024-04-02T17:25:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd39a00640 1 -- 172.21.15.59:0/945528603 learned_addr learned my addr 172.21.15.59:0/945528603 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd2be00640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 0x55fd65de78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.107+0000 7fdd39000640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 -- 172.21.15.59:0/945528603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 msgr2=0x55fd65dd4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 -- 172.21.15.59:0/945528603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 msgr2=0x55fd65df2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 -- 172.21.15.59:0/945528603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd65b1c620 con 0x55fd65dd68c0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd39a00640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd2be00640 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 0x55fd65de78f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd200349e0 tx=0x7fdd20009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98cf03481abbc51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd29600640 1 -- 172.21.15.59:0/945528603 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20039070 con 0x55fd65dd68c0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd29600640 1 -- 172.21.15.59:0/945528603 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd20053070 con 0x55fd65dd68c0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.108+0000 7fdd29600640 1 -- 172.21.15.59:0/945528603 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd200329e0 con 0x55fd65dd68c0 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/945528603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 msgr2=0x55fd65de78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 0x55fd65de78f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd200349e0 tx=0x7fdd20009490 comp rx=0 tx=0).stop 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/945528603 shutdown_connections 2024-04-02T17:25:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65df2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd68c0 0x55fd65de78f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/945528603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd5ed0 0x55fd65dd4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/945528603 >> 172.21.15.59:0/945528603 conn(0x55fd65dd1440 msgr2=0x55fd65dd46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/945528603 shutdown_connections 2024-04-02T17:25:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.109+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/945528603 wait complete. 2024-04-02T17:25:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 -- start start 2024-04-02T17:25:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 Processor -- start 2024-04-02T17:25:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65de5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd68c0 0x55fd65de5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ac6e0 con 0x55fd65ddba30 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341abfb0 con 0x55fd65dd5ed0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341a91f0 con 0x55fd65dd68c0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.110+0000 7fdd3e5ec740 10 client.0 _ll_register_callbacks cb 0x7fdd20034770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41496/0 (socket says 172.21.15.59:41496) 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 -- 172.21.15.59:0/340121552 learned_addr learned my addr 172.21.15.59:0/340121552 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39000640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65de5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd2be00640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd68c0 0x55fd65de5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 -- 172.21.15.59:0/340121552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd68c0 msgr2=0x55fd65de5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd68c0 0x55fd65de5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 -- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 msgr2=0x55fd65de5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65de5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 -- 172.21.15.59:0/340121552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd65de6c70 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39000640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65de5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd39a00640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c099650 tx=0x7fdd2c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611be8c05a2d335a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.111+0000 7fdd26e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0bb060 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fdd2c0a5460 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd20053070 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdd2c0a1030 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 10 client.16851 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fd65e52a10 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd20053070 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdd2c098af0 con 0x55fd65ddba30 2024-04-02T17:25:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 10 client.16851 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 20 client.16851 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65e533f0 0x55fd65e55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 4 client.16851 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.112+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd65d492b0 con 0x55fd65e533f0 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.113+0000 7fdd39000640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65e533f0 0x55fd65e55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.113+0000 7fdd26e00640 10 client.16851 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.113+0000 7fdd39000640 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65e533f0 0x55fd65e55810 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d03d5ddf98d1da5 server_cookie=38084dda723e0d6f in_seq=0 out_seq=0 2024-04-02T17:25:38.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440522 3.75 MB/sec execute 545 sec latency 3492.434 ms 2024-04-02T17:25:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd26e00640 1 -- 172.21.15.59:0/340121552 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd65d492b0 con 0x55fd65e533f0 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd26e00640 10 client.16851 handle_command_reply: tid=0 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 client.16851 shutdown 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 msgr2=0x55fd65de6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c099650 tx=0x7fdd2c0051d0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 shutdown_connections 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fd65e533f0 0x55fd65e55810 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd65ddba30 0x55fd65de6680 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd65dd68c0 0x55fd65de5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 --2- 172.21.15.59:0/340121552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fd65dd5ed0 0x55fd65de5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd26e00640 0 client.16851 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 >> 172.21.15.59:0/340121552 conn(0x55fd65dd1440 msgr2=0x55fd65dd4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 shutdown_connections 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.157+0000 7fdd3e5ec740 1 -- 172.21.15.59:0/340121552 wait complete. 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3e5ec740 20 client.16851 trim_cache size 0 max 16384 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:25:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 msgr2=0x7fdd100a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 0x7fdd100a6110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002c180 tx=0x7fdd2002e040 comp rx=0 tx=0).stop 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 msgr2=0x7fdd341a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00eb10 tx=0x7fdd2c00ef10 comp rx=0 tx=0).stop 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 shutdown_connections 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdd100a3c50 0x7fdd100a6110 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd34104f40 0x7fdd3419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd34105900 0x7fdd341a1560 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.158+0000 7fdd3ba00640 1 --2- 172.21.15.59:0/4152012846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34103d50 0x7fdd34199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.159+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 >> 172.21.15.59:0/4152012846 conn(0x7fdd340ff2c0 msgr2=0x7fdd34108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.159+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 shutdown_connections 2024-04-02T17:25:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:38.159+0000 7fdd3ba00640 1 -- 172.21.15.59:0/4152012846 wait complete. 2024-04-02T17:25:39.114 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440578 3.75 MB/sec execute 546 sec latency 4100.354 ms 2024-04-02T17:25:39.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:39 smithi154 ceph-mon[36479]: pgmap v765: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-04-02T17:25:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:39 smithi059 ceph-mon[31664]: pgmap v765: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-04-02T17:25:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:39 smithi164 ceph-mon[36499]: pgmap v765: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-04-02T17:25:40.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440578 3.74 MB/sec execute 547 sec latency 1297.625 ms 2024-04-02T17:25:41.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440578 3.73 MB/sec execute 548 sec latency 2297.778 ms 2024-04-02T17:25:41.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:41 smithi154 ceph-mon[36479]: pgmap v766: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 811 KiB/s wr, 139 op/s 2024-04-02T17:25:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:41 smithi059 ceph-mon[31664]: pgmap v766: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 811 KiB/s wr, 139 op/s 2024-04-02T17:25:41.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:41 smithi164 ceph-mon[36499]: pgmap v766: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 811 KiB/s wr, 139 op/s 2024-04-02T17:25:42.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440578 3.73 MB/sec execute 549 sec latency 3297.942 ms 2024-04-02T17:25:43.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440578 3.72 MB/sec execute 550 sec latency 4298.074 ms 2024-04-02T17:25:43.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:43 smithi154 ceph-mon[36479]: pgmap v767: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 603 KiB/s wr, 96 op/s 2024-04-02T17:25:43.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:43 smithi059 ceph-mon[31664]: pgmap v767: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 603 KiB/s wr, 96 op/s 2024-04-02T17:25:43.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:43 smithi164 ceph-mon[36499]: pgmap v767: 97 pgs: 97 active+clean; 112 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 603 KiB/s wr, 96 op/s 2024-04-02T17:25:44.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 440898 3.72 MB/sec execute 551 sec latency 4906.367 ms 2024-04-02T17:25:45.115 INFO:tasks.workunit.client.0.smithi059.stdout: 1 442432 3.72 MB/sec execute 552 sec latency 18.317 ms 2024-04-02T17:25:45.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:45 smithi154 ceph-mon[36479]: pgmap v768: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 663 KiB/s wr, 107 op/s 2024-04-02T17:25:45.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:45 smithi059 ceph-mon[31664]: pgmap v768: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 663 KiB/s wr, 107 op/s 2024-04-02T17:25:45.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:45 smithi164 ceph-mon[36499]: pgmap v768: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 663 KiB/s wr, 107 op/s 2024-04-02T17:25:46.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 443641 3.73 MB/sec execute 553 sec latency 22.550 ms 2024-04-02T17:25:47.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 443885 3.73 MB/sec execute 554 sec latency 26.108 ms 2024-04-02T17:25:47.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:47 smithi154 ceph-mon[36479]: pgmap v769: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 469 KiB/s wr, 69 op/s 2024-04-02T17:25:47.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:47 smithi059 ceph-mon[31664]: pgmap v769: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 469 KiB/s wr, 69 op/s 2024-04-02T17:25:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:47 smithi164 ceph-mon[36499]: pgmap v769: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 469 KiB/s wr, 69 op/s 2024-04-02T17:25:48.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 444127 3.72 MB/sec execute 555 sec latency 677.637 ms 2024-04-02T17:25:49.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 444463 3.72 MB/sec execute 556 sec latency 1280.774 ms 2024-04-02T17:25:49.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:49 smithi154 ceph-mon[36479]: pgmap v770: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 454 KiB/s wr, 72 op/s 2024-04-02T17:25:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:49 smithi059 ceph-mon[31664]: pgmap v770: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 454 KiB/s wr, 72 op/s 2024-04-02T17:25:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:49 smithi164 ceph-mon[36499]: pgmap v770: 97 pgs: 97 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 454 KiB/s wr, 72 op/s 2024-04-02T17:25:50.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 446153 3.73 MB/sec execute 557 sec latency 14.994 ms 2024-04-02T17:25:51.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 447195 3.73 MB/sec execute 558 sec latency 23.894 ms 2024-04-02T17:25:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 -- 172.21.15.59:0/3842990798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904104f40 msgr2=0x7f09041053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 --2- 172.21.15.59:0/3842990798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904104f40 0x7f09041053c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009900 tx=0x7f08ec02bc30 comp rx=0 tx=0).stop 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 -- 172.21.15.59:0/3842990798 shutdown_connections 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 --2- 172.21.15.59:0/3842990798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904105900 0x7f090410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 --2- 172.21.15.59:0/3842990798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904104f40 0x7f09041053c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 --2- 172.21.15.59:0/3842990798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904103d50 0x7f0904104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 -- 172.21.15.59:0/3842990798 >> 172.21.15.59:0/3842990798 conn(0x7f09040ff2c0 msgr2=0x7f0904101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 -- 172.21.15.59:0/3842990798 shutdown_connections 2024-04-02T17:25:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.321+0000 7f0909200640 1 -- 172.21.15.59:0/3842990798 wait complete. 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.322+0000 7f0909200640 1 Processor -- start 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.322+0000 7f0909200640 1 -- start start 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904103d50 0x7f0904199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 0x7f090419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f090410fa00 con 0x7f0904103d50 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f090410f880 con 0x7f0904105900 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0909200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f090410fb80 con 0x7f0904104f40 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0902800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0901e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904103d50 0x7f0904199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0902800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57594/0 (socket says 172.21.15.59:57594) 2024-04-02T17:25:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0902800640 1 -- 172.21.15.59:0/2250721751 learned_addr learned my addr 172.21.15.59:0/2250721751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.323+0000 7f0901400640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 0x7f090419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 -- 172.21.15.59:0/2250721751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 msgr2=0x7f090419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 0x7f090419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 -- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904103d50 msgr2=0x7f0904199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904103d50 0x7f0904199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 -- 172.21.15.59:0/2250721751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09041a1bb0 con 0x7f0904105900 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0901400640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 0x7f090419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0902800640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00eb10 tx=0x7f08fc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f08faa00640 1 -- 172.21.15.59:0/2250721751 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc004b20 con 0x7f0904105900 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f08faa00640 1 -- 172.21.15.59:0/2250721751 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc004cc0 con 0x7f0904105900 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09041a1ea0 con 0x7f0904105900 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f08faa00640 1 -- 172.21.15.59:0/2250721751 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc010720 con 0x7f0904105900 2024-04-02T17:25:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.324+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a2310 con 0x7f0904105900 2024-04-02T17:25:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.328+0000 7f08faa00640 1 -- 172.21.15.59:0/2250721751 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f08fc005020 con 0x7f0904105900 2024-04-02T17:25:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.328+0000 7f090bfa0740 1 Processor -- start 2024-04-02T17:25:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 -- start start 2024-04-02T17:25:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73bed0 0x55da3d73c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d7413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da3d6a1d70 con 0x55da3d73c8c0 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da3d64a2a0 con 0x55da3d741a30 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f090bfa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da3d69dd20 con 0x55da3d73bed0 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f0901e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d7413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f0901400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f0901400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57606/0 (socket says 172.21.15.59:57606) 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f0901400640 1 -- 172.21.15.59:0/18972272 learned_addr learned my addr 172.21.15.59:0/18972272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.329+0000 7f0901e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d7413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34382/0 (socket says 172.21.15.59:34382) 2024-04-02T17:25:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73bed0 0x55da3d73c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 -- 172.21.15.59:0/18972272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73bed0 msgr2=0x55da3d73c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73bed0 0x55da3d73c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 -- 172.21.15.59:0/18972272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 msgr2=0x55da3d7413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d7413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 -- 172.21.15.59:0/18972272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3d65e840 con 0x55da3d741a30 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f08faa00640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 0x7f08e80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f0901400640 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000c00 tx=0x7f08ec005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12912c3ba7999065 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.330+0000 7f08faa00640 1 -- 172.21.15.59:0/2250721751 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f08fc0d0ae0 con 0x7f0904105900 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.331+0000 7f0901e00640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 0x7f08e80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.331+0000 7f08f3400640 1 -- 172.21.15.59:0/18972272 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec049070 con 0x55da3d741a30 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.331+0000 7f08f3400640 1 -- 172.21.15.59:0/18972272 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec005530 con 0x55da3d741a30 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.331+0000 7f08f3400640 1 -- 172.21.15.59:0/18972272 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004a70 con 0x55da3d741a30 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- 172.21.15.59:0/18972272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 msgr2=0x55da3d743ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f0901e00640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 0x7f08e80a6110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55da3d73ccc0 tx=0x7f08f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000c00 tx=0x7f08ec005b50 comp rx=0 tx=0).stop 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- 172.21.15.59:0/18972272 shutdown_connections 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d7413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d741a30 0x55da3d743ec0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 --2- 172.21.15.59:0/18972272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73bed0 0x55da3d73c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- 172.21.15.59:0/18972272 >> 172.21.15.59:0/18972272 conn(0x55da3d737440 msgr2=0x55da3d737840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- 172.21.15.59:0/18972272 shutdown_connections 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- 172.21.15.59:0/18972272 wait complete. 2024-04-02T17:25:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.332+0000 7f090bfa0740 1 -- start start 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 Processor -- start 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d7463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 0x55da3d746920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d741a30 0x55da3d74d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da3d744980 con 0x55da3d741a30 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da3d65d660 con 0x55da3d73bed0 2024-04-02T17:25:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da3d744b00 con 0x55da3d73c8c0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.333+0000 7f090bfa0740 10 client.0 _ll_register_callbacks cb 0x55da3d65d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d7463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 0x55da3d746920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d7463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57616/0 (socket says 172.21.15.59:57616) 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901e00640 1 -- 172.21.15.59:0/4169825886 learned_addr learned my addr 172.21.15.59:0/4169825886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0902800640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d741a30 0x55da3d74d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 -- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 msgr2=0x55da3d7463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d7463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 -- 172.21.15.59:0/4169825886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d741a30 msgr2=0x55da3d74d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d741a30 0x55da3d74d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 -- 172.21.15.59:0/4169825886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3d66ce80 con 0x55da3d73c8c0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.334+0000 7f0901400640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 0x55da3d746920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08ec036040 tx=0x7f08ec004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8e6711f3ead1db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f08e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec049040 con 0x55da3d73c8c0 2024-04-02T17:25:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec005530 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f08ec0035f0 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f08ec0037f0 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f090bfa0740 10 client.26968 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.335+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da3d3b6210 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.336+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec005530 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.336+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f08ec04c070 con 0x55da3d73c8c0 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.336+0000 7f090bfa0740 10 client.26968 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.336+0000 7f090bfa0740 10 client.26968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f090bfa0740 20 client.26968 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d751e50 0x55da3d754240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f090bfa0740 4 client.26968 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3d651cd0 con 0x55da3d751e50 2024-04-02T17:25:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f0901e00640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d751e50 0x55da3d754240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f0901e00640 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d751e50 0x55da3d754240 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b712431f45e618 server_cookie=fae235aec6dfe511 in_seq=0 out_seq=0 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.337+0000 7f08e7e00640 10 client.26968 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f08e7e00640 1 -- 172.21.15.59:0/4169825886 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da3d651cd0 con 0x55da3d751e50 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f08e7e00640 10 client.26968 handle_command_reply: tid=0 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f090bfa0740 1 client.26968 shutdown 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 msgr2=0x55da3d746920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 0x55da3d746920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08ec036040 tx=0x7f08ec004510 comp rx=0 tx=0).stop 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 shutdown_connections 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.338+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d751e50 0x55da3d754240 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d741a30 0x55da3d74d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d73c8c0 0x55da3d746920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 --2- 172.21.15.59:0/4169825886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d7463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f08e7e00640 0 client.26968 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 >> 172.21.15.59:0/4169825886 conn(0x55da3d737440 msgr2=0x55da3d736640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 shutdown_connections 2024-04-02T17:25:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 1 -- 172.21.15.59:0/4169825886 wait complete. 2024-04-02T17:25:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.339+0000 7f090bfa0740 20 client.26968 trim_cache size 0 max 16384 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 Processor -- start 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 -- start start 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d73adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d758970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab290 con 0x55da3d73c8c0 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ace30 con 0x55da3d73bed0 2024-04-02T17:25:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f090bfa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab960 con 0x55da3d741a30 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f0901400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f0901e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d73adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f0901400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34402/0 (socket says 172.21.15.59:34402) 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f0901400640 1 -- 172.21.15.59:0/817684517 learned_addr learned my addr 172.21.15.59:0/817684517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.357+0000 7f0902800640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d758970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 -- 172.21.15.59:0/817684517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 msgr2=0x55da3d758970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d758970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 -- 172.21.15.59:0/817684517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 msgr2=0x55da3d73adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d73adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 -- 172.21.15.59:0/817684517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3d482620 con 0x55da3d73c8c0 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0902800640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d758970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901e00640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d73adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f0901400640 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f08ec043470 tx=0x7f08ec0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f46a01d2e59c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f08f2a00640 1 -- 172.21.15.59:0/817684517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec049040 con 0x55da3d73c8c0 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f08f2a00640 1 -- 172.21.15.59:0/817684517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec057c00 con 0x55da3d73c8c0 2024-04-02T17:25:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.358+0000 7f08f2a00640 1 -- 172.21.15.59:0/817684517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003e30 con 0x55da3d73c8c0 2024-04-02T17:25:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 -- 172.21.15.59:0/817684517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 msgr2=0x55da3d74d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f08ec043470 tx=0x7f08ec0043d0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 -- 172.21.15.59:0/817684517 shutdown_connections 2024-04-02T17:25:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d73c8c0 0x55da3d74d920 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d758970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 --2- 172.21.15.59:0/817684517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d73adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.359+0000 7f090bfa0740 1 -- 172.21.15.59:0/817684517 >> 172.21.15.59:0/817684517 conn(0x55da3d737440 msgr2=0x55da3d73a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.360+0000 7f090bfa0740 1 -- 172.21.15.59:0/817684517 shutdown_connections 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.360+0000 7f090bfa0740 1 -- 172.21.15.59:0/817684517 wait complete. 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.360+0000 7f090bfa0740 1 -- start start 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.360+0000 7f090bfa0740 1 Processor -- start 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d74adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d74c340 0x55da3d74c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ac760 con 0x55da3d74c340 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ac030 con 0x55da3d73bed0 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09041a9270 con 0x55da3d741a30 2024-04-02T17:25:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f090bfa0740 10 client.0 _ll_register_callbacks cb 0x7f08f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f0902800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d74c340 0x55da3d74c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f0901400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f0901e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d74adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f0901400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58332/0 (socket says 172.21.15.59:58332) 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.361+0000 7f0901400640 1 -- 172.21.15.59:0/2884501364 learned_addr learned my addr 172.21.15.59:0/2884501364 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 -- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 msgr2=0x55da3d74adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d74adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 -- 172.21.15.59:0/2884501364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d74c340 msgr2=0x55da3d74c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d74c340 0x55da3d74c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 -- 172.21.15.59:0/2884501364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3d74cd00 con 0x55da3d741a30 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f0901400640 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08ec057b80 tx=0x7f08ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2313c33a409dc40b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f08e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec05c070 con 0x55da3d741a30 2024-04-02T17:25:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f08ec057680 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec057c00 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.362+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f08ec049050 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 10 client.26974 fetch_fsmap learned FSMap version 37 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da3d7b83a0 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec057c00 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f08ec04c070 con 0x55da3d741a30 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 10 client.26974 fetch_fsmap finished waiting for FSMap version 37 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 20 client.26974 populate_metadata read hostname 'smithi059' 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d7b8ab0 0x55da3d7baed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 4 client.26974 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da3d6af2b0 con 0x55da3d7b8ab0 2024-04-02T17:25:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.363+0000 7f0901e00640 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d7b8ab0 0x55da3d7baed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:25:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.364+0000 7f0901e00640 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d7b8ab0 0x55da3d7baed0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c592e6914b2fb8 server_cookie=5a796bf5c9597a83 in_seq=0 out_seq=0 2024-04-02T17:25:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.364+0000 7f08e7e00640 10 client.26974 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.393+0000 7f08e7e00640 1 -- 172.21.15.59:0/2884501364 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da3d6af2b0 con 0x55da3d7b8ab0 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.393+0000 7f08e7e00640 10 client.26974 handle_command_reply: tid=0 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 client.26974 shutdown 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 msgr2=0x55da3d74b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08ec057b80 tx=0x7f08ec000bc0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 shutdown_connections 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da3d7b8ab0 0x55da3d7baed0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da3d74c340 0x55da3d74c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f08e7e00640 0 client.26974 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:25:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da3d741a30 0x55da3d74b330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 --2- 172.21.15.59:0/2884501364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da3d73bed0 0x55da3d74adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 >> 172.21.15.59:0/2884501364 conn(0x55da3d737440 msgr2=0x55da3d73af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 shutdown_connections 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.394+0000 7f090bfa0740 1 -- 172.21.15.59:0/2884501364 wait complete. 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.395+0000 7f090bfa0740 20 client.26974 trim_cache size 0 max 16384 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:25:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:25:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.395+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 msgr2=0x7f08e80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.395+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 0x7f08e80a6110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55da3d73ccc0 tx=0x7f08f4005f90 comp rx=0 tx=0).stop 2024-04-02T17:25:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.395+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 msgr2=0x7f09041a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:25:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00eb10 tx=0x7f08fc00c2d0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 shutdown_connections 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0904105900 0x7f09041a1540 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0904104f40 0x7f090419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f08e80a3c50 0x7f08e80a6110 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 --2- 172.21.15.59:0/2250721751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0904103d50 0x7f0904199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 >> 172.21.15.59:0/2250721751 conn(0x7f09040ff2c0 msgr2=0x7f0904108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.396+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 shutdown_connections 2024-04-02T17:25:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:25:51.397+0000 7f0909200640 1 -- 172.21.15.59:0/2250721751 wait complete. 2024-04-02T17:25:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:51 smithi059 ceph-mon[31664]: pgmap v771: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 519 KiB/s wr, 85 op/s 2024-04-02T17:25:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:51 smithi164 ceph-mon[36499]: pgmap v771: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 519 KiB/s wr, 85 op/s 2024-04-02T17:25:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:51 smithi154 ceph-mon[36479]: pgmap v771: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 519 KiB/s wr, 85 op/s 2024-04-02T17:25:52.116 INFO:tasks.workunit.client.0.smithi059.stdout: 1 447424 3.73 MB/sec execute 559 sec latency 110.147 ms 2024-04-02T17:25:53.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 448235 3.73 MB/sec execute 560 sec latency 19.655 ms 2024-04-02T17:25:53.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:53 smithi059 ceph-mon[31664]: pgmap v772: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 556 KiB/s wr, 87 op/s 2024-04-02T17:25:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:53 smithi164 ceph-mon[36499]: pgmap v772: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 556 KiB/s wr, 87 op/s 2024-04-02T17:25:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:53 smithi154 ceph-mon[36479]: pgmap v772: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 556 KiB/s wr, 87 op/s 2024-04-02T17:25:54.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 449968 3.74 MB/sec execute 561 sec latency 15.405 ms 2024-04-02T17:25:55.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 450819 3.74 MB/sec execute 562 sec latency 29.765 ms 2024-04-02T17:25:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:55 smithi059 ceph-mon[31664]: pgmap v773: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 797 KiB/s wr, 117 op/s 2024-04-02T17:25:55.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:55 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:25:55.612 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:55 smithi164 ceph-mon[36499]: pgmap v773: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 797 KiB/s wr, 117 op/s 2024-04-02T17:25:55.612 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:55 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:25:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:55 smithi154 ceph-mon[36479]: pgmap v773: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 797 KiB/s wr, 117 op/s 2024-04-02T17:25:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:55 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:25:56.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 451285 3.74 MB/sec execute 563 sec latency 24.758 ms 2024-04-02T17:25:57.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 452890 3.74 MB/sec execute 564 sec latency 30.383 ms 2024-04-02T17:25:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:57 smithi059 ceph-mon[31664]: pgmap v774: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 819 KiB/s wr, 136 op/s 2024-04-02T17:25:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:57 smithi164 ceph-mon[36499]: pgmap v774: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 819 KiB/s wr, 136 op/s 2024-04-02T17:25:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:57 smithi154 ceph-mon[36479]: pgmap v774: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 819 KiB/s wr, 136 op/s 2024-04-02T17:25:58.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 454178 3.75 MB/sec execute 565 sec latency 18.177 ms 2024-04-02T17:25:59.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 454467 3.75 MB/sec execute 566 sec latency 27.257 ms 2024-04-02T17:25:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:25:59 smithi059 ceph-mon[31664]: pgmap v775: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-04-02T17:25:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:25:59 smithi164 ceph-mon[36499]: pgmap v775: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-04-02T17:25:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:25:59 smithi154 ceph-mon[36479]: pgmap v775: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-04-02T17:26:00.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 455130 3.75 MB/sec execute 567 sec latency 21.749 ms 2024-04-02T17:26:01.117 INFO:tasks.workunit.client.0.smithi059.stdout: 1 456471 3.75 MB/sec execute 568 sec latency 203.459 ms 2024-04-02T17:26:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:01 smithi059 ceph-mon[31664]: pgmap v776: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:26:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:01 smithi164 ceph-mon[36499]: pgmap v776: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:26:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:01 smithi154 ceph-mon[36479]: pgmap v776: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T17:26:02.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 457781 3.76 MB/sec execute 569 sec latency 15.334 ms 2024-04-02T17:26:03.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 458021 3.76 MB/sec execute 570 sec latency 25.710 ms 2024-04-02T17:26:03.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:03 smithi154 ceph-mon[36479]: pgmap v777: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:26:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:03 smithi059 ceph-mon[31664]: pgmap v777: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:26:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:03 smithi164 ceph-mon[36499]: pgmap v777: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T17:26:04.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 458345 3.75 MB/sec execute 571 sec latency 74.047 ms 2024-04-02T17:26:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 -- 172.21.15.59:0/3502522610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 msgr2=0x7ff94c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 --2- 172.21.15.59:0/3502522610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c100e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009900 tx=0x7ff93c02bc40 comp rx=0 tx=0).stop 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 -- 172.21.15.59:0/3502522610 shutdown_connections 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 --2- 172.21.15.59:0/3502522610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 --2- 172.21.15.59:0/3502522610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 --2- 172.21.15.59:0/3502522610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c100e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 -- 172.21.15.59:0/3502522610 >> 172.21.15.59:0/3502522610 conn(0x7ff94c0fbf90 msgr2=0x7ff94c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 -- 172.21.15.59:0/3502522610 shutdown_connections 2024-04-02T17:26:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.547+0000 7ff953a00640 1 -- 172.21.15.59:0/3502522610 wait complete. 2024-04-02T17:26:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.548+0000 7ff953a00640 1 Processor -- start 2024-04-02T17:26:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.548+0000 7ff953a00640 1 -- start start 2024-04-02T17:26:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c10c670 con 0x7ff94c1025d0 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c10c4f0 con 0x7ff94c101c10 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff953a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c10c7f0 con 0x7ff94c100a20 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff943e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff951000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff943e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43654/0 (socket says 172.21.15.59:43654) 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff943e00640 1 -- 172.21.15.59:0/4203962836 learned_addr learned my addr 172.21.15.59:0/4203962836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.549+0000 7ff951a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 -- 172.21.15.59:0/4203962836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 msgr2=0x7ff94c102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 -- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 msgr2=0x7ff94c103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 -- 172.21.15.59:0/4203962836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c103df0 con 0x7ff94c100a20 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff951000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c104960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02c180 tx=0x7ff93c02c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff94a000640 1 -- 172.21.15.59:0/4203962836 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c041070 con 0x7ff94c100a20 2024-04-02T17:26:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff94a000640 1 -- 172.21.15.59:0/4203962836 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c004450 con 0x7ff94c100a20 2024-04-02T17:26:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff94a000640 1 -- 172.21.15.59:0/4203962836 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c034440 con 0x7ff94c100a20 2024-04-02T17:26:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.550+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c1a1ec0 con 0x7ff94c100a20 2024-04-02T17:26:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.551+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1a2320 con 0x7ff94c100a20 2024-04-02T17:26:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.555+0000 7ff9567a0740 1 Processor -- start 2024-04-02T17:26:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.555+0000 7ff94a000640 1 -- 172.21.15.59:0/4203962836 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff93c044070 con 0x7ff94c100a20 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.555+0000 7ff9567a0740 1 -- start start 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.555+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90aeeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff9567a0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac90a4cd70 con 0x55ac90ae6ed0 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff9567a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac909f52a0 con 0x55ac90ae78c0 2024-04-02T17:26:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff9567a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac90a48d20 con 0x55ac90aeca30 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52688/0 (socket says 172.21.15.59:52688) 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 -- 172.21.15.59:0/3701032380 learned_addr learned my addr 172.21.15.59:0/3701032380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951000640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43666/0 (socket says 172.21.15.59:43666) 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff943e00640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90aeeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 -- 172.21.15.59:0/3701032380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 msgr2=0x55ac90aeeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90aeeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 -- 172.21.15.59:0/3701032380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 msgr2=0x55ac90aec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951a00640 1 -- 172.21.15.59:0/3701032380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac90a09840 con 0x55ac90ae6ed0 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.556+0000 7ff951000640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff951a00640 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff944009fb0 tx=0x7ff944010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd7895cc19bcf50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff942000640 1 -- 172.21.15.59:0/3701032380 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944016070 con 0x55ac90ae6ed0 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff94a000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 0x7ff9300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff942000640 1 -- 172.21.15.59:0/3701032380 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff944012bf0 con 0x55ac90ae6ed0 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff942000640 1 -- 172.21.15.59:0/3701032380 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94401bcc0 con 0x55ac90ae6ed0 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff94a000640 1 -- 172.21.15.59:0/4203962836 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff93c0f2060 con 0x7ff94c100a20 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.557+0000 7ff951000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 0x7ff9300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 -- 172.21.15.59:0/3701032380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 msgr2=0x55ac90ae72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff944009fb0 tx=0x7ff944010800 comp rx=0 tx=0).stop 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 -- 172.21.15.59:0/3701032380 shutdown_connections 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90aeeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90aec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3701032380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae72f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 -- 172.21.15.59:0/3701032380 >> 172.21.15.59:0/3701032380 conn(0x55ac90ae2440 msgr2=0x55ac90ae2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 -- 172.21.15.59:0/3701032380 shutdown_connections 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff9567a0740 1 -- 172.21.15.59:0/3701032380 wait complete. 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.558+0000 7ff951000640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 0x7ff9300a6190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0bdf80 tx=0x7ff93c0f5aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 -- start start 2024-04-02T17:26:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 Processor -- start 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 0x55ac90af8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac90aef9e0 con 0x55ac90aeca30 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ac90a08660 con 0x55ac90ae78c0 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ac90aefb60 con 0x55ac90ae6ed0 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.559+0000 7ff9567a0740 10 client.0 _ll_register_callbacks cb 0x55ac90a08050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43678/0 (socket says 172.21.15.59:43678) 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 -- 172.21.15.59:0/3595053268 learned_addr learned my addr 172.21.15.59:0/3595053268 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff951a00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff951a00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 0x55ac90af8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 -- 172.21.15.59:0/3595053268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 msgr2=0x55ac90af13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 -- 172.21.15.59:0/3595053268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 msgr2=0x55ac90af8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 0x55ac90af8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 -- 172.21.15.59:0/3595053268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac90a17e80 con 0x55ac90ae78c0 2024-04-02T17:26:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff951a00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 0x55ac90af8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff943e00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff9340126b0 tx=0x7ff934012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f44c6177b400d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.560+0000 7ff93aa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934004030 con 0x55ac90ae78c0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff9340041d0 con 0x55ac90ae78c0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff944012bf0 con 0x55ac90ae78c0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff9340043d0 con 0x55ac90ae78c0 2024-04-02T17:26:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff9567a0740 10 client.26981 fetch_fsmap learned FSMap version 38 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac90761210 con 0x55ac90ae78c0 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff944012bf0 con 0x55ac90ae78c0 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.561+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff934025050 con 0x55ac90ae78c0 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 10 client.26981 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 10 client.26981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 20 client.26981 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90afd260 0x55ac90aff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 4 client.26981 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:26:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac909fccd0 con 0x55ac90afd260 2024-04-02T17:26:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.562+0000 7ff951a00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90afd260 0x55ac90aff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.563+0000 7ff951a00640 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90afd260 0x55ac90aff6c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb04cf11055c5af0 server_cookie=60704802554f81be in_seq=0 out_seq=0 2024-04-02T17:26:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.563+0000 7ff93aa00640 10 client.26981 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff93aa00640 1 -- 172.21.15.59:0/3595053268 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac909fccd0 con 0x55ac90afd260 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff93aa00640 10 client.26981 handle_command_reply: tid=0 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 client.26981 shutdown 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 msgr2=0x55ac90af1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff9340126b0 tx=0x7ff934012b70 comp rx=0 tx=0).stop 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 shutdown_connections 2024-04-02T17:26:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90afd260 0x55ac90aff6c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90aeca30 0x55ac90af8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af1920 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3595053268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.564+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 >> 172.21.15.59:0/3595053268 conn(0x55ac90ae2440 msgr2=0x55ac90ae1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.565+0000 7ff93aa00640 0 client.26981 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.565+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 shutdown_connections 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.565+0000 7ff9567a0740 1 -- 172.21.15.59:0/3595053268 wait complete. 2024-04-02T17:26:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.565+0000 7ff9567a0740 20 client.26981 trim_cache size 0 max 16384 2024-04-02T17:26:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 Processor -- start 2024-04-02T17:26:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 -- start start 2024-04-02T17:26:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90ae12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ab310 con 0x55ac90ae6ed0 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ace80 con 0x55ac90ae78c0 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.581+0000 7ff9567a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ab9e0 con 0x55ac90aeca30 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52702/0 (socket says 172.21.15.59:52702) 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 -- 172.21.15.59:0/3406359791 learned_addr learned my addr 172.21.15.59:0/3406359791 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff943e00640 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 -- 172.21.15.59:0/3406359791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 msgr2=0x55ac90ae12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90ae12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 -- 172.21.15.59:0/3406359791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 msgr2=0x55ac90af8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.582+0000 7ff951a00640 1 -- 172.21.15.59:0/3406359791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9082d620 con 0x55ac90ae6ed0 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.583+0000 7ff951a00640 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff944002230 tx=0x7ff944010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38abb968418824a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.583+0000 7ff941600640 1 -- 172.21.15.59:0/3406359791 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944016040 con 0x55ac90ae6ed0 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.583+0000 7ff941600640 1 -- 172.21.15.59:0/3406359791 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94402c070 con 0x55ac90ae6ed0 2024-04-02T17:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.583+0000 7ff941600640 1 -- 172.21.15.59:0/3406359791 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944012770 con 0x55ac90ae6ed0 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- 172.21.15.59:0/3406359791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 msgr2=0x55ac90ae0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff944002230 tx=0x7ff944010f70 comp rx=0 tx=0).stop 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- 172.21.15.59:0/3406359791 shutdown_connections 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90aeca30 0x55ac90ae12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90ae78c0 0x55ac90af8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 --2- 172.21.15.59:0/3406359791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae6ed0 0x55ac90ae0db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- 172.21.15.59:0/3406359791 >> 172.21.15.59:0/3406359791 conn(0x55ac90ae2440 msgr2=0x55ac90ae8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- 172.21.15.59:0/3406359791 shutdown_connections 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- 172.21.15.59:0/3406359791 wait complete. 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 -- start start 2024-04-02T17:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.584+0000 7ff9567a0740 1 Processor -- start 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 0x55ac90af64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90aeca30 0x55ac90af7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ac7b0 con 0x55ac90ae78c0 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ac0b0 con 0x55ac90aeca30 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1a92f0 con 0x55ac90ae6ed0 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff9567a0740 10 client.0 _ll_register_callbacks cb 0x7ff934009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff951a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff943e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 0x55ac90af64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff951a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56000/0 (socket says 172.21.15.59:56000) 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff951a00640 1 -- 172.21.15.59:0/1792521402 learned_addr learned my addr 172.21.15.59:0/1792521402 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.585+0000 7ff951a00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90aeca30 0x55ac90af7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 -- 172.21.15.59:0/1792521402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 msgr2=0x55ac90af5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 -- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90aeca30 msgr2=0x55ac90af7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90aeca30 0x55ac90af7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 -- 172.21.15.59:0/1792521402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac90af7a60 con 0x55ac90ae78c0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff943e00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 0x55ac90af64d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff934003e60 tx=0x7ff934005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed90ed3fd562d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff93aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93402b030 con 0x55ac90ae78c0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff934018040 con 0x55ac90ae78c0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.586+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94402c070 con 0x55ac90ae78c0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff934035050 con 0x55ac90ae78c0 2024-04-02T17:26:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 10 client.16884 fetch_fsmap learned FSMap version 38 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac90af8d70 con 0x55ac90ae78c0 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94402c070 con 0x55ac90ae78c0 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff934025040 con 0x55ac90ae78c0 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 10 client.16884 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 20 client.16884 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90b640c0 0x55ac90b664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 4 client.16884 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.587+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac90a5a2b0 con 0x55ac90b640c0 2024-04-02T17:26:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.588+0000 7ff951a00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90b640c0 0x55ac90b664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.590+0000 7ff951a00640 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90b640c0 0x55ac90b664e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb8fb50aa15972e server_cookie=23a62a94d5d2f44a in_seq=0 out_seq=0 2024-04-02T17:26:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.590+0000 7ff93aa00640 10 client.16884 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff93aa00640 1 -- 172.21.15.59:0/1792521402 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac90a5a2b0 con 0x55ac90b640c0 2024-04-02T17:26:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff93aa00640 10 client.16884 handle_command_reply: tid=0 2024-04-02T17:26:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 client.16884 shutdown 2024-04-02T17:26:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 msgr2=0x55ac90af64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 0x55ac90af64d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff934003e60 tx=0x7ff934005c50 comp rx=0 tx=0).stop 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 shutdown_connections 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ac90b640c0 0x55ac90b664e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac90ae78c0 0x55ac90af64d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ac90aeca30 0x55ac90af7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff93aa00640 0 client.16884 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 --2- 172.21.15.59:0/1792521402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ac90ae6ed0 0x55ac90af5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 >> 172.21.15.59:0/1792521402 conn(0x55ac90ae2440 msgr2=0x55ac90ae56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 shutdown_connections 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.640+0000 7ff9567a0740 1 -- 172.21.15.59:0/1792521402 wait complete. 2024-04-02T17:26:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.641+0000 7ff9567a0740 20 client.16884 trim_cache size 0 max 16384 2024-04-02T17:26:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:26:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:26:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:26:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:26:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.641+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 msgr2=0x7ff9300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.641+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 0x7ff9300a6190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0bdf80 tx=0x7ff93c0f5aa0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.641+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 msgr2=0x7ff94c104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.641+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c104960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02c180 tx=0x7ff93c02c6d0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 shutdown_connections 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff9300a3cd0 0x7ff9300a6190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c1025d0 0x7ff94c103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff94c101c10 0x7ff94c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 --2- 172.21.15.59:0/4203962836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff94c100a20 0x7ff94c104960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 >> 172.21.15.59:0/4203962836 conn(0x7ff94c0fbf90 msgr2=0x7ff94c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 shutdown_connections 2024-04-02T17:26:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:04.642+0000 7ff953a00640 1 -- 172.21.15.59:0/4203962836 wait complete. 2024-04-02T17:26:05.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 459247 3.78 MB/sec execute 572 sec latency 101.480 ms 2024-04-02T17:26:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:05 smithi059 ceph-mon[31664]: pgmap v778: 97 pgs: 97 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:26:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:05 smithi164 ceph-mon[36499]: pgmap v778: 97 pgs: 97 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:26:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:05 smithi154 ceph-mon[36479]: pgmap v778: 97 pgs: 97 active+clean; 115 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-02T17:26:06.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 460811 3.79 MB/sec execute 573 sec latency 19.984 ms 2024-04-02T17:26:07.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462115 3.80 MB/sec execute 574 sec latency 22.273 ms 2024-04-02T17:26:07.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:07 smithi059 ceph-mon[31664]: pgmap v779: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:26:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:07 smithi164 ceph-mon[36499]: pgmap v779: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:26:07.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:07 smithi154 ceph-mon[36479]: pgmap v779: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-04-02T17:26:08.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462372 3.79 MB/sec execute 575 sec latency 30.586 ms 2024-04-02T17:26:08.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:08 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:08.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:08 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:08.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:08 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:09.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462495 3.79 MB/sec execute 576 sec latency 800.240 ms 2024-04-02T17:26:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:09 smithi059 ceph-mon[31664]: pgmap v780: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:26:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:09 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:09 smithi164 ceph-mon[36499]: pgmap v780: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:26:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:09 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:09 smithi154 ceph-mon[36479]: pgmap v780: 97 pgs: 97 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-02T17:26:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:09 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:10.118 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462495 3.78 MB/sec execute 577 sec latency 1800.323 ms 2024-04-02T17:26:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:10 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:10.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:10 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:10.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:10 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:11.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462495 3.77 MB/sec execute 578 sec latency 2800.460 ms 2024-04-02T17:26:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:11 smithi059 ceph-mon[31664]: pgmap v781: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-04-02T17:26:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:11 smithi164 ceph-mon[36499]: pgmap v781: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-04-02T17:26:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:11 smithi154 ceph-mon[36479]: pgmap v781: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-04-02T17:26:12.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462495 3.77 MB/sec execute 579 sec latency 3800.596 ms 2024-04-02T17:26:13.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 462495 3.76 MB/sec execute 580 sec latency 4800.730 ms 2024-04-02T17:26:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:13 smithi059 ceph-mon[31664]: pgmap v782: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 171 op/s 2024-04-02T17:26:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:13 smithi164 ceph-mon[36499]: pgmap v782: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 171 op/s 2024-04-02T17:26:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:13 smithi154 ceph-mon[36479]: pgmap v782: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 171 op/s 2024-04-02T17:26:14.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 463462 3.76 MB/sec execute 581 sec latency 4862.316 ms 2024-04-02T17:26:15.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 465163 3.77 MB/sec execute 582 sec latency 15.849 ms 2024-04-02T17:26:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:15 smithi059 ceph-mon[31664]: pgmap v783: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 928 KiB/s wr, 151 op/s 2024-04-02T17:26:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:15 smithi164 ceph-mon[36499]: pgmap v783: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 928 KiB/s wr, 151 op/s 2024-04-02T17:26:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:15 smithi154 ceph-mon[36479]: pgmap v783: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 928 KiB/s wr, 151 op/s 2024-04-02T17:26:16.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 465820 3.77 MB/sec execute 583 sec latency 28.032 ms 2024-04-02T17:26:17.119 INFO:tasks.workunit.client.0.smithi059.stdout: 1 466424 3.77 MB/sec execute 584 sec latency 21.999 ms 2024-04-02T17:26:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:17 smithi059 ceph-mon[31664]: pgmap v784: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 710 KiB/s wr, 135 op/s 2024-04-02T17:26:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:17 smithi164 ceph-mon[36499]: pgmap v784: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 710 KiB/s wr, 135 op/s 2024-04-02T17:26:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:17 smithi154 ceph-mon[36479]: pgmap v784: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 710 KiB/s wr, 135 op/s 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.803+0000 7fe0f2000640 1 -- 172.21.15.59:0/3853266827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec104ef0 msgr2=0x7fe0ec1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.803+0000 7fe0f2000640 1 --2- 172.21.15.59:0/3853266827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec1072e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d802bc30 comp rx=0 tx=0).stop 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 -- 172.21.15.59:0/3853266827 shutdown_connections 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 --2- 172.21.15.59:0/3853266827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec107820 0x7fe0ec109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 --2- 172.21.15.59:0/3853266827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec1072e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 --2- 172.21.15.59:0/3853266827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec1025c0 0x7fe0ec1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 -- 172.21.15.59:0/3853266827 >> 172.21.15.59:0/3853266827 conn(0x7fe0ec0fbfb0 msgr2=0x7fe0ec0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 -- 172.21.15.59:0/3853266827 shutdown_connections 2024-04-02T17:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.804+0000 7fe0f2000640 1 -- 172.21.15.59:0/3853266827 wait complete. 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 Processor -- start 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 -- start start 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec1025c0 0x7fe0ec199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec107820 0x7fe0ec1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d7b0 con 0x7fe0ec1025c0 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d630 con 0x7fe0ec107820 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.805+0000 7fe0f2000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec10d930 con 0x7fe0ec104ef0 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec1025c0 0x7fe0ec199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55336/0 (socket says 172.21.15.59:55336) 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 -- 172.21.15.59:0/2162751859 learned_addr learned my addr 172.21.15.59:0/2162751859 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec107820 0x7fe0ec1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 -- 172.21.15.59:0/2162751859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec107820 msgr2=0x7fe0ec1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec107820 0x7fe0ec1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 -- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec1025c0 msgr2=0x7fe0ec199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec1025c0 0x7fe0ec199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 -- 172.21.15.59:0/2162751859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1a1b40 con 0x7fe0ec104ef0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.806+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.807+0000 7fe0e3e00640 1 -- 172.21.15.59:0/2162751859 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8041070 con 0x7fe0ec104ef0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.807+0000 7fe0e3e00640 1 -- 172.21.15.59:0/2162751859 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d80044d0 con 0x7fe0ec104ef0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.807+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec1a1dd0 con 0x7fe0ec104ef0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.807+0000 7fe0e3e00640 1 -- 172.21.15.59:0/2162751859 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8034440 con 0x7fe0ec104ef0 2024-04-02T17:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.807+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec1a2260 con 0x7fe0ec104ef0 2024-04-02T17:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.810+0000 7fe0e3e00640 1 -- 172.21.15.59:0/2162751859 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe0d8044070 con 0x7fe0ec104ef0 2024-04-02T17:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.810+0000 7fe0f4dcd740 1 Processor -- start 2024-04-02T17:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 -- start start 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a5053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a505a30 0x55a83a507ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a83a465d70 con 0x55a83a4ffed0 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a83a40e2a0 con 0x55a83a505a30 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a83a461d20 con 0x55a83a5008c0 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39750/0 (socket says 172.21.15.59:39750) 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0ebe00640 1 -- 172.21.15.59:0/220162953 learned_addr learned my addr 172.21.15.59:0/220162953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a505a30 0x55a83a507ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.811+0000 7fe0eb400640 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a5053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 -- 172.21.15.59:0/220162953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 msgr2=0x55a83a5053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a5053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 -- 172.21.15.59:0/220162953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a505a30 msgr2=0x55a83a507ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a505a30 0x55a83a507ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 -- 172.21.15.59:0/220162953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83a422840 con 0x55a83a4ffed0 2024-04-02T17:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0e3e00640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 0x7fe0d00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007f40 tx=0x7fe0e400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed703f6712fb0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0e3e00640 1 -- 172.21.15.59:0/2162751859 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe0d80f1f70 con 0x7fe0ec104ef0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.812+0000 7fe0eb400640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 0x7fe0d00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.813+0000 7fe0d7e00640 1 -- 172.21.15.59:0/220162953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4010900 con 0x55a83a4ffed0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.813+0000 7fe0d7e00640 1 -- 172.21.15.59:0/220162953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e40183f0 con 0x55a83a4ffed0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.813+0000 7fe0d7e00640 1 -- 172.21.15.59:0/220162953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40175a0 con 0x55a83a4ffed0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/220162953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 msgr2=0x55a83a5002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0eb400640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 0x7fe0d00a6110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55a83a500cc0 tx=0x7fe0dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007f40 tx=0x7fe0e400f850 comp rx=0 tx=0).stop 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/220162953 shutdown_connections 2024-04-02T17:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a505a30 0x55a83a507ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a5053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/220162953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a4ffed0 0x55a83a5002f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/220162953 >> 172.21.15.59:0/220162953 conn(0x55a83a4fb440 msgr2=0x55a83a4fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/220162953 shutdown_connections 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/220162953 wait complete. 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.814+0000 7fe0f4dcd740 1 -- start start 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 Processor -- start 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a50a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a83a5089e0 con 0x55a83a505a30 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a83a421660 con 0x55a83a4ffed0 2024-04-02T17:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a83a508b60 con 0x55a83a5008c0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0f4dcd740 10 client.0 _ll_register_callbacks cb 0x55a83a421050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38874/0 (socket says 172.21.15.59:38874) 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0eb400640 1 -- 172.21.15.59:0/3847375720 learned_addr learned my addr 172.21.15.59:0/3847375720 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.815+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a50a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 -- 172.21.15.59:0/3847375720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 msgr2=0x55a83a50a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a50a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 -- 172.21.15.59:0/3847375720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 msgr2=0x55a83a5119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5119c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 -- 172.21.15.59:0/3847375720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83a430e80 con 0x55a83a4ffed0 2024-04-02T17:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0eb400640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009730 tx=0x7fe0dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e093f2eca0809e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.816+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc009e60 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e40183f0 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe0dc01f640 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc01f840 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0f4dcd740 10 client.26999 fetch_fsmap learned FSMap version 41 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a83a17a210 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e40183f0 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe0dc031020 con 0x55a83a4ffed0 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.817+0000 7fe0f4dcd740 10 client.26999 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0f4dcd740 10 client.26999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0f4dcd740 20 client.26999 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a5162a0 0x55a83a5186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0f4dcd740 4 client.26999 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a83a415cd0 con 0x55a83a5162a0 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a5162a0 0x55a83a5186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a5162a0 0x55a83a5186d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a6436c8290bb24 server_cookie=f844d256fb8b22f2 in_seq=0 out_seq=0 2024-04-02T17:26:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.818+0000 7fe0d4c00640 10 client.26999 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0d4c00640 1 -- 172.21.15.59:0/3847375720 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a83a415cd0 con 0x55a83a5162a0 2024-04-02T17:26:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0d4c00640 10 client.26999 handle_command_reply: tid=0 2024-04-02T17:26:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 client.26999 shutdown 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 msgr2=0x55a83a50a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009730 tx=0x7fe0dc004730 comp rx=0 tx=0).stop 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 shutdown_connections 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a5162a0 0x55a83a5186d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a5008c0 0x55a83a50a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/3847375720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a4ffed0 0x55a83a50a440 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0d4c00640 0 client.26999 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.819+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 >> 172.21.15.59:0/3847375720 conn(0x55a83a4fb440 msgr2=0x55a83a4fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.820+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 shutdown_connections 2024-04-02T17:26:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.820+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/3847375720 wait complete. 2024-04-02T17:26:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.820+0000 7fe0f4dcd740 20 client.26999 trim_cache size 0 max 16384 2024-04-02T17:26:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 Processor -- start 2024-04-02T17:26:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 -- start start 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a511950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ab280 con 0x55a83a505a30 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ace20 con 0x55a83a5008c0 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.841+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ab980 con 0x55a83a4ffed0 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39774/0 (socket says 172.21.15.59:39774) 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0ebe00640 1 -- 172.21.15.59:0/535006846 learned_addr learned my addr 172.21.15.59:0/535006846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a511950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 -- 172.21.15.59:0/535006846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 msgr2=0x55a83a511950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a511950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 -- 172.21.15.59:0/535006846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 msgr2=0x55a83a4fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 -- 172.21.15.59:0/535006846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83a246620 con 0x55a83a4ffed0 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a511950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.842+0000 7fe0eb400640 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50a8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc012d20 tx=0x7fe0dc0092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02992d4522a42e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.843+0000 7fe0d7400640 1 -- 172.21.15.59:0/535006846 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc01f640 con 0x55a83a4ffed0 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.843+0000 7fe0d7400640 1 -- 172.21.15.59:0/535006846 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc038020 con 0x55a83a4ffed0 2024-04-02T17:26:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.843+0000 7fe0d7400640 1 -- 172.21.15.59:0/535006846 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0296a0 con 0x55a83a4ffed0 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/535006846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 msgr2=0x55a83a50a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50a8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc012d20 tx=0x7fe0dc0092a0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/535006846 shutdown_connections 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a4fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a511950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/535006846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50a8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/535006846 >> 172.21.15.59:0/535006846 conn(0x55a83a4fb440 msgr2=0x55a83a4fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/535006846 shutdown_connections 2024-04-02T17:26:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/535006846 wait complete. 2024-04-02T17:26:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 -- start start 2024-04-02T17:26:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.844+0000 7fe0f4dcd740 1 Processor -- start 2024-04-02T17:26:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a50f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ac750 con 0x55a83a505a30 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ac050 con 0x55a83a5008c0 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a9260 con 0x55a83a4ffed0 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0f4dcd740 10 client.0 _ll_register_callbacks cb 0x7fe0dc0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a50f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a50f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38896/0 (socket says 172.21.15.59:38896) 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0eaa00640 1 -- 172.21.15.59:0/1058222398 learned_addr learned my addr 172.21.15.59:0/1058222398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.845+0000 7fe0ebe00640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 -- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 msgr2=0x55a83a50f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a50f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 -- 172.21.15.59:0/1058222398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 msgr2=0x55a83a5103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5103c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 -- 172.21.15.59:0/1058222398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83a5109b0 con 0x55a83a4ffed0 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0eb400640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50ee70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc033c90 tx=0x7fe0dc01fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fa1c52e2ce2354 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:26:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc041020 con 0x55a83a4ffed0 2024-04-02T17:26:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0dc038020 con 0x55a83a4ffed0 2024-04-02T17:26:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.846+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe0dc01f640 con 0x55a83a4ffed0 2024-04-02T17:26:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc04c050 con 0x55a83a4ffed0 2024-04-02T17:26:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 10 client.27007 fetch_fsmap learned FSMap version 41 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a83a511cc0 con 0x55a83a4ffed0 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc038020 con 0x55a83a4ffed0 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fe0dc031070 con 0x55a83a4ffed0 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 10 client.27007 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 20 client.27007 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a585750 0x55a83a587b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 4 client.27007 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.847+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a83a4732b0 con 0x55a83a585750 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.848+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a585750 0x55a83a587b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.848+0000 7fe0eaa00640 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a585750 0x55a83a587b70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77eb1628f64d261a server_cookie=7b63d5d1c1480b84 in_seq=0 out_seq=0 2024-04-02T17:26:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.848+0000 7fe0d4c00640 10 client.27007 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.890+0000 7fe0d4c00640 1 -- 172.21.15.59:0/1058222398 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a83a4732b0 con 0x55a83a585750 2024-04-02T17:26:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.890+0000 7fe0d4c00640 10 client.27007 handle_command_reply: tid=0 2024-04-02T17:26:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.890+0000 7fe0f4dcd740 1 client.27007 shutdown 2024-04-02T17:26:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 msgr2=0x55a83a50ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50ee70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc033c90 tx=0x7fe0dc01fc90 comp rx=0 tx=0).stop 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 shutdown_connections 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a83a585750 0x55a83a587b70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a83a505a30 0x55a83a5103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a83a5008c0 0x55a83a50f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 --2- 172.21.15.59:0/1058222398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a83a4ffed0 0x55a83a50ee70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0d4c00640 0 client.27007 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 >> 172.21.15.59:0/1058222398 conn(0x55a83a4fb440 msgr2=0x55a83a4fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 shutdown_connections 2024-04-02T17:26:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 1 -- 172.21.15.59:0/1058222398 wait complete. 2024-04-02T17:26:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.891+0000 7fe0f4dcd740 20 client.27007 trim_cache size 0 max 16384 2024-04-02T17:26:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:26:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:26:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:26:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:26:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 msgr2=0x7fe0d00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 0x7fe0d00a6110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55a83a500cc0 tx=0x7fe0dc005f90 comp rx=0 tx=0).stop 2024-04-02T17:26:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 msgr2=0x7fe0ec19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000c00 tx=0x7fe0d8005d50 comp rx=0 tx=0).stop 2024-04-02T17:26:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 shutdown_connections 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec107820 0x7fe0ec1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0ec104ef0 0x7fe0ec19a460 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe0d00a3c50 0x7fe0d00a6110 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 --2- 172.21.15.59:0/2162751859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0ec1025c0 0x7fe0ec199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 >> 172.21.15.59:0/2162751859 conn(0x7fe0ec0fbfb0 msgr2=0x7fe0ec101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 shutdown_connections 2024-04-02T17:26:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:17.892+0000 7fe0f2000640 1 -- 172.21.15.59:0/2162751859 wait complete. 2024-04-02T17:26:18.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 468043 3.78 MB/sec execute 585 sec latency 20.692 ms 2024-04-02T17:26:19.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 469155 3.79 MB/sec execute 586 sec latency 208.954 ms 2024-04-02T17:26:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:19 smithi059 ceph-mon[31664]: pgmap v785: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 721 KiB/s wr, 115 op/s 2024-04-02T17:26:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:19 smithi164 ceph-mon[36499]: pgmap v785: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 721 KiB/s wr, 115 op/s 2024-04-02T17:26:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:19 smithi154 ceph-mon[36479]: pgmap v785: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 721 KiB/s wr, 115 op/s 2024-04-02T17:26:20.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 469425 3.78 MB/sec execute 587 sec latency 31.908 ms 2024-04-02T17:26:21.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 470061 3.78 MB/sec execute 588 sec latency 26.623 ms 2024-04-02T17:26:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:21 smithi059 ceph-mon[31664]: pgmap v786: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 784 KiB/s wr, 120 op/s 2024-04-02T17:26:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:21 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:21 smithi164 ceph-mon[36499]: pgmap v786: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 784 KiB/s wr, 120 op/s 2024-04-02T17:26:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:21 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:21 smithi154 ceph-mon[36479]: pgmap v786: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 784 KiB/s wr, 120 op/s 2024-04-02T17:26:21.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:21 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:22.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 471465 3.78 MB/sec execute 589 sec latency 22.463 ms 2024-04-02T17:26:23.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 472658 3.79 MB/sec execute 590 sec latency 18.450 ms 2024-04-02T17:26:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:23 smithi059 ceph-mon[31664]: pgmap v787: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 663 KiB/s wr, 115 op/s 2024-04-02T17:26:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:23 smithi164 ceph-mon[36499]: pgmap v787: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 663 KiB/s wr, 115 op/s 2024-04-02T17:26:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:23 smithi154 ceph-mon[36479]: pgmap v787: 97 pgs: 97 active+clean; 104 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 663 KiB/s wr, 115 op/s 2024-04-02T17:26:24.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 472962 3.79 MB/sec execute 591 sec latency 28.422 ms 2024-04-02T17:26:25.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 473589 3.79 MB/sec execute 592 sec latency 35.113 ms 2024-04-02T17:26:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:25 smithi059 ceph-mon[31664]: pgmap v788: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 922 KiB/s wr, 148 op/s 2024-04-02T17:26:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:25 smithi164 ceph-mon[36499]: pgmap v788: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 922 KiB/s wr, 148 op/s 2024-04-02T17:26:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:25 smithi154 ceph-mon[36479]: pgmap v788: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 922 KiB/s wr, 148 op/s 2024-04-02T17:26:26.120 INFO:tasks.workunit.client.0.smithi059.stdout: 1 474987 3.79 MB/sec execute 593 sec latency 24.475 ms 2024-04-02T17:26:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:26 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:26 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:26:26.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:26 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:26.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:26 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:26:26.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:26 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:26.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:26 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:26:27.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 476287 3.80 MB/sec execute 594 sec latency 23.932 ms 2024-04-02T17:26:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:27 smithi164 ceph-mon[36499]: pgmap v789: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1015 KiB/s wr, 156 op/s 2024-04-02T17:26:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:27 smithi154 ceph-mon[36479]: pgmap v789: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1015 KiB/s wr, 156 op/s 2024-04-02T17:26:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:27 smithi059 ceph-mon[31664]: pgmap v789: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1015 KiB/s wr, 156 op/s 2024-04-02T17:26:28.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 476680 3.80 MB/sec execute 595 sec latency 29.388 ms 2024-04-02T17:26:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:26:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:26:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:26:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:26:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:26:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:26:28.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:26:28.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:26:28.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:26:29.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 477646 3.80 MB/sec execute 596 sec latency 22.836 ms 2024-04-02T17:26:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:29 smithi164 ceph-mon[36499]: pgmap v790: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-02T17:26:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:29 smithi154 ceph-mon[36479]: pgmap v790: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-02T17:26:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:29 smithi059 ceph-mon[31664]: pgmap v790: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-02T17:26:30.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 479108 3.80 MB/sec execute 597 sec latency 20.602 ms 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 -- 172.21.15.59:0/2525621446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924102410 msgr2=0x7f4924104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 --2- 172.21.15.59:0/2525621446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924102410 0x7f4924104800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f491402bc30 comp rx=0 tx=0).stop 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 -- 172.21.15.59:0/2525621446 shutdown_connections 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 --2- 172.21.15.59:0/2525621446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924107670 0x7f4924109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 --2- 172.21.15.59:0/2525621446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924104d40 0x7f4924107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 --2- 172.21.15.59:0/2525621446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924102410 0x7f4924104800 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.053+0000 7f4929000640 1 -- 172.21.15.59:0/2525621446 >> 172.21.15.59:0/2525621446 conn(0x7f49240fbe00 msgr2=0x7f49240fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.054+0000 7f4929000640 1 -- 172.21.15.59:0/2525621446 shutdown_connections 2024-04-02T17:26:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.054+0000 7f4929000640 1 -- 172.21.15.59:0/2525621446 wait complete. 2024-04-02T17:26:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 Processor -- start 2024-04-02T17:26:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 -- start start 2024-04-02T17:26:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 0x7f492419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924107670 0x7f49241a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f492410d640 con 0x7f4924104d40 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f492410d4c0 con 0x7f4924102410 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.055+0000 7f4929000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f492410d7c0 con 0x7f4924107670 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924107670 0x7f49241a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58104/0 (socket says 172.21.15.59:58104) 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 -- 172.21.15.59:0/2487384058 learned_addr learned my addr 172.21.15.59:0/2487384058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f491aa00640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 0x7f492419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 -- 172.21.15.59:0/2487384058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924107670 msgr2=0x7f49241a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924107670 0x7f49241a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 -- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 msgr2=0x7f492419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 0x7f492419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f4923400640 1 -- 172.21.15.59:0/2487384058 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241a1b10 con 0x7f4924102410 2024-04-02T17:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.056+0000 7f491aa00640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 0x7f492419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4923400640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4920c00640 1 -- 172.21.15.59:0/2487384058 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914041070 con 0x7f4924102410 2024-04-02T17:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49241a1da0 con 0x7f4924102410 2024-04-02T17:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4920c00640 1 -- 172.21.15.59:0/2487384058 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49140044d0 con 0x7f4924102410 2024-04-02T17:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4920c00640 1 -- 172.21.15.59:0/2487384058 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914034440 con 0x7f4924102410 2024-04-02T17:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.057+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49241a2220 con 0x7f4924102410 2024-04-02T17:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.062+0000 7f492cf22740 1 Processor -- start 2024-04-02T17:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.062+0000 7f4920c00640 1 -- 172.21.15.59:0/2487384058 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4914044070 con 0x7f4924102410 2024-04-02T17:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.062+0000 7f492cf22740 1 -- start start 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.062+0000 7f492cf22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f492cf22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f492cf22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f492cf22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5584cbe72d70 con 0x5584cbf0daf0 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f492cf22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584cbe1b2a0 con 0x5584cbf12c60 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f492cf22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584cbe6ed20 con 0x5584cbf0d100 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f4923e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58110/0 (socket says 172.21.15.59:58110) 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 -- 172.21.15.59:0/2226365027 learned_addr learned my addr 172.21.15.59:0/2226365027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f4923400640 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 -- 172.21.15.59:0/2226365027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 msgr2=0x5584cbf0d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 -- 172.21.15.59:0/2226365027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 msgr2=0x5584cbf125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.063+0000 7f491aa00640 1 -- 172.21.15.59:0/2226365027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cbe2f840 con 0x5584cbf12c60 2024-04-02T17:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f491aa00640 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f490c009cb0 tx=0x7f490c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80823df8cb5e62b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4920c00640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 0x7f49080a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4918c00640 1 -- 172.21.15.59:0/2226365027 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c012450 con 0x5584cbf12c60 2024-04-02T17:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4918c00640 1 -- 172.21.15.59:0/2226365027 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c012ad0 con 0x5584cbf12c60 2024-04-02T17:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4918c00640 1 -- 172.21.15.59:0/2226365027 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00fc70 con 0x5584cbf12c60 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4920c00640 1 -- 172.21.15.59:0/2487384058 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f49140f2030 con 0x7f4924102410 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.064+0000 7f4923e00640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 0x7f49080a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 -- 172.21.15.59:0/2226365027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 msgr2=0x5584cbf150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f490c009cb0 tx=0x7f490c0117b0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f4923e00640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 0x7f49080a60e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5584cbf0e350 tx=0x7f491c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 -- 172.21.15.59:0/2226365027 shutdown_connections 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf12c60 0x5584cbf150f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 --2- 172.21.15.59:0/2226365027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 -- 172.21.15.59:0/2226365027 >> 172.21.15.59:0/2226365027 conn(0x5584cbf08670 msgr2=0x5584cbf08a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.065+0000 7f492cf22740 1 -- 172.21.15.59:0/2226365027 shutdown_connections 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.066+0000 7f492cf22740 1 -- 172.21.15.59:0/2226365027 wait complete. 2024-04-02T17:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.066+0000 7f492cf22740 1 -- start start 2024-04-02T17:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.066+0000 7f492cf22740 1 Processor -- start 2024-04-02T17:26:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf1f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5584cbf15c10 con 0x5584cbf0d100 2024-04-02T17:26:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5584cbe2e660 con 0x5584cbf1cd10 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5584cbf15d90 con 0x5584cbf0daf0 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f492cf22740 10 client.0 _ll_register_callbacks cb 0x5584cbe2e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f491aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f4923400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f491aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51838/0 (socket says 172.21.15.59:51838) 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f491aa00640 1 -- 172.21.15.59:0/2611359482 learned_addr learned my addr 172.21.15.59:0/2611359482 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.067+0000 7f4923e00640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf1f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 -- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 msgr2=0x5584cbf1f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf1f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 -- 172.21.15.59:0/2611359482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 msgr2=0x5584cbf177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 -- 172.21.15.59:0/2611359482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584cbe3de80 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f491aa00640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f490c011cb0 tx=0x7f490c009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9464296629f75b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.068+0000 7f4911600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c029020 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f490c00fbe0 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c012ad0 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f490c034050 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f492cf22740 10 client.27019 fetch_fsmap learned FSMap version 43 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5584cbb87210 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.069+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c012ad0 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f490c0046c0 con 0x5584cbf0daf0 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 10 client.27019 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 10 client.27019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 20 client.27019 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf23750 0x5584cbf25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 4 client.27019 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:26:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584cbe22cd0 con 0x5584cbf23750 2024-04-02T17:26:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.070+0000 7f4923400640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf23750 0x5584cbf25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.071+0000 7f4923400640 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf23750 0x5584cbf25bd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927a492f6aced806 server_cookie=936e76d724c0c1eb in_seq=0 out_seq=0 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.071+0000 7f4911600640 10 client.27019 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.071+0000 7f4911600640 1 -- 172.21.15.59:0/2611359482 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584cbe22cd0 con 0x5584cbf23750 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.071+0000 7f4911600640 10 client.27019 handle_command_reply: tid=0 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 client.27019 shutdown 2024-04-02T17:26:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 msgr2=0x5584cbf17ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f490c011cb0 tx=0x7f490c009820 comp rx=0 tx=0).stop 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 shutdown_connections 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf23750 0x5584cbf25bd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf1f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0daf0 0x5584cbf17ce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 --2- 172.21.15.59:0/2611359482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 >> 172.21.15.59:0/2611359482 conn(0x5584cbf08670 msgr2=0x5584cbf07870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f4911600640 0 client.27019 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 shutdown_connections 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.072+0000 7f492cf22740 1 -- 172.21.15.59:0/2611359482 wait complete. 2024-04-02T17:26:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.073+0000 7f492cf22740 20 client.27019 trim_cache size 0 max 16384 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 Processor -- start 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 -- start start 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49241ab1a0 con 0x5584cbf0daf0 2024-04-02T17:26:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49241ace00 con 0x5584cbf1cd10 2024-04-02T17:26:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f492cf22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49241ab8a0 con 0x5584cbf0d100 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f4923e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f4923e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58128/0 (socket says 172.21.15.59:58128) 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f4923e00640 1 -- 172.21.15.59:0/3214458984 learned_addr learned my addr 172.21.15.59:0/3214458984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923400640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.092+0000 7f491aa00640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 -- 172.21.15.59:0/3214458984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 msgr2=0x5584cbf0f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 -- 172.21.15.59:0/3214458984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 msgr2=0x5584cbf0f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf0f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 -- 172.21.15.59:0/3214458984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584cbc53620 con 0x5584cbf1cd10 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923400640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.093+0000 7f4923e00640 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f491c01e7a0 tx=0x7f491c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abde6d5c80446e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f4913e00640 1 -- 172.21.15.59:0/3214458984 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c0119c0 con 0x5584cbf1cd10 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f4913e00640 1 -- 172.21.15.59:0/3214458984 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f491c01c430 con 0x5584cbf1cd10 2024-04-02T17:26:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f4913e00640 1 -- 172.21.15.59:0/3214458984 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c010670 con 0x5584cbf1cd10 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f492cf22740 1 -- 172.21.15.59:0/3214458984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 msgr2=0x5584cbf06fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f492cf22740 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f491c01e7a0 tx=0x7f491c019000 comp rx=0 tx=0).stop 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f492cf22740 1 -- 172.21.15.59:0/3214458984 shutdown_connections 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.094+0000 7f492cf22740 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0daf0 0x5584cbf0f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf1cd10 0x5584cbf06fe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 --2- 172.21.15.59:0/3214458984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf0d100 0x5584cbf0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 -- 172.21.15.59:0/3214458984 >> 172.21.15.59:0/3214458984 conn(0x5584cbf08670 msgr2=0x5584cbf177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 -- 172.21.15.59:0/3214458984 shutdown_connections 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 -- 172.21.15.59:0/3214458984 wait complete. 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 -- start start 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.095+0000 7f492cf22740 1 Processor -- start 2024-04-02T17:26:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf1cd10 0x5584cbf35870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49241ac670 con 0x5584cbf0d100 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49241abf70 con 0x5584cbf0daf0 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49241a9180 con 0x5584cbf1cd10 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f492cf22740 10 client.0 _ll_register_callbacks cb 0x7f490c00f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f491aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f491aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58132/0 (socket says 172.21.15.59:58132) 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f491aa00640 1 -- 172.21.15.59:0/1571927839 learned_addr learned my addr 172.21.15.59:0/1571927839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f4923400640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.096+0000 7f4923e00640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf1cd10 0x5584cbf35870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 -- 172.21.15.59:0/1571927839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf1cd10 msgr2=0x5584cbf35870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf1cd10 0x5584cbf35870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 -- 172.21.15.59:0/1571927839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 msgr2=0x5584cbf34370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf34370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 -- 172.21.15.59:0/1571927839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584cbf35e60 con 0x5584cbf0daf0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f491aa00640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f490c004c50 tx=0x7f490c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8985750a8b7a8d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f4911600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c029070 con 0x5584cbf0daf0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f491c01c430 con 0x5584cbf0daf0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.097+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f490c00fbe0 con 0x5584cbf0daf0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f490c004860 con 0x5584cbf0daf0 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f492cf22740 10 client.27023 fetch_fsmap learned FSMap version 43 2024-04-02T17:26:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5584cbf37170 con 0x5584cbf0daf0 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f491c01c430 con 0x5584cbf0daf0 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 43) ==== 8382+0+0 (secure 0 0 0) 0x7f490c004d60 con 0x5584cbf0daf0 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f492cf22740 10 client.27023 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f492cf22740 10 client.27023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.098+0000 7f492cf22740 20 client.27023 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.099+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf89e40 0x5584cbf8c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.099+0000 7f492cf22740 4 client.27023 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:26:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.099+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584cbe802b0 con 0x5584cbf89e40 2024-04-02T17:26:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.099+0000 7f4923400640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf89e40 0x5584cbf8c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.100+0000 7f4923400640 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf89e40 0x5584cbf8c260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5af8c00b9880f6 server_cookie=c61575593d43d266 in_seq=0 out_seq=0 2024-04-02T17:26:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.100+0000 7f4911600640 10 client.27023 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:31.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 479953 3.81 MB/sec execute 598 sec latency 37.110 ms 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.136+0000 7f4911600640 1 -- 172.21.15.59:0/1571927839 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584cbe802b0 con 0x5584cbf89e40 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.136+0000 7f4911600640 10 client.27023 handle_command_reply: tid=0 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.136+0000 7f492cf22740 1 client.27023 shutdown 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.136+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 msgr2=0x5584cbf348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.136+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f490c004c50 tx=0x7f490c0117b0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 shutdown_connections 2024-04-02T17:26:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5584cbf89e40 0x5584cbf8c260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5584cbf1cd10 0x5584cbf35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5584cbf0daf0 0x5584cbf348b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 --2- 172.21.15.59:0/1571927839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5584cbf0d100 0x5584cbf34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 >> 172.21.15.59:0/1571927839 conn(0x5584cbf08670 msgr2=0x5584cbf17780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f4911600640 0 client.27023 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 shutdown_connections 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 1 -- 172.21.15.59:0/1571927839 wait complete. 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.137+0000 7f492cf22740 20 client.27023 trim_cache size 0 max 16384 2024-04-02T17:26:31.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:26:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:26:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:26:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:26:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 msgr2=0x7f49080a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 0x7f49080a60e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5584cbf0e350 tx=0x7f491c0096a0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 msgr2=0x7f4924199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914005d50 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 shutdown_connections 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f49080a3c20 0x7f49080a60e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4924104d40 0x7f492419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4924107670 0x7f49241a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 --2- 172.21.15.59:0/2487384058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4924102410 0x7f4924199ef0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 >> 172.21.15.59:0/2487384058 conn(0x7f49240fbe00 msgr2=0x7f4924101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 shutdown_connections 2024-04-02T17:26:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:31.138+0000 7f4929000640 1 -- 172.21.15.59:0/2487384058 wait complete. 2024-04-02T17:26:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:31 smithi164 ceph-mon[36499]: pgmap v791: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-02T17:26:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:31 smithi154 ceph-mon[36479]: pgmap v791: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-02T17:26:31.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:31 smithi059 ceph-mon[31664]: pgmap v791: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-02T17:26:32.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 480288 3.80 MB/sec execute 599 sec latency 81.337 ms 2024-04-02T17:26:33.121 INFO:tasks.workunit.client.0.smithi059.stdout: 1 cleanup 600 sec 2024-04-02T17:26:33.472 INFO:tasks.workunit.client.0.smithi059.stdout: 0 cleanup 600 sec 2024-04-02T17:26:33.472 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:26:33.472 INFO:tasks.workunit.client.0.smithi059.stdout: Operation Count AvgLat MaxLat 2024-04-02T17:26:33.472 INFO:tasks.workunit.client.0.smithi059.stdout: ---------------------------------------- 2024-04-02T17:26:33.472 INFO:tasks.workunit.client.0.smithi059.stdout: NTCreateX 72650 4.729 4933.267 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Close 53375 0.010 0.622 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Rename 3078 11.549 4667.429 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Unlink 14655 4.085 4861.737 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Deltree 2 153.777 206.092 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Mkdir 1 0.039 0.039 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Qpathinfo 65820 1.584 223.129 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Qfileinfo 11545 0.005 0.164 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Qfsinfo 12070 0.719 8.881 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Sfileinfo 5934 0.089 9.420 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Find 25451 0.463 46.091 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: WriteX 36262 0.077 22.583 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: ReadX 113842 0.026 10.541 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: LockX 236 1.033 2.766 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: UnlockX 236 0.892 6.835 2024-04-02T17:26:33.473 INFO:tasks.workunit.client.0.smithi059.stdout: Flush 5101 5.828 208.929 2024-04-02T17:26:33.474 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:26:33.474 INFO:tasks.workunit.client.0.smithi059.stdout:Throughput 3.80169 MB/sec 1 clients 1 procs max_latency=4933.289 ms 2024-04-02T17:26:33.474 INFO:tasks.workunit.client.0.smithi059.stderr:+ dbench 10 2024-04-02T17:26:33.476 INFO:tasks.workunit.client.0.smithi059.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-02T17:26:33.476 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:26:33.476 INFO:tasks.workunit.client.0.smithi059.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-02T17:26:33.480 INFO:tasks.workunit.client.0.smithi059.stdout:9 of 10 processes prepared for launch 0 sec 2024-04-02T17:26:33.580 INFO:tasks.workunit.client.0.smithi059.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-02T17:26:33.580 INFO:tasks.workunit.client.0.smithi059.stdout:releasing clients 2024-04-02T17:26:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:33 smithi154 ceph-mon[36479]: pgmap v792: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-04-02T17:26:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:33 smithi059 ceph-mon[31664]: pgmap v792: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-04-02T17:26:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:33 smithi164 ceph-mon[36499]: pgmap v792: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-04-02T17:26:34.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 312 114.12 MB/sec warmup 1 sec latency 157.113 ms 2024-04-02T17:26:35.580 INFO:tasks.workunit.client.0.smithi059.stdout: 10 679 99.66 MB/sec warmup 2 sec latency 1157.231 ms 2024-04-02T17:26:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:35 smithi154 ceph-mon[36479]: pgmap v793: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-02T17:26:35.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:35 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:35 smithi059 ceph-mon[31664]: pgmap v793: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-02T17:26:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:35 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:35 smithi164 ceph-mon[36499]: pgmap v793: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-02T17:26:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:35 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:36.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 1046 71.55 MB/sec warmup 3 sec latency 2157.358 ms 2024-04-02T17:26:37.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 1625 61.13 MB/sec warmup 4 sec latency 3157.487 ms 2024-04-02T17:26:37.601 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:37 smithi059 ceph-mon[31664]: pgmap v794: 97 pgs: 97 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T17:26:37.624 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:37 smithi164 ceph-mon[36499]: pgmap v794: 97 pgs: 97 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T17:26:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:37 smithi154 ceph-mon[36479]: pgmap v794: 97 pgs: 97 active+clean; 101 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T17:26:38.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 2300 53.38 MB/sec warmup 5 sec latency 4157.611 ms 2024-04-02T17:26:39.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 2943 53.81 MB/sec warmup 6 sec latency 4812.137 ms 2024-04-02T17:26:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:39 smithi154 ceph-mon[36479]: pgmap v795: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T17:26:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:39 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:39 smithi059 ceph-mon[31664]: pgmap v795: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T17:26:39.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:39 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:39 smithi164 ceph-mon[36499]: pgmap v795: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T17:26:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:39 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:26:40.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 3491 54.01 MB/sec warmup 7 sec latency 175.947 ms 2024-04-02T17:26:41.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 4019 48.42 MB/sec warmup 8 sec latency 110.930 ms 2024-04-02T17:26:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:41 smithi154 ceph-mon[36479]: pgmap v796: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-04-02T17:26:41.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:41 smithi059 ceph-mon[31664]: pgmap v796: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-04-02T17:26:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:41 smithi164 ceph-mon[36499]: pgmap v796: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-04-02T17:26:42.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 5223 49.09 MB/sec warmup 9 sec latency 63.596 ms 2024-04-02T17:26:43.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 6625 53.66 MB/sec warmup 10 sec latency 42.858 ms 2024-04-02T17:26:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:43 smithi154 ceph-mon[36479]: pgmap v797: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-04-02T17:26:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:43 smithi059 ceph-mon[31664]: pgmap v797: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-04-02T17:26:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:43 smithi164 ceph-mon[36499]: pgmap v797: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 -- 172.21.15.59:0/2157200299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 msgr2=0x7fe98c104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 --2- 172.21.15.59:0/2157200299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c104fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe978009900 tx=0x7fe97802bc40 comp rx=0 tx=0).stop 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 -- 172.21.15.59:0/2157200299 shutdown_connections 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 --2- 172.21.15.59:0/2157200299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 0x7fe98c105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 --2- 172.21.15.59:0/2157200299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c104fa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 --2- 172.21.15.59:0/2157200299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 -- 172.21.15.59:0/2157200299 >> 172.21.15.59:0/2157200299 conn(0x7fe98c0fbde0 msgr2=0x7fe98c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.304+0000 7fe993000640 1 -- 172.21.15.59:0/2157200299 shutdown_connections 2024-04-02T17:26:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.305+0000 7fe993000640 1 -- 172.21.15.59:0/2157200299 wait complete. 2024-04-02T17:26:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 Processor -- start 2024-04-02T17:26:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 -- start start 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 0x7fe98c1a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113bb0 con 0x7fe98c1054e0 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113a30 con 0x7fe98c1001c0 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.306+0000 7fe993000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113d30 con 0x7fe98c102b30 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48056/0 (socket says 172.21.15.59:48056) 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 -- 172.21.15.59:0/3550775033 learned_addr learned my addr 172.21.15.59:0/3550775033 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe992600640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 0x7fe98c1a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991c00640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 -- 172.21.15.59:0/3550775033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 msgr2=0x7fe98c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 -- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 msgr2=0x7fe98c1a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 0x7fe98c1a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991200640 1 -- 172.21.15.59:0/3550775033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1a5db0 con 0x7fe98c102b30 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.307+0000 7fe991c00640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.308+0000 7fe991200640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe978009230 tx=0x7fe97802c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.308+0000 7fe98a000640 1 -- 172.21.15.59:0/3550775033 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978041070 con 0x7fe98c102b30 2024-04-02T17:26:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.308+0000 7fe98a000640 1 -- 172.21.15.59:0/3550775033 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978004450 con 0x7fe98c102b30 2024-04-02T17:26:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.308+0000 7fe98a000640 1 -- 172.21.15.59:0/3550775033 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978034440 con 0x7fe98c102b30 2024-04-02T17:26:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.309+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a6040 con 0x7fe98c102b30 2024-04-02T17:26:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.309+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a64e0 con 0x7fe98c102b30 2024-04-02T17:26:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 Processor -- start 2024-04-02T17:26:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 -- start start 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac42305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac42330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ac4190d70 con 0x558ac422baf0 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.313+0000 7fe997246740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558ac41392a0 con 0x558ac4230c60 2024-04-02T17:26:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe997246740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ac418cd20 con 0x558ac422b100 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:48064/0 (socket says 172.21.15.59:48064) 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 -- 172.21.15.59:0/1125286648 learned_addr learned my addr 172.21.15.59:0/1125286648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe991c00640 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac42305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 -- 172.21.15.59:0/1125286648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 msgr2=0x558ac42330f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:26:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac42330f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 -- 172.21.15.59:0/1125286648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 msgr2=0x558ac42305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac42305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.314+0000 7fe992600640 1 -- 172.21.15.59:0/1125286648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac414d840 con 0x558ac422b100 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.315+0000 7fe992600640 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe984009fd0 tx=0x7fe9840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cf72a141a3a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.315+0000 7fe97ea00640 1 -- 172.21.15.59:0/1125286648 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984004d20 con 0x558ac422b100 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.315+0000 7fe97ea00640 1 -- 172.21.15.59:0/1125286648 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984004050 con 0x558ac422b100 2024-04-02T17:26:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.315+0000 7fe97ea00640 1 -- 172.21.15.59:0/1125286648 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984005260 con 0x558ac422b100 2024-04-02T17:26:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe98a000640 1 -- 172.21.15.59:0/3550775033 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe978044070 con 0x7fe98c102b30 2024-04-02T17:26:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 -- 172.21.15.59:0/1125286648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 msgr2=0x558ac422b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe984009fd0 tx=0x7fe9840118d0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 -- 172.21.15.59:0/1125286648 shutdown_connections 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac42305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac42330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 --2- 172.21.15.59:0/1125286648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422b100 0x558ac422b520 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 -- 172.21.15.59:0/1125286648 >> 172.21.15.59:0/1125286648 conn(0x558ac4226670 msgr2=0x558ac4226a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.316+0000 7fe997246740 1 -- 172.21.15.59:0/1125286648 shutdown_connections 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.317+0000 7fe997246740 1 -- 172.21.15.59:0/1125286648 wait complete. 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.317+0000 7fe997246740 1 -- start start 2024-04-02T17:26:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.317+0000 7fe997246740 1 Processor -- start 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac42466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ac4244c20 con 0x558ac422b100 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558ac414c660 con 0x558ac422baf0 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.318+0000 7fe997246740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ac4244da0 con 0x558ac4230c60 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe991c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe992600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac42466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe991c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:41792/0 (socket says 172.21.15.59:41792) 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe991c00640 1 -- 172.21.15.59:0/910721460 learned_addr learned my addr 172.21.15.59:0/910721460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe98a000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 0x7fe9740a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe991200640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe997246740 10 client.0 _ll_register_callbacks cb 0x558ac414c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.319+0000 7fe98a000640 1 -- 172.21.15.59:0/3550775033 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe9780f1a80 con 0x7fe98c102b30 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 -- 172.21.15.59:0/910721460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 msgr2=0x558ac424dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 -- 172.21.15.59:0/910721460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 msgr2=0x558ac42466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac42466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 -- 172.21.15.59:0/910721460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac415be80 con 0x558ac422baf0 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe992600640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 0x7fe9740a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe991c00640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe980012b40 tx=0x7fe980012f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5000706159a3b958 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe973400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.320+0000 7fe992600640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 0x7fe9740a60c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe984004740 tx=0x7fe9840046d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980010e60 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe980005020 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe984004050 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe980005220 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe997246740 10 client.27029 fetch_fsmap learned FSMap version 45 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.321+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558ac3ea5210 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984004050 con 0x558ac422baf0 2024-04-02T17:26:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7fe98001a4a0 con 0x558ac422baf0 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 10 client.27029 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 10 client.27029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 20 client.27029 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42525a0 0x558ac4254a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 4 client.27029 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ac4140cd0 con 0x558ac42525a0 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.322+0000 7fe991200640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42525a0 0x558ac4254a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.323+0000 7fe991200640 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42525a0 0x558ac4254a30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67fa45231fed1cd server_cookie=cee9fad98b03341a in_seq=0 out_seq=0 2024-04-02T17:26:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.323+0000 7fe973400640 10 client.27029 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.326+0000 7fe973400640 1 -- 172.21.15.59:0/910721460 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ac4140cd0 con 0x558ac42525a0 2024-04-02T17:26:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.326+0000 7fe973400640 10 client.27029 handle_command_reply: tid=0 2024-04-02T17:26:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 client.27029 shutdown 2024-04-02T17:26:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 msgr2=0x558ac4246c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe980012b40 tx=0x7fe980012f40 comp rx=0 tx=0).stop 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 shutdown_connections 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42525a0 0x558ac4254a30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.327+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422baf0 0x558ac4246c20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 1 --2- 172.21.15.59:0/910721460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac42466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 >> 172.21.15.59:0/910721460 conn(0x558ac4226670 msgr2=0x558ac4225870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe973400640 0 client.27029 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 shutdown_connections 2024-04-02T17:26:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 1 -- 172.21.15.59:0/910721460 wait complete. 2024-04-02T17:26:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.328+0000 7fe997246740 20 client.27029 trim_cache size 0 max 16384 2024-04-02T17:26:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.351+0000 7fe997246740 1 Processor -- start 2024-04-02T17:26:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.351+0000 7fe997246740 1 -- start start 2024-04-02T17:26:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac422d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1a6d60 con 0x558ac422baf0 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1a8a20 con 0x558ac422b100 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe997246740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1a7460 con 0x558ac4230c60 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe991c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe991c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:41808/0 (socket says 172.21.15.59:41808) 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe991c00640 1 -- 172.21.15.59:0/935739725 learned_addr learned my addr 172.21.15.59:0/935739725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe991200640 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac422d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.352+0000 7fe992600640 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 -- 172.21.15.59:0/935739725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 msgr2=0x558ac424dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 -- 172.21.15.59:0/935739725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 msgr2=0x558ac422d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac422d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 -- 172.21.15.59:0/935739725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac3f71620 con 0x558ac422b100 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe991c00640 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe980012b10 tx=0x7fe980005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e5ea133b3e5eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe97e000640 1 -- 172.21.15.59:0/935739725 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980028070 con 0x558ac422b100 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe97e000640 1 -- 172.21.15.59:0/935739725 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98001ad50 con 0x558ac422b100 2024-04-02T17:26:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.353+0000 7fe97e000640 1 -- 172.21.15.59:0/935739725 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980012040 con 0x558ac422b100 2024-04-02T17:26:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 -- 172.21.15.59:0/935739725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 msgr2=0x558ac422d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe980012b10 tx=0x7fe980005870 comp rx=0 tx=0).stop 2024-04-02T17:26:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 -- 172.21.15.59:0/935739725 shutdown_connections 2024-04-02T17:26:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422baf0 0x558ac422d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac4230c60 0x558ac424dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 --2- 172.21.15.59:0/935739725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac422b100 0x558ac422d0c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 -- 172.21.15.59:0/935739725 >> 172.21.15.59:0/935739725 conn(0x558ac4226670 msgr2=0x558ac42466e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 -- 172.21.15.59:0/935739725 shutdown_connections 2024-04-02T17:26:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.355+0000 7fe997246740 1 -- 172.21.15.59:0/935739725 wait complete. 2024-04-02T17:26:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 -- start start 2024-04-02T17:26:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 Processor -- start 2024-04-02T17:26:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac424d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 0x558ac424a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac424a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1a8290 con 0x558ac422b100 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1a7b60 con 0x558ac4230c60 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c10d530 con 0x558ac422baf0 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.356+0000 7fe997246740 10 client.0 _ll_register_callbacks cb 0x7fe9800127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe992600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac424a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 0x558ac424a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe992600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac424a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:41820/0 (socket says 172.21.15.59:41820) 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe992600640 1 -- 172.21.15.59:0/1334210497 learned_addr learned my addr 172.21.15.59:0/1334210497 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991c00640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac424d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 -- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 msgr2=0x558ac424a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac424a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 -- 172.21.15.59:0/1334210497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 msgr2=0x558ac424d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac424d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991200640 1 -- 172.21.15.59:0/1334210497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac424ae00 con 0x558ac422baf0 2024-04-02T17:26:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.357+0000 7fe991c00640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac424d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:26:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe991200640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 0x558ac424a350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9780ef4c0 tx=0x7fe9780b9f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72000f94fbff7d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe973400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:26:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9780d6070 con 0x558ac422baf0 2024-04-02T17:26:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7fe9780ba310 con 0x558ac422baf0 2024-04-02T17:26:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98001ad50 con 0x558ac422baf0 2024-04-02T17:26:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe9780dce60 con 0x558ac422baf0 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe997246740 10 client.27043 fetch_fsmap learned FSMap version 45 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.358+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558ac42b7bd0 con 0x558ac422baf0 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98001ad50 con 0x558ac422baf0 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7fe9780bdf00 con 0x558ac422baf0 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 10 client.27043 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 10 client.27043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 20 client.27043 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42b8590 0x558ac42ba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 4 client.27043 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.359+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ac419e2b0 con 0x558ac42b8590 2024-04-02T17:26:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.360+0000 7fe991c00640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42b8590 0x558ac42ba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.361+0000 7fe991c00640 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42b8590 0x558ac42ba9b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568a786a5fa6ada7 server_cookie=3a490aaeb99562ac in_seq=0 out_seq=0 2024-04-02T17:26:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.361+0000 7fe973400640 10 client.27043 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.571+0000 7fe973400640 1 -- 172.21.15.59:0/1334210497 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ac419e2b0 con 0x558ac42b8590 2024-04-02T17:26:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.571+0000 7fe973400640 10 client.27043 handle_command_reply: tid=0 2024-04-02T17:26:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.571+0000 7fe997246740 1 client.27043 shutdown 2024-04-02T17:26:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 msgr2=0x558ac424a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 0x558ac424a350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9780ef4c0 tx=0x7fe9780b9f40 comp rx=0 tx=0).stop 2024-04-02T17:26:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 shutdown_connections 2024-04-02T17:26:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558ac42b8590 0x558ac42ba9b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558ac4230c60 0x558ac424a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ac422baf0 0x558ac424a350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 --2- 172.21.15.59:0/1334210497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ac422b100 0x558ac424d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 >> 172.21.15.59:0/1334210497 conn(0x558ac4226670 msgr2=0x558ac4246620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.572+0000 7fe973400640 0 client.27043 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.573+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 shutdown_connections 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.573+0000 7fe997246740 1 -- 172.21.15.59:0/1334210497 wait complete. 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.573+0000 7fe997246740 20 client.27043 trim_cache size 0 max 16384 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:26:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:26:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:26:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:26:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 msgr2=0x7fe9740a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 0x7fe9740a60c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe984004740 tx=0x7fe9840046d0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 msgr2=0x7fe98c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe978009230 tx=0x7fe97802c6d0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 shutdown_connections 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9740a3c00 0x7fe9740a60c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe98c1054e0 0x7fe98c1a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe98c102b30 0x7fe98c19e6d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 --2- 172.21.15.59:0/3550775033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c1001c0 0x7fe98c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 >> 172.21.15.59:0/3550775033 conn(0x7fe98c0fbde0 msgr2=0x7fe98c100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.575+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 shutdown_connections 2024-04-02T17:26:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:44.576+0000 7fe993000640 1 -- 172.21.15.59:0/3550775033 wait complete. 2024-04-02T17:26:44.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 7204 52.07 MB/sec warmup 11 sec latency 122.087 ms 2024-04-02T17:26:45.581 INFO:tasks.workunit.client.0.smithi059.stdout: 10 7987 50.31 MB/sec warmup 12 sec latency 61.558 ms 2024-04-02T17:26:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:45 smithi154 ceph-mon[36479]: pgmap v798: 97 pgs: 97 active+clean; 125 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-04-02T17:26:45.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:45 smithi059 ceph-mon[31664]: pgmap v798: 97 pgs: 97 active+clean; 125 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-04-02T17:26:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:45 smithi164 ceph-mon[36499]: pgmap v798: 97 pgs: 97 active+clean; 125 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-04-02T17:26:46.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 9316 50.99 MB/sec warmup 13 sec latency 102.955 ms 2024-04-02T17:26:47.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 10426 53.64 MB/sec warmup 14 sec latency 64.727 ms 2024-04-02T17:26:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:47 smithi154 ceph-mon[36479]: pgmap v799: 97 pgs: 97 active+clean; 130 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-02T17:26:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:47 smithi059 ceph-mon[31664]: pgmap v799: 97 pgs: 97 active+clean; 130 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-02T17:26:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:47 smithi164 ceph-mon[36499]: pgmap v799: 97 pgs: 97 active+clean; 130 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-02T17:26:48.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 10990 51.06 MB/sec warmup 15 sec latency 60.407 ms 2024-04-02T17:26:49.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 12278 51.16 MB/sec warmup 16 sec latency 66.757 ms 2024-04-02T17:26:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:49 smithi154 ceph-mon[36479]: pgmap v800: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-04-02T17:26:49.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:49 smithi059 ceph-mon[31664]: pgmap v800: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-04-02T17:26:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:49 smithi164 ceph-mon[36499]: pgmap v800: 97 pgs: 97 active+clean; 137 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-04-02T17:26:50.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 13664 54.22 MB/sec warmup 17 sec latency 36.781 ms 2024-04-02T17:26:51.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 14293 52.64 MB/sec warmup 18 sec latency 74.041 ms 2024-04-02T17:26:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:51 smithi154 ceph-mon[36479]: pgmap v801: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 547 op/s 2024-04-02T17:26:51.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:51 smithi059 ceph-mon[31664]: pgmap v801: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 547 op/s 2024-04-02T17:26:51.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:51 smithi164 ceph-mon[36499]: pgmap v801: 97 pgs: 97 active+clean; 138 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 547 op/s 2024-04-02T17:26:52.582 INFO:tasks.workunit.client.0.smithi059.stdout: 10 15173 52.46 MB/sec warmup 19 sec latency 135.243 ms 2024-04-02T17:26:53.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 16503 53.12 MB/sec warmup 20 sec latency 37.294 ms 2024-04-02T17:26:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:53 smithi154 ceph-mon[36479]: pgmap v802: 97 pgs: 97 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-04-02T17:26:53.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:53 smithi059 ceph-mon[31664]: pgmap v802: 97 pgs: 97 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-04-02T17:26:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:53 smithi164 ceph-mon[36499]: pgmap v802: 97 pgs: 97 active+clean; 148 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-04-02T17:26:54.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 17487 53.74 MB/sec warmup 21 sec latency 34.413 ms 2024-04-02T17:26:55.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 18237 52.19 MB/sec warmup 22 sec latency 55.162 ms 2024-04-02T17:26:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:55 smithi154 ceph-mon[36479]: pgmap v803: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 748 op/s 2024-04-02T17:26:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:55 smithi059 ceph-mon[31664]: pgmap v803: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 748 op/s 2024-04-02T17:26:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:55 smithi164 ceph-mon[36499]: pgmap v803: 97 pgs: 97 active+clean; 161 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 748 op/s 2024-04-02T17:26:56.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 19797 53.46 MB/sec warmup 23 sec latency 28.855 ms 2024-04-02T17:26:57.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 20664 54.06 MB/sec warmup 24 sec latency 196.868 ms 2024-04-02T17:26:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:57 smithi154 ceph-mon[36479]: pgmap v804: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-02T17:26:57.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 -- 172.21.15.59:0/3896734211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00108160 msgr2=0x7f9a00108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 --2- 172.21.15.59:0/3896734211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00108160 0x7f9a00108580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009900 tx=0x7f99f002bc40 comp rx=0 tx=0).stop 2024-04-02T17:26:57.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 -- 172.21.15.59:0/3896734211 shutdown_connections 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 --2- 172.21.15.59:0/3896734211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00109350 0x7f9a001097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 --2- 172.21.15.59:0/3896734211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 --2- 172.21.15.59:0/3896734211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00108160 0x7f9a00108580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 -- 172.21.15.59:0/3896734211 >> 172.21.15.59:0/3896734211 conn(0x7f9a000770f0 msgr2=0x7f9a00079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 -- 172.21.15.59:0/3896734211 shutdown_connections 2024-04-02T17:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.782+0000 7f9a06800640 1 -- 172.21.15.59:0/3896734211 wait complete. 2024-04-02T17:26:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 Processor -- start 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 -- start start 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00108160 0x7f9a0019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00109350 0x7f9a0019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113e40 con 0x7f9a00108160 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113cc0 con 0x7f9a00109350 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.784+0000 7f9a06800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113fc0 con 0x7f9a00109d10 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.785+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00109350 0x7f9a0019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.785+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.785+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46354/0 (socket says 172.21.15.59:46354) 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.785+0000 7f99ffe00640 1 -- 172.21.15.59:0/4222427475 learned_addr learned my addr 172.21.15.59:0/4222427475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ff400640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00108160 0x7f9a0019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ffe00640 1 -- 172.21.15.59:0/4222427475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00109350 msgr2=0x7f9a0019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ffe00640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00109350 0x7f9a0019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ffe00640 1 -- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00108160 msgr2=0x7f9a0019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ffe00640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00108160 0x7f9a0019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.786+0000 7f99ffe00640 1 -- 172.21.15.59:0/4222427475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a5fb0 con 0x7f9a00109d10 2024-04-02T17:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.788+0000 7f99ffe00640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99f800efd0 tx=0x7f99f800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.788+0000 7f99fcc00640 1 -- 172.21.15.59:0/4222427475 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8003b70 con 0x7f9a00109d10 2024-04-02T17:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.788+0000 7f99fcc00640 1 -- 172.21.15.59:0/4222427475 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f8003d10 con 0x7f9a00109d10 2024-04-02T17:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.788+0000 7f99fcc00640 1 -- 172.21.15.59:0/4222427475 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8003b70 con 0x7f9a00109d10 2024-04-02T17:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.789+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a62a0 con 0x7f9a00109d10 2024-04-02T17:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.789+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a6740 con 0x7f9a00109d10 2024-04-02T17:26:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.795+0000 7f99fcc00640 1 -- 172.21.15.59:0/4222427475 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99f8004020 con 0x7f9a00109d10 2024-04-02T17:26:57.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:57 smithi059 ceph-mon[31664]: pgmap v804: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-02T17:26:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.799+0000 7f99fcc00640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 0x7f99e40a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.800+0000 7f99fcc00640 1 -- 172.21.15.59:0/4222427475 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f99f80d0500 con 0x7f9a00109d10 2024-04-02T17:26:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.800+0000 7f9a0959d740 1 Processor -- start 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 -- start start 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 0x559ffc7ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 0x559ffc7b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ffc702d70 con 0x559ffc7adee0 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559ffc6ab2a0 con 0x559ffc7b3a40 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.801+0000 7f9a0959d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ffc6fed20 con 0x559ffc7ae8d0 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 0x559ffc7b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 0x559ffc7b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53306/0 (socket says 172.21.15.59:53306) 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99ffe00640 1 -- 172.21.15.59:0/38091774 learned_addr learned my addr 172.21.15.59:0/38091774 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99f7e00640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99f7e00640 1 -- 172.21.15.59:0/38091774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 msgr2=0x559ffc7b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.802+0000 7f99f7e00640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 0x559ffc7b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.803+0000 7f99f7e00640 1 -- 172.21.15.59:0/38091774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 msgr2=0x559ffc7ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.804+0000 7f99ff400640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 0x559ffc7ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.804+0000 7f99f7e00640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 0x559ffc7ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.804+0000 7f99f7e00640 1 -- 172.21.15.59:0/38091774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffc6bf840 con 0x559ffc7ae8d0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.804+0000 7f99ff400640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 0x559ffc7ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.804+0000 7f99ff400640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 0x7f99e40a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.805+0000 7f99f7e00640 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b33d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99e800abc0 tx=0x7f99e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b51a692adea759 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.805+0000 7f99f4c00640 1 -- 172.21.15.59:0/38091774 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8005220 con 0x559ffc7ae8d0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.805+0000 7f99f4c00640 1 -- 172.21.15.59:0/38091774 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8004c80 con 0x559ffc7ae8d0 2024-04-02T17:26:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.805+0000 7f99f4c00640 1 -- 172.21.15.59:0/38091774 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80188b0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.806+0000 7f9a0959d740 1 -- 172.21.15.59:0/38091774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 msgr2=0x559ffc7b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.806+0000 7f9a0959d740 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b33d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99e800abc0 tx=0x7f99e800f790 comp rx=0 tx=0).stop 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.808+0000 7f9a0959d740 1 -- 172.21.15.59:0/38091774 shutdown_connections 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.808+0000 7f9a0959d740 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7b3a40 0x559ffc7b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.808+0000 7f9a0959d740 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b33d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.808+0000 7f9a0959d740 1 --2- 172.21.15.59:0/38091774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7adee0 0x559ffc7ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.808+0000 7f9a0959d740 1 -- 172.21.15.59:0/38091774 >> 172.21.15.59:0/38091774 conn(0x559ffc798440 msgr2=0x559ffc798840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.809+0000 7f9a0959d740 1 -- 172.21.15.59:0/38091774 shutdown_connections 2024-04-02T17:26:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.809+0000 7f9a0959d740 1 -- 172.21.15.59:0/38091774 wait complete. 2024-04-02T17:26:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.809+0000 7f9a0959d740 1 -- start start 2024-04-02T17:26:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.809+0000 7f9a0959d740 1 Processor -- start 2024-04-02T17:26:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.810+0000 7f99ff400640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 0x7f99e40a6470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009850 tx=0x7f99f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.810+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.810+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.810+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53308/0 (socket says 172.21.15.59:53308) 2024-04-02T17:26:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57560/0 (socket says 172.21.15.59:57560) 2024-04-02T17:26:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46376/0 (socket says 172.21.15.59:46376) 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ffc7b69f0 con 0x559ffc7b3a40 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f99ff400640 1 -- 172.21.15.59:0/1406687144 learned_addr learned my addr 172.21.15.59:0/1406687144 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559ffc6be660 con 0x559ffc7adee0 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ffc7b6b70 con 0x559ffc7ae8d0 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.811+0000 7f9a0959d740 10 client.0 _ll_register_callbacks cb 0x559ffc6be050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.812+0000 7f99ffe00640 1 -- 172.21.15.59:0/1406687144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 msgr2=0x559ffc7b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.812+0000 7f99ffe00640 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.812+0000 7f99ffe00640 1 -- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 msgr2=0x559ffc7b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.812+0000 7f99ffe00640 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.812+0000 7f99ffe00640 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffc6cde80 con 0x559ffc7b3a40 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f99ffe00640 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99f80a4e60 tx=0x7f99f809c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9c5c55329b662 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f99ed600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f809c720 con 0x559ffc7b3a40 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f99f80d00d0 con 0x559ffc7b3a40 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e8004c80 con 0x559ffc7b3a40 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.814+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f99f80d02d0 con 0x559ffc7b3a40 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 10 client.16941 fetch_fsmap learned FSMap version 45 2024-04-02T17:26:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559ffc417210 con 0x559ffc7b3a40 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8004c80 con 0x559ffc7b3a40 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f99f801f020 con 0x559ffc7b3a40 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 10 client.16941 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 20 client.16941 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc7c42c0 0x559ffc7c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 4 client.16941 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:26:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.815+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ffc6b2cd0 con 0x559ffc7c42c0 2024-04-02T17:26:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.816+0000 7f99ff400640 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc7c42c0 0x559ffc7c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.817+0000 7f99ff400640 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc7c42c0 0x559ffc7c6760 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa49d065fefd2b59 server_cookie=8af2ba51df842bc6 in_seq=0 out_seq=0 2024-04-02T17:26:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.817+0000 7f99ed600640 10 client.16941 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.819+0000 7f99ed600640 1 -- 172.21.15.59:0/1406687144 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ffc6b2cd0 con 0x559ffc7c42c0 2024-04-02T17:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.819+0000 7f99ed600640 10 client.16941 handle_command_reply: tid=0 2024-04-02T17:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.819+0000 7f9a0959d740 1 client.16941 shutdown 2024-04-02T17:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 msgr2=0x559ffc7bfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99f80a4e60 tx=0x7f99f809c140 comp rx=0 tx=0).stop 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 shutdown_connections 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc7c42c0 0x559ffc7c6760 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7bfa00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7ae8d0 0x559ffc7b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.821+0000 7f9a0959d740 1 --2- 172.21.15.59:0/1406687144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.821+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 >> 172.21.15.59:0/1406687144 conn(0x559ffc798440 msgr2=0x559ffc797640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.820+0000 7f99ed600640 0 client.16941 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.821+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 shutdown_connections 2024-04-02T17:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.821+0000 7f9a0959d740 1 -- 172.21.15.59:0/1406687144 wait complete. 2024-04-02T17:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.822+0000 7f9a0959d740 20 client.16941 trim_cache size 0 max 16384 2024-04-02T17:26:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:57 smithi164 ceph-mon[36499]: pgmap v804: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 Processor -- start 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 -- start start 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7adee0 0x559ffc796db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a6f60 con 0x559ffc7b3a40 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a8c20 con 0x559ffc7ae8d0 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f9a0959d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a7660 con 0x559ffc7adee0 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53310/0 (socket says 172.21.15.59:53310) 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.863+0000 7f99f7e00640 1 -- 172.21.15.59:0/865253669 learned_addr learned my addr 172.21.15.59:0/865253669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99ffe00640 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99ff400640 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7adee0 0x559ffc796db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 -- 172.21.15.59:0/865253669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7adee0 msgr2=0x559ffc796db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7adee0 0x559ffc796db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 -- 172.21.15.59:0/865253669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 msgr2=0x559ffc7972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 -- 172.21.15.59:0/865253669 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffc4e3620 con 0x559ffc7ae8d0 2024-04-02T17:26:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.864+0000 7f99f7e00640 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f99e800abc0 tx=0x7f99e800a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647312d16be612b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.865+0000 7f99efe00640 1 -- 172.21.15.59:0/865253669 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8028070 con 0x559ffc7ae8d0 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.865+0000 7f99efe00640 1 -- 172.21.15.59:0/865253669 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e80054e0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.865+0000 7f99efe00640 1 -- 172.21.15.59:0/865253669 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8004d70 con 0x559ffc7ae8d0 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.865+0000 7f9a0959d740 1 -- 172.21.15.59:0/865253669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 msgr2=0x559ffc7bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f99e800abc0 tx=0x7f99e800a6e0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 -- 172.21.15.59:0/865253669 shutdown_connections 2024-04-02T17:26:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7b3a40 0x559ffc7972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bf9c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 --2- 172.21.15.59:0/865253669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7adee0 0x559ffc796db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 -- 172.21.15.59:0/865253669 >> 172.21.15.59:0/865253669 conn(0x559ffc798440 msgr2=0x559ffc7b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 -- 172.21.15.59:0/865253669 shutdown_connections 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.866+0000 7f9a0959d740 1 -- 172.21.15.59:0/865253669 wait complete. 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.867+0000 7f9a0959d740 1 -- start start 2024-04-02T17:26:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.867+0000 7f9a0959d740 1 Processor -- start 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7b3a40 0x559ffc7be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a8490 con 0x559ffc7ae8d0 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a7d60 con 0x559ffc7adee0 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00072270 con 0x559ffc7b3a40 2024-04-02T17:26:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f9a0959d740 10 client.0 _ll_register_callbacks cb 0x7f99f809c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53316/0 (socket says 172.21.15.59:53316) 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99ff400640 1 -- 172.21.15.59:0/3890269077 learned_addr learned my addr 172.21.15.59:0/3890269077 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99ffe00640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7b3a40 0x559ffc7be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 -- 172.21.15.59:0/3890269077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7b3a40 msgr2=0x559ffc7be600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7b3a40 0x559ffc7be600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 -- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 msgr2=0x559ffc7bd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7bd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.868+0000 7f99f7e00640 1 -- 172.21.15.59:0/3890269077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffc7bebf0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.869+0000 7f99f7e00640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bd5f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99e800ab10 tx=0x7f99e8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e25d6f573f741d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:26:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.869+0000 7f99ed600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.869+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8028070 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.869+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f99e802d670 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.870+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e80054e0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.870+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f99e8005690 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.870+0000 7f9a0959d740 10 client.16947 fetch_fsmap learned FSMap version 45 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.870+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559ffc82a8e0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.870+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e80054e0 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f99e8018c90 con 0x559ffc7ae8d0 2024-04-02T17:26:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 10 client.16947 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 20 client.16947 populate_metadata read hostname 'smithi059' 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc82b040 0x559ffc82d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 4 client.16947 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.871+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ffc7102b0 con 0x559ffc82b040 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.872+0000 7f99ff400640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc82b040 0x559ffc82d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:26:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.872+0000 7f99ff400640 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc82b040 0x559ffc82d460 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8882bf125d95111 server_cookie=3cda2906ba0f32be in_seq=0 out_seq=0 2024-04-02T17:26:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:57.872+0000 7f99ed600640 10 client.16947 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.101+0000 7f99ed600640 1 -- 172.21.15.59:0/3890269077 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ffc7102b0 con 0x559ffc82b040 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.101+0000 7f99ed600640 10 client.16947 handle_command_reply: tid=0 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.101+0000 7f9a0959d740 1 client.16947 shutdown 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 msgr2=0x559ffc7bd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bd5f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99e800ab10 tx=0x7f99e8005230 comp rx=0 tx=0).stop 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 shutdown_connections 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559ffc82b040 0x559ffc82d460 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ffc7ae8d0 0x559ffc7bd5f0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99e800ab10 tx=0x7f99e8005230 comp rx=0 tx=0).stop 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ffc7b3a40 0x559ffc7be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 --2- 172.21.15.59:0/3890269077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ffc7adee0 0x559ffc7bd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f99ed600640 0 client.16947 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:26:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.102+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 >> 172.21.15.59:0/3890269077 conn(0x559ffc798440 msgr2=0x559ffc79b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.103+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 shutdown_connections 2024-04-02T17:26:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.103+0000 7f9a0959d740 1 -- 172.21.15.59:0/3890269077 wait complete. 2024-04-02T17:26:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.103+0000 7f9a0959d740 20 client.16947 trim_cache size 0 max 16384 2024-04-02T17:26:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:26:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:26:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:26:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.104+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 msgr2=0x7f99e40a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.104+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 0x7f99e40a6470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009850 tx=0x7f99f0000bc0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.104+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 msgr2=0x7f9a001a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.104+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99f800efd0 tx=0x7f99f800c6a0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 shutdown_connections 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a00109d10 0x7f9a001a5940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a00109350 0x7f9a0019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f99e40a3fb0 0x7f99e40a6470 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 --2- 172.21.15.59:0/4222427475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a00108160 0x7f9a0019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:26:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 >> 172.21.15.59:0/4222427475 conn(0x7f9a000770f0 msgr2=0x7f9a0010c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:26:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 shutdown_connections 2024-04-02T17:26:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:26:58.105+0000 7f9a06800640 1 -- 172.21.15.59:0/4222427475 wait complete. 2024-04-02T17:26:58.583 INFO:tasks.workunit.client.0.smithi059.stdout: 10 21024 52.72 MB/sec warmup 25 sec latency 176.568 ms 2024-04-02T17:26:59.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 21721 51.52 MB/sec warmup 26 sec latency 66.292 ms 2024-04-02T17:26:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:26:59 smithi154 ceph-mon[36479]: pgmap v805: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-04-02T17:26:59.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:26:59 smithi059 ceph-mon[31664]: pgmap v805: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-04-02T17:26:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:26:59 smithi164 ceph-mon[36499]: pgmap v805: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.0 MiB/s wr, 737 op/s 2024-04-02T17:27:00.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 23225 52.45 MB/sec warmup 27 sec latency 20.239 ms 2024-04-02T17:27:01.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 24423 53.67 MB/sec warmup 28 sec latency 21.686 ms 2024-04-02T17:27:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:01 smithi154 ceph-mon[36479]: pgmap v806: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 658 op/s 2024-04-02T17:27:01.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:01 smithi059 ceph-mon[31664]: pgmap v806: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 658 op/s 2024-04-02T17:27:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:01 smithi164 ceph-mon[36499]: pgmap v806: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 658 op/s 2024-04-02T17:27:02.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 24962 52.32 MB/sec warmup 29 sec latency 76.658 ms 2024-04-02T17:27:03.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 25348 51.17 MB/sec warmup 30 sec latency 423.061 ms 2024-04-02T17:27:03.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:03 smithi059 ceph-mon[31664]: pgmap v807: 97 pgs: 97 active+clean; 238 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-02T17:27:03.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:03 smithi164 ceph-mon[36499]: pgmap v807: 97 pgs: 97 active+clean; 238 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-02T17:27:03.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:03 smithi154 ceph-mon[36479]: pgmap v807: 97 pgs: 97 active+clean; 238 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-02T17:27:04.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 25428 49.94 MB/sec warmup 31 sec latency 776.739 ms 2024-04-02T17:27:05.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 26240 49.04 MB/sec warmup 32 sec latency 611.876 ms 2024-04-02T17:27:05.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:05 smithi059 ceph-mon[31664]: pgmap v808: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T17:27:05.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:05 smithi164 ceph-mon[36499]: pgmap v808: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T17:27:05.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:05 smithi154 ceph-mon[36479]: pgmap v808: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T17:27:06.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 26977 49.05 MB/sec warmup 33 sec latency 134.549 ms 2024-04-02T17:27:07.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 27855 49.66 MB/sec warmup 34 sec latency 262.033 ms 2024-04-02T17:27:07.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:07 smithi059 ceph-mon[31664]: pgmap v809: 97 pgs: 97 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 784 op/s 2024-04-02T17:27:07.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:07 smithi164 ceph-mon[36499]: pgmap v809: 97 pgs: 97 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 784 op/s 2024-04-02T17:27:07.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:07 smithi154 ceph-mon[36479]: pgmap v809: 97 pgs: 97 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 20 MiB/s wr, 784 op/s 2024-04-02T17:27:08.584 INFO:tasks.workunit.client.0.smithi059.stdout: 10 28377 48.72 MB/sec warmup 35 sec latency 83.010 ms 2024-04-02T17:27:09.585 INFO:tasks.workunit.client.0.smithi059.stdout: 10 29216 48.31 MB/sec warmup 36 sec latency 178.396 ms 2024-04-02T17:27:09.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:09 smithi059 ceph-mon[31664]: pgmap v810: 97 pgs: 97 active+clean; 396 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 28 MiB/s wr, 937 op/s 2024-04-02T17:27:09.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:09 smithi164 ceph-mon[36499]: pgmap v810: 97 pgs: 97 active+clean; 396 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 28 MiB/s wr, 937 op/s 2024-04-02T17:27:09.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:09 smithi154 ceph-mon[36479]: pgmap v810: 97 pgs: 97 active+clean; 396 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 28 MiB/s wr, 937 op/s 2024-04-02T17:27:10.585 INFO:tasks.workunit.client.0.smithi059.stdout: 10 30606 49.08 MB/sec warmup 37 sec latency 31.792 ms 2024-04-02T17:27:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 -- 172.21.15.59:0/572490326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718104f50 msgr2=0x7f07181053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 --2- 172.21.15.59:0/572490326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718104f50 0x7f07181053d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0704009900 tx=0x7f070402bc30 comp rx=0 tx=0).stop 2024-04-02T17:27:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 -- 172.21.15.59:0/572490326 shutdown_connections 2024-04-02T17:27:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 --2- 172.21.15.59:0/572490326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718105910 0x7f071810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 --2- 172.21.15.59:0/572490326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718104f50 0x7f07181053d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 --2- 172.21.15.59:0/572490326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718103d60 0x7f0718104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 -- 172.21.15.59:0/572490326 >> 172.21.15.59:0/572490326 conn(0x7f07180ff2f0 msgr2=0x7f0718101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 -- 172.21.15.59:0/572490326 shutdown_connections 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.294+0000 7f071f400640 1 -- 172.21.15.59:0/572490326 wait complete. 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.295+0000 7f071f400640 1 Processor -- start 2024-04-02T17:27:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 -- start start 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718103d60 0x7f071819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718105910 0x7f07181a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f071810fa30 con 0x7f0718103d60 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f071810f8b0 con 0x7f0718105910 2024-04-02T17:27:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.296+0000 7f071f400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f071810fbb0 con 0x7f0718104f50 2024-04-02T17:27:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51330/0 (socket says 172.21.15.59:51330) 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 -- 172.21.15.59:0/2178732597 learned_addr learned my addr 172.21.15.59:0/2178732597 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f071d400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718105910 0x7f07181a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f071ca00640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718103d60 0x7f071819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 -- 172.21.15.59:0/2178732597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718105910 msgr2=0x7f07181a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718105910 0x7f07181a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 -- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718103d60 msgr2=0x7f071819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718103d60 0x7f071819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.297+0000 7f0717e00640 1 -- 172.21.15.59:0/2178732597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07181a1c90 con 0x7f0718104f50 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.298+0000 7f0717e00640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f070402c240 tx=0x7f0704004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.298+0000 7f0715600640 1 -- 172.21.15.59:0/2178732597 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704043020 con 0x7f0718104f50 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.298+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07181a1f20 con 0x7f0718104f50 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.298+0000 7f0715600640 1 -- 172.21.15.59:0/2178732597 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07040054e0 con 0x7f0718104f50 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.298+0000 7f0715600640 1 -- 172.21.15.59:0/2178732597 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704003c20 con 0x7f0718104f50 2024-04-02T17:27:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.299+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181a23e0 con 0x7f0718104f50 2024-04-02T17:27:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.303+0000 7f07221ec740 1 Processor -- start 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 -- start start 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 0x55e4e83edb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 0x55e4e83f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e4e834ad70 con 0x55e4e83ed6e0 2024-04-02T17:27:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e4e82f32a0 con 0x55e4e83f3240 2024-04-02T17:27:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f07221ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4e8346d20 con 0x55e4e83ee0d0 2024-04-02T17:27:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.304+0000 7f071ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51346/0 (socket says 172.21.15.59:51346) 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 -- 172.21.15.59:0/1699146034 learned_addr learned my addr 172.21.15.59:0/1699146034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071d400640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 0x55e4e83edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f0715600640 1 -- 172.21.15.59:0/2178732597 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0704046020 con 0x7f0718104f50 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f0717e00640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 0x55e4e83f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 -- 172.21.15.59:0/1699146034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 msgr2=0x55e4e83f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 0x55e4e83f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 -- 172.21.15.59:0/1699146034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 msgr2=0x55e4e83edb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 0x55e4e83edb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071ca00640 1 -- 172.21.15.59:0/1699146034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e8307840 con 0x55e4e83ee0d0 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f071d400640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 0x55e4e83edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.305+0000 7f0717e00640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 0x55e4e83f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.306+0000 7f071ca00640 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f070c00bc80 tx=0x7f070c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ca7a976b460cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.306+0000 7f0709c00640 1 -- 172.21.15.59:0/1699146034 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c015070 con 0x55e4e83ee0d0 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.306+0000 7f0709c00640 1 -- 172.21.15.59:0/1699146034 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c011920 con 0x55e4e83ee0d0 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.307+0000 7f0709c00640 1 -- 172.21.15.59:0/1699146034 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c010890 con 0x55e4e83ee0d0 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.307+0000 7f07221ec740 1 -- 172.21.15.59:0/1699146034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 msgr2=0x55e4e83f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.307+0000 7f07221ec740 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f070c00bc80 tx=0x7f070c00f730 comp rx=0 tx=0).stop 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 -- 172.21.15.59:0/1699146034 shutdown_connections 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83f3240 0x55e4e83f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f2bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 --2- 172.21.15.59:0/1699146034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83ed6e0 0x55e4e83edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 -- 172.21.15.59:0/1699146034 >> 172.21.15.59:0/1699146034 conn(0x55e4e83e0440 msgr2=0x55e4e83e0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 -- 172.21.15.59:0/1699146034 shutdown_connections 2024-04-02T17:27:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 -- 172.21.15.59:0/1699146034 wait complete. 2024-04-02T17:27:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.308+0000 7f07221ec740 1 -- start start 2024-04-02T17:27:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.309+0000 7f07221ec740 1 Processor -- start 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.309+0000 7f07221ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e4e83f6240 con 0x55e4e83f3240 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e4e8306660 con 0x55e4e83ed6e0 2024-04-02T17:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4e83f63c0 con 0x55e4e83ee0d0 2024-04-02T17:27:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51356/0 (socket says 172.21.15.59:51356) 2024-04-02T17:27:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 -- 172.21.15.59:0/4125165045 learned_addr learned my addr 172.21.15.59:0/4125165045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f0717e00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f0715600640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 0x7f07000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071d400640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f07221ec740 10 client.0 _ll_register_callbacks cb 0x55e4e8306050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 -- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 msgr2=0x55e4e83f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f0715600640 1 -- 172.21.15.59:0/2178732597 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f07040f25e0 con 0x7f0718104f50 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 -- 172.21.15.59:0/4125165045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 msgr2=0x55e4e83ff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83ff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f071ca00640 1 -- 172.21.15.59:0/4125165045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e8315e80 con 0x55e4e83ee0d0 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.310+0000 7f0717e00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.311+0000 7f071d400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 0x7f07000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.311+0000 7f071ca00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f070c019000 tx=0x7f070c00bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cfdba9c77d6c899 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.311+0000 7f06fe000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.311+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c015070 con 0x55e4e83ee0d0 2024-04-02T17:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070c011920 con 0x55e4e83ee0d0 2024-04-02T17:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f070c011ad0 con 0x55e4e83ee0d0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f070c011750 con 0x55e4e83ee0d0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f07221ec740 10 client.27073 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e4e805f210 con 0x55e4e83ee0d0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c011920 con 0x55e4e83ee0d0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f071d400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 0x7f07000a6070 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55e4e83ee930 tx=0x7f0710004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.312+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f070c0046c0 con 0x55e4e83ee0d0 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 10 client.27073 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 10 client.27073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 20 client.27073 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e8403690 0x55e4e8405ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 4 client.27073 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e82facd0 con 0x55e4e8403690 2024-04-02T17:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.313+0000 7f0717e00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e8403690 0x55e4e8405ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.314+0000 7f06fe000640 10 client.27073 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.314+0000 7f0717e00640 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e8403690 0x55e4e8405ac0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ef48ea9d7ddd25 server_cookie=2b3118a01bcbadda in_seq=0 out_seq=0 2024-04-02T17:27:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f06fe000640 1 -- 172.21.15.59:0/4125165045 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4e82facd0 con 0x55e4e8403690 2024-04-02T17:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f06fe000640 10 client.27073 handle_command_reply: tid=0 2024-04-02T17:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 client.27073 shutdown 2024-04-02T17:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 msgr2=0x55e4e83f8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f070c019000 tx=0x7f070c00bfa0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 shutdown_connections 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e8403690 0x55e4e8405ac0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f06fe000640 0 client.27073 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83f8170 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 --2- 172.21.15.59:0/4125165045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.316+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 >> 172.21.15.59:0/4125165045 conn(0x55e4e83e0440 msgr2=0x55e4e83df640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.317+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 shutdown_connections 2024-04-02T17:27:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.317+0000 7f07221ec740 1 -- 172.21.15.59:0/4125165045 wait complete. 2024-04-02T17:27:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.318+0000 7f07221ec740 20 client.27073 trim_cache size 0 max 16384 2024-04-02T17:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.346+0000 7f07221ec740 1 Processor -- start 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.346+0000 7f07221ec740 1 -- start start 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a7010 con 0x55e4e83f3240 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a8cd0 con 0x55e4e83ed6e0 2024-04-02T17:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f07221ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a7710 con 0x55e4e83ee0d0 2024-04-02T17:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f071ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f071ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43736/0 (socket says 172.21.15.59:43736) 2024-04-02T17:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f071ca00640 1 -- 172.21.15.59:0/1863263928 learned_addr learned my addr 172.21.15.59:0/1863263928 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f0717e00640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.347+0000 7f071d400640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 -- 172.21.15.59:0/1863263928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 msgr2=0x55e4e83ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 -- 172.21.15.59:0/1863263928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 msgr2=0x55e4e83dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83dedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 -- 172.21.15.59:0/1863263928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e812b620 con 0x55e4e83ed6e0 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071d400640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.348+0000 7f071ca00640 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f070c004b30 tx=0x7f070c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2faf01a056fd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.349+0000 7f0709200640 1 -- 172.21.15.59:0/1863263928 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c031070 con 0x55e4e83ed6e0 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.349+0000 7f0709200640 1 -- 172.21.15.59:0/1863263928 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c011dd0 con 0x55e4e83ed6e0 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 -- 172.21.15.59:0/1863263928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 msgr2=0x55e4e83f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f0709200640 1 -- 172.21.15.59:0/1863263928 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c015070 con 0x55e4e83ed6e0 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f070c004b30 tx=0x7f070c011440 comp rx=0 tx=0).stop 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 -- 172.21.15.59:0/1863263928 shutdown_connections 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 --2- 172.21.15.59:0/1863263928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83f7650 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.350+0000 7f07221ec740 1 -- 172.21.15.59:0/1863263928 >> 172.21.15.59:0/1863263928 conn(0x55e4e83e0440 msgr2=0x55e4e83f6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.351+0000 7f07221ec740 1 -- 172.21.15.59:0/1863263928 shutdown_connections 2024-04-02T17:27:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.351+0000 7f07221ec740 1 -- 172.21.15.59:0/1863263928 wait complete. 2024-04-02T17:27:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.351+0000 7f07221ec740 1 -- start start 2024-04-02T17:27:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 Processor -- start 2024-04-02T17:27:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83fe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a8540 con 0x55e4e83f3240 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a7e10 con 0x55e4e83ed6e0 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a4ff0 con 0x55e4e83ee0d0 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f07221ec740 10 client.0 _ll_register_callbacks cb 0x7f070c00b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f0717e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f071ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43742/0 (socket says 172.21.15.59:43742) 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 -- 172.21.15.59:0/203516972 learned_addr learned my addr 172.21.15.59:0/203516972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.352+0000 7f0717e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51370/0 (socket says 172.21.15.59:51370) 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071d400640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83fe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 -- 172.21.15.59:0/203516972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 msgr2=0x55e4e83fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 -- 172.21.15.59:0/203516972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 msgr2=0x55e4e83fe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83fe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f071ca00640 1 -- 172.21.15.59:0/203516972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e83fe790 con 0x55e4e83ed6e0 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.353+0000 7f0717e00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.354+0000 7f071ca00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f070c00f770 tx=0x7f070c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f3e70c71be957f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.354+0000 7f06fe000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:27:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.354+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c031070 con 0x55e4e83ed6e0 2024-04-02T17:27:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.354+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f070c005190 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.354+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f070c011dd0 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f070c015040 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 10 client.27065 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e4e83e8ef0 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c011dd0 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f070c03d020 con 0x55e4e83ed6e0 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 10 client.27065 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 10 client.27065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 20 client.27065 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e83e9930 0x55e4e83ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 4 client.27065 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:27:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.355+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4e83582b0 con 0x55e4e83e9930 2024-04-02T17:27:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.356+0000 7f0717e00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e83e9930 0x55e4e83ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.359+0000 7f06fe000640 10 client.27065 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.359+0000 7f0717e00640 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e83e9930 0x55e4e83ebd50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae944ddc5adfd07 server_cookie=995d748a27a8f865 in_seq=0 out_seq=0 2024-04-02T17:27:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.499+0000 7f06fe000640 1 -- 172.21.15.59:0/203516972 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4e83582b0 con 0x55e4e83e9930 2024-04-02T17:27:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.499+0000 7f06fe000640 10 client.27065 handle_command_reply: tid=0 2024-04-02T17:27:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.500+0000 7f07221ec740 1 client.27065 shutdown 2024-04-02T17:27:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.500+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 msgr2=0x55e4e83fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.500+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f070c00f770 tx=0x7f070c00b610 comp rx=0 tx=0).stop 2024-04-02T17:27:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 shutdown_connections 2024-04-02T17:27:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e4e83e9930 0x55e4e83ebd50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4e83f3240 0x55e4e83fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e83ee0d0 0x55e4e83fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 --2- 172.21.15.59:0/203516972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e4e83ed6e0 0x55e4e83fac40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 >> 172.21.15.59:0/203516972 conn(0x55e4e83e0440 msgr2=0x55e4e83dedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f06fe000640 0 client.27065 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 shutdown_connections 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 1 -- 172.21.15.59:0/203516972 wait complete. 2024-04-02T17:27:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.501+0000 7f07221ec740 20 client.27065 trim_cache size 0 max 16384 2024-04-02T17:27:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:27:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:27:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:27:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 msgr2=0x7f07000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 0x7f07000a6070 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55e4e83ee930 tx=0x7f0710004ab0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 msgr2=0x7f071819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f070402c240 tx=0x7f0704004e30 comp rx=0 tx=0).stop 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 shutdown_connections 2024-04-02T17:27:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0718105910 0x7f07181a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0718104f50 0x7f071819a5b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f07000a3bb0 0x7f07000a6070 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 --2- 172.21.15.59:0/2178732597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0718103d60 0x7f071819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 >> 172.21.15.59:0/2178732597 conn(0x7f07180ff2f0 msgr2=0x7f0718108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 shutdown_connections 2024-04-02T17:27:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:11.503+0000 7f071f400640 1 -- 172.21.15.59:0/2178732597 wait complete. 2024-04-02T17:27:11.585 INFO:tasks.workunit.client.0.smithi059.stdout: 10 31368 49.30 MB/sec warmup 38 sec latency 93.310 ms 2024-04-02T17:27:11.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:11 smithi059 ceph-mon[31664]: pgmap v811: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 35 MiB/s wr, 1.10k op/s 2024-04-02T17:27:11.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:11 smithi164 ceph-mon[36499]: pgmap v811: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 35 MiB/s wr, 1.10k op/s 2024-04-02T17:27:11.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:11 smithi154 ceph-mon[36479]: pgmap v811: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 35 MiB/s wr, 1.10k op/s 2024-04-02T17:27:12.585 INFO:tasks.workunit.client.0.smithi059.stdout: 10 31935 48.63 MB/sec warmup 39 sec latency 156.226 ms 2024-04-02T17:27:13.588 INFO:tasks.workunit.client.0.smithi059.stdout: 10 32828 48.34 MB/sec warmup 40 sec latency 42.086 ms 2024-04-02T17:27:13.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:13 smithi059 ceph-mon[31664]: pgmap v812: 97 pgs: 97 active+clean; 384 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-04-02T17:27:13.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:13 smithi164 ceph-mon[36499]: pgmap v812: 97 pgs: 97 active+clean; 384 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-04-02T17:27:13.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:13 smithi154 ceph-mon[36479]: pgmap v812: 97 pgs: 97 active+clean; 384 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-04-02T17:27:14.588 INFO:tasks.workunit.client.0.smithi059.stdout: 10 34172 49.10 MB/sec warmup 41 sec latency 35.150 ms 2024-04-02T17:27:15.588 INFO:tasks.workunit.client.0.smithi059.stdout: 10 34970 49.22 MB/sec warmup 42 sec latency 151.972 ms 2024-04-02T17:27:15.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:15 smithi059 ceph-mon[31664]: pgmap v813: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 31 MiB/s wr, 1.12k op/s 2024-04-02T17:27:15.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:15 smithi164 ceph-mon[36499]: pgmap v813: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 31 MiB/s wr, 1.12k op/s 2024-04-02T17:27:15.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:15 smithi154 ceph-mon[36479]: pgmap v813: 97 pgs: 97 active+clean; 390 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 31 MiB/s wr, 1.12k op/s 2024-04-02T17:27:16.588 INFO:tasks.workunit.client.0.smithi059.stdout: 10 35658 48.80 MB/sec warmup 43 sec latency 55.278 ms 2024-04-02T17:27:17.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 36788 48.89 MB/sec warmup 44 sec latency 57.911 ms 2024-04-02T17:27:17.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:17 smithi059 ceph-mon[31664]: pgmap v814: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 30 MiB/s wr, 1.04k op/s 2024-04-02T17:27:17.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:17 smithi164 ceph-mon[36499]: pgmap v814: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 30 MiB/s wr, 1.04k op/s 2024-04-02T17:27:17.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:17 smithi154 ceph-mon[36479]: pgmap v814: 97 pgs: 97 active+clean; 394 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 30 MiB/s wr, 1.04k op/s 2024-04-02T17:27:18.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 37927 49.43 MB/sec warmup 45 sec latency 38.295 ms 2024-04-02T17:27:19.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 38714 49.20 MB/sec warmup 46 sec latency 33.312 ms 2024-04-02T17:27:19.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:19 smithi059 ceph-mon[31664]: pgmap v815: 97 pgs: 97 active+clean; 404 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-04-02T17:27:19.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:19 smithi164 ceph-mon[36499]: pgmap v815: 97 pgs: 97 active+clean; 404 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-04-02T17:27:19.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:19 smithi154 ceph-mon[36479]: pgmap v815: 97 pgs: 97 active+clean; 404 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-04-02T17:27:20.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 39537 48.91 MB/sec warmup 47 sec latency 35.937 ms 2024-04-02T17:27:21.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 40871 49.37 MB/sec warmup 48 sec latency 36.800 ms 2024-04-02T17:27:21.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:21 smithi059 ceph-mon[31664]: pgmap v816: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 892 op/s 2024-04-02T17:27:21.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:21 smithi164 ceph-mon[36499]: pgmap v816: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 892 op/s 2024-04-02T17:27:21.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:21 smithi154 ceph-mon[36479]: pgmap v816: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 892 op/s 2024-04-02T17:27:22.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 41828 49.81 MB/sec warmup 49 sec latency 37.198 ms 2024-04-02T17:27:23.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 42413 49.16 MB/sec warmup 50 sec latency 37.487 ms 2024-04-02T17:27:23.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:23 smithi059 ceph-mon[31664]: pgmap v817: 97 pgs: 97 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-04-02T17:27:23.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:23 smithi164 ceph-mon[36499]: pgmap v817: 97 pgs: 97 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-04-02T17:27:23.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:23 smithi154 ceph-mon[36479]: pgmap v817: 97 pgs: 97 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 648 op/s 2024-04-02T17:27:24.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 43519 49.17 MB/sec warmup 51 sec latency 35.682 ms 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 -- 172.21.15.59:0/3127145178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0103d60 msgr2=0x7f2fa0104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/3127145178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0103d60 0x7f2fa0104180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009900 tx=0x7f2f8c02bc60 comp rx=0 tx=0).stop 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 -- 172.21.15.59:0/3127145178 shutdown_connections 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/3127145178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 0x7f2fa01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/3127145178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0105910 0x7f2fa010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/3127145178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0103d60 0x7f2fa0104180 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009900 tx=0x7f2f8c02bc60 comp rx=0 tx=0).stop 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 -- 172.21.15.59:0/3127145178 >> 172.21.15.59:0/3127145178 conn(0x7f2fa00ff2f0 msgr2=0x7f2fa0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.692+0000 7f2fa6c00640 1 -- 172.21.15.59:0/3127145178 shutdown_connections 2024-04-02T17:27:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.693+0000 7f2fa6c00640 1 -- 172.21.15.59:0/3127145178 wait complete. 2024-04-02T17:27:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.694+0000 7f2fa6c00640 1 Processor -- start 2024-04-02T17:27:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.695+0000 7f2fa6c00640 1 -- start start 2024-04-02T17:27:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.695+0000 7f2fa6c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 0x7f2fa0195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.695+0000 7f2fa6c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 0x7f2fa01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.695+0000 7f2fa6c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa6c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010faa0 con 0x7f2fa0104f50 2024-04-02T17:27:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa6c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010f920 con 0x7f2fa019b420 2024-04-02T17:27:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa6c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010fc20 con 0x7f2fa0105910 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53198/0 (socket says 172.21.15.59:53198) 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2fa4c00640 1 -- 172.21.15.59:0/418458715 learned_addr learned my addr 172.21.15.59:0/418458715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 0x7f2fa0195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.696+0000 7f2f9f400640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 0x7f2fa01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 -- 172.21.15.59:0/418458715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 msgr2=0x7f2fa01963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 0x7f2fa01963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 -- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 msgr2=0x7f2fa0195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 0x7f2fa0195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 -- 172.21.15.59:0/418458715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa01aeee0 con 0x7f2fa019b420 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2f9f400640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 0x7f2fa01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.697+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b990 tx=0x7f2f9800be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.698+0000 7f2f9cc00640 1 -- 172.21.15.59:0/418458715 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98019070 con 0x7f2fa019b420 2024-04-02T17:27:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.698+0000 7f2f9cc00640 1 -- 172.21.15.59:0/418458715 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9800cba0 con 0x7f2fa019b420 2024-04-02T17:27:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.698+0000 7f2f9cc00640 1 -- 172.21.15.59:0/418458715 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98007650 con 0x7f2fa019b420 2024-04-02T17:27:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.698+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa01af1d0 con 0x7f2fa019b420 2024-04-02T17:27:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.698+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01af6a0 con 0x7f2fa019b420 2024-04-02T17:27:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.714+0000 7f2f9cc00640 1 -- 172.21.15.59:0/418458715 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f9801c070 con 0x7f2fa019b420 2024-04-02T17:27:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.716+0000 7f2fa9990740 1 Processor -- start 2024-04-02T17:27:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.716+0000 7f2f9cc00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 0x7f2f840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.716+0000 7f2fa9990740 1 -- start start 2024-04-02T17:27:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2f9cc00640 1 -- 172.21.15.59:0/418458715 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2f980cf990 con 0x7f2fa019b420 2024-04-02T17:27:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x55741529c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x55741529eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574151fcd70 con 0x55741529ca30 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5574151a52a0 con 0x557415296ed0 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa9990740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5574151f8d20 con 0x5574152978c0 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2f9f400640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 0x7f2f840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x55741529eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x55741529eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34464/0 (socket says 172.21.15.59:34464) 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.717+0000 7f2fa4c00640 1 -- 172.21.15.59:0/661702656 learned_addr learned my addr 172.21.15.59:0/661702656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x55741529c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 -- 172.21.15.59:0/661702656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 msgr2=0x5574152972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 -- 172.21.15.59:0/661702656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 msgr2=0x55741529eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x55741529eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.718+0000 7f2f9f400640 1 -- 172.21.15.59:0/661702656 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574151b9840 con 0x5574152978c0 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.719+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.719+0000 7f2f9f400640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 0x7f2f840a6190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0197390 tx=0x7f2f9000b4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.719+0000 7f2f9f400640 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x55741529c3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011b90 tx=0x7f2f9001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3e59f686118234 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.719+0000 7f2f95200640 1 -- 172.21.15.59:0/661702656 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90024070 con 0x5574152978c0 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.720+0000 7f2f95200640 1 -- 172.21.15.59:0/661702656 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90005500 con 0x5574152978c0 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 -- 172.21.15.59:0/661702656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 msgr2=0x55741529c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x55741529c3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011b90 tx=0x7f2f9001b000 comp rx=0 tx=0).stop 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2f95200640 1 -- 172.21.15.59:0/661702656 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000e7b0 con 0x5574152978c0 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 -- 172.21.15.59:0/661702656 shutdown_connections 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x55741529eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x55741529c3c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 --2- 172.21.15.59:0/661702656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 -- 172.21.15.59:0/661702656 >> 172.21.15.59:0/661702656 conn(0x557415292440 msgr2=0x557415292840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 -- 172.21.15.59:0/661702656 shutdown_connections 2024-04-02T17:27:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.721+0000 7f2fa9990740 1 -- 172.21.15.59:0/661702656 wait complete. 2024-04-02T17:27:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.722+0000 7f2fa9990740 1 -- start start 2024-04-02T17:27:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.722+0000 7f2fa9990740 1 Processor -- start 2024-04-02T17:27:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.723+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55741529fa20 con 0x55741529ca30 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5574151b8660 con 0x557415296ed0 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55741529fba0 con 0x5574152978c0 2024-04-02T17:27:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.724+0000 7f2fa9990740 10 client.0 _ll_register_callbacks cb 0x5574151b8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.725+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.725+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.725+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36788/0 (socket says 172.21.15.59:36788) 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.725+0000 7f2f9f400640 1 -- 172.21.15.59:0/776484468 learned_addr learned my addr 172.21.15.59:0/776484468 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9fe00640 1 -- 172.21.15.59:0/776484468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 msgr2=0x5574152a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9fe00640 1 -- 172.21.15.59:0/776484468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 msgr2=0x5574152a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9fe00640 1 -- 172.21.15.59:0/776484468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574151c7e80 con 0x557415296ed0 2024-04-02T17:27:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.726+0000 7f2f9f400640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.727+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a1410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01038a0 tx=0x7f2f8c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5f670c0f4f2c6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.727+0000 7f2f89600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.728+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c005040 con 0x557415296ed0 2024-04-02T17:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.728+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2f8c0051e0 con 0x557415296ed0 2024-04-02T17:27:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.728+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f90005500 con 0x557415296ed0 2024-04-02T17:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.729+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2f8c0053e0 con 0x557415296ed0 2024-04-02T17:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.729+0000 7f2fa9990740 10 client.27074 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.729+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557414f11210 con 0x557415296ed0 2024-04-02T17:27:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.729+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90005500 con 0x557415296ed0 2024-04-02T17:27:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f2f8c003830 con 0x557415296ed0 2024-04-02T17:27:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 10 client.27074 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 20 client.27074 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574152ad6a0 0x5574152afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 4 client.27074 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:27:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.730+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574151accd0 con 0x5574152ad6a0 2024-04-02T17:27:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.731+0000 7f2f9f400640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574152ad6a0 0x5574152afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.734+0000 7f2f9f400640 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574152ad6a0 0x5574152afb40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47f1d9838e76483 server_cookie=cfeb5e790b649fbd in_seq=0 out_seq=0 2024-04-02T17:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.735+0000 7f2f89600640 10 client.27074 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.735+0000 7f2f89600640 1 -- 172.21.15.59:0/776484468 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574151accd0 con 0x5574152ad6a0 2024-04-02T17:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.735+0000 7f2f89600640 10 client.27074 handle_command_reply: tid=0 2024-04-02T17:27:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.735+0000 7f2fa9990740 1 client.27074 shutdown 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.736+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 msgr2=0x5574152a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.736+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a1410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01038a0 tx=0x7f2f8c004270 comp rx=0 tx=0).stop 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 shutdown_connections 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574152ad6a0 0x5574152afb40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 --2- 172.21.15.59:0/776484468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a1410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 >> 172.21.15.59:0/776484468 conn(0x557415292440 msgr2=0x557415291640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2f89600640 0 client.27074 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.737+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 shutdown_connections 2024-04-02T17:27:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.738+0000 7f2fa9990740 1 -- 172.21.15.59:0/776484468 wait complete. 2024-04-02T17:27:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.738+0000 7f2fa9990740 20 client.27074 trim_cache size 0 max 16384 2024-04-02T17:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.767+0000 7f2fa9990740 1 Processor -- start 2024-04-02T17:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 -- start start 2024-04-02T17:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01b8660 con 0x55741529ca30 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01ba320 con 0x557415296ed0 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.768+0000 7f2fa9990740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01b8d60 con 0x5574152978c0 2024-04-02T17:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.769+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.769+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53222/0 (socket says 172.21.15.59:53222) 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.769+0000 7f2f9fe00640 1 -- 172.21.15.59:0/1765513031 learned_addr learned my addr 172.21.15.59:0/1765513031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.769+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 -- 172.21.15.59:0/1765513031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 msgr2=0x5574152a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 -- 172.21.15.59:0/1765513031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 msgr2=0x5574152a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 -- 172.21.15.59:0/1765513031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557414fdd620 con 0x5574152978c0 2024-04-02T17:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.770+0000 7f2f9f400640 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a8950 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5574152b0390 tx=0x7f2f90005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d671d91ebf070c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.771+0000 7f2f8be00640 1 -- 172.21.15.59:0/1765513031 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90024040 con 0x5574152978c0 2024-04-02T17:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.771+0000 7f2f8be00640 1 -- 172.21.15.59:0/1765513031 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90037070 con 0x5574152978c0 2024-04-02T17:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.771+0000 7f2f8be00640 1 -- 172.21.15.59:0/1765513031 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900035f0 con 0x5574152978c0 2024-04-02T17:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 -- 172.21.15.59:0/1765513031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 msgr2=0x5574152a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a8950 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5574152b0390 tx=0x7f2f90005fd0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 -- 172.21.15.59:0/1765513031 shutdown_connections 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55741529ca30 0x5574152a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5574152978c0 0x5574152a8950 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 --2- 172.21.15.59:0/1765513031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557415296ed0 0x5574152a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 -- 172.21.15.59:0/1765513031 >> 172.21.15.59:0/1765513031 conn(0x557415292440 msgr2=0x557415295f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.772+0000 7f2fa9990740 1 -- 172.21.15.59:0/1765513031 shutdown_connections 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.773+0000 7f2fa9990740 1 -- 172.21.15.59:0/1765513031 wait complete. 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.773+0000 7f2fa9990740 1 -- start start 2024-04-02T17:27:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.773+0000 7f2fa9990740 1 Processor -- start 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55741529ca30 0x5574152a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 0x5574152a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01b9b90 con 0x557415296ed0 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01b9460 con 0x5574152a6f80 2024-04-02T17:27:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01b6640 con 0x55741529ca30 2024-04-02T17:27:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.774+0000 7f2fa9990740 10 client.0 _ll_register_callbacks cb 0x7f2f8c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34498/0 (socket says 172.21.15.59:34498) 2024-04-02T17:27:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 -- 172.21.15.59:0/2875582475 learned_addr learned my addr 172.21.15.59:0/2875582475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 0x5574152a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9f400640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55741529ca30 0x5574152a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 -- 172.21.15.59:0/2875582475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55741529ca30 msgr2=0x5574152a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55741529ca30 0x5574152a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 -- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 msgr2=0x5574152a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 0x5574152a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.775+0000 7f2f9fe00640 1 -- 172.21.15.59:0/2875582475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574152a7940 con 0x557415296ed0 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2fa4c00640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 0x5574152a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2f9fe00640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c003d50 tx=0x7f2f8c02cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00e1498b89f5312 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2f89600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c056030 con 0x557415296ed0 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f2f8c005600 con 0x557415296ed0 2024-04-02T17:27:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.776+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f90037070 con 0x557415296ed0 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.777+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2f8c005000 con 0x557415296ed0 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.777+0000 7f2fa9990740 10 client.16983 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.777+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5574152a8c50 con 0x557415296ed0 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.777+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90037070 con 0x557415296ed0 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.777+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f2f8c004b70 con 0x557415296ed0 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 10 client.16983 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 20 client.16983 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574153144d0 0x5574153168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 4 client.16983 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:27:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55741520a2b0 con 0x5574153144d0 2024-04-02T17:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.778+0000 7f2f9f400640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574153144d0 0x5574153168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.779+0000 7f2f9f400640 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574153144d0 0x5574153168f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1a6af7a25fceaae server_cookie=b2ca2f31249c05b in_seq=0 out_seq=0 2024-04-02T17:27:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.779+0000 7f2f89600640 10 client.16983 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.952+0000 7f2f89600640 1 -- 172.21.15.59:0/2875582475 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55741520a2b0 con 0x5574153144d0 2024-04-02T17:27:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.952+0000 7f2f89600640 10 client.16983 handle_command_reply: tid=0 2024-04-02T17:27:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.952+0000 7f2fa9990740 1 client.16983 shutdown 2024-04-02T17:27:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.952+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 msgr2=0x5574152a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.952+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c003d50 tx=0x7f2f8c02cd50 comp rx=0 tx=0).stop 2024-04-02T17:27:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 shutdown_connections 2024-04-02T17:27:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5574153144d0 0x5574153168f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2f89600640 0 client.16983 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574152a6f80 0x5574152a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55741529ca30 0x5574152a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 --2- 172.21.15.59:0/2875582475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557415296ed0 0x5574152a5af0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 >> 172.21.15.59:0/2875582475 conn(0x557415292440 msgr2=0x557415290db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.953+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 shutdown_connections 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.954+0000 7f2fa9990740 1 -- 172.21.15.59:0/2875582475 wait complete. 2024-04-02T17:27:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.955+0000 7f2fa9990740 20 client.16983 trim_cache size 0 max 16384 2024-04-02T17:27:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:27:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:27:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:27:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:27:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 msgr2=0x7f2f840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 0x7f2f840a6190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0197390 tx=0x7f2f9000b4f0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 msgr2=0x7f2fa01ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b990 tx=0x7f2f9800be50 comp rx=0 tx=0).stop 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 shutdown_connections 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa019b420 0x7f2fa01ae9a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa0105910 0x7f2fa01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2f840a3cd0 0x7f2f840a6190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 --2- 172.21.15.59:0/418458715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104f50 0x7f2fa0195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 >> 172.21.15.59:0/418458715 conn(0x7f2fa00ff2f0 msgr2=0x7f2fa0077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.957+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 shutdown_connections 2024-04-02T17:27:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:24.958+0000 7f2fa6c00640 1 -- 172.21.15.59:0/418458715 wait complete. 2024-04-02T17:27:25.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 44576 49.42 MB/sec warmup 52 sec latency 140.280 ms 2024-04-02T17:27:25.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:25 smithi059 ceph-mon[31664]: pgmap v818: 97 pgs: 97 active+clean; 430 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 757 op/s 2024-04-02T17:27:25.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:25 smithi164 ceph-mon[36499]: pgmap v818: 97 pgs: 97 active+clean; 430 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 757 op/s 2024-04-02T17:27:25.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:25 smithi154 ceph-mon[36479]: pgmap v818: 97 pgs: 97 active+clean; 430 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 757 op/s 2024-04-02T17:27:26.589 INFO:tasks.workunit.client.0.smithi059.stdout: 10 45388 49.68 MB/sec warmup 53 sec latency 36.431 ms 2024-04-02T17:27:27.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 45788 48.93 MB/sec warmup 54 sec latency 188.004 ms 2024-04-02T17:27:27.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:27 smithi059 ceph-mon[31664]: pgmap v819: 97 pgs: 97 active+clean; 438 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 697 op/s 2024-04-02T17:27:27.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:27 smithi164 ceph-mon[36499]: pgmap v819: 97 pgs: 97 active+clean; 438 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 697 op/s 2024-04-02T17:27:28.225 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:27 smithi154 ceph-mon[36479]: pgmap v819: 97 pgs: 97 active+clean; 438 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 697 op/s 2024-04-02T17:27:28.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 46687 48.76 MB/sec warmup 55 sec latency 185.697 ms 2024-04-02T17:27:28.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:27:28.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:27:28.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:27:29.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 47864 49.06 MB/sec warmup 56 sec latency 43.557 ms 2024-04-02T17:27:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:29 smithi059 ceph-mon[31664]: pgmap v820: 97 pgs: 97 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 713 op/s 2024-04-02T17:27:29.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:29 smithi164 ceph-mon[36499]: pgmap v820: 97 pgs: 97 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 713 op/s 2024-04-02T17:27:29.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:29 smithi154 ceph-mon[36479]: pgmap v820: 97 pgs: 97 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 713 op/s 2024-04-02T17:27:30.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 48694 49.28 MB/sec warmup 57 sec latency 215.116 ms 2024-04-02T17:27:30.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:27:30.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:27:30.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:27:30.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:27:30.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:27:30.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:27:30.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:27:30.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:27:30.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:27:31.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 49108 48.78 MB/sec warmup 58 sec latency 132.915 ms 2024-04-02T17:27:31.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:31 smithi059 ceph-mon[31664]: pgmap v821: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 769 op/s 2024-04-02T17:27:31.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:31 smithi164 ceph-mon[36499]: pgmap v821: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 769 op/s 2024-04-02T17:27:31.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:31 smithi154 ceph-mon[36479]: pgmap v821: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 769 op/s 2024-04-02T17:27:32.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 49744 48.29 MB/sec warmup 59 sec latency 283.180 ms 2024-04-02T17:27:33.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 50954 48.53 MB/sec warmup 60 sec latency 41.250 ms 2024-04-02T17:27:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:33 smithi059 ceph-mon[31664]: pgmap v822: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 695 op/s 2024-04-02T17:27:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:33 smithi164 ceph-mon[36499]: pgmap v822: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 695 op/s 2024-04-02T17:27:33.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:33 smithi154 ceph-mon[36479]: pgmap v822: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 695 op/s 2024-04-02T17:27:34.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 52100 49.01 MB/sec warmup 61 sec latency 26.134 ms 2024-04-02T17:27:35.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 52757 48.75 MB/sec warmup 62 sec latency 52.758 ms 2024-04-02T17:27:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:35 smithi059 ceph-mon[31664]: pgmap v823: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.6 MiB/s wr, 852 op/s 2024-04-02T17:27:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:35 smithi164 ceph-mon[36499]: pgmap v823: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.6 MiB/s wr, 852 op/s 2024-04-02T17:27:35.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:35 smithi154 ceph-mon[36479]: pgmap v823: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.6 MiB/s wr, 852 op/s 2024-04-02T17:27:36.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 52859 48.09 MB/sec warmup 63 sec latency 760.958 ms 2024-04-02T17:27:37.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 52964 47.38 MB/sec warmup 64 sec latency 671.294 ms 2024-04-02T17:27:37.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:37 smithi059 ceph-mon[31664]: pgmap v824: 97 pgs: 97 active+clean; 465 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 775 op/s 2024-04-02T17:27:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:37 smithi164 ceph-mon[36499]: pgmap v824: 97 pgs: 97 active+clean; 465 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 775 op/s 2024-04-02T17:27:37.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:37 smithi154 ceph-mon[36479]: pgmap v824: 97 pgs: 97 active+clean; 465 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 775 op/s 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.185+0000 7f8084a00640 1 -- 172.21.15.59:0/1867981421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808010e100 msgr2=0x7f80801104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.185+0000 7f8084a00640 1 --2- 172.21.15.59:0/1867981421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808010e100 0x7f80801104f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8078009c10 tx=0x7f807802bee0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 -- 172.21.15.59:0/1867981421 shutdown_connections 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 --2- 172.21.15.59:0/1867981421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8080074560 0x7f80800749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 --2- 172.21.15.59:0/1867981421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808010e100 0x7f80801104f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 --2- 172.21.15.59:0/1867981421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8080075b40 0x7f8080073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 -- 172.21.15.59:0/1867981421 >> 172.21.15.59:0/1867981421 conn(0x7f80800fd0c0 msgr2=0x7f80800ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 -- 172.21.15.59:0/1867981421 shutdown_connections 2024-04-02T17:27:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.186+0000 7f8084a00640 1 -- 172.21.15.59:0/1867981421 wait complete. 2024-04-02T17:27:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.187+0000 7f8084a00640 1 Processor -- start 2024-04-02T17:27:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.187+0000 7f8084a00640 1 -- start start 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8080074560 0x7f808019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 0x7f80801a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113d90 con 0x7f808010e100 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113c10 con 0x7f8080075b40 2024-04-02T17:27:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f8084a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8080113f10 con 0x7f8080074560 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f807d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f807d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45522/0 (socket says 172.21.15.59:45522) 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.188+0000 7f807d400640 1 -- 172.21.15.59:0/2581758616 learned_addr learned my addr 172.21.15.59:0/2581758616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 -- 172.21.15.59:0/2581758616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8080074560 msgr2=0x7f808019e350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8080074560 0x7f808019e350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 -- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 msgr2=0x7f80801a5900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807e800640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 0x7f80801a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 0x7f80801a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 -- 172.21.15.59:0/2581758616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80801a5f70 con 0x7f8080075b40 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807e800640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 0x7f80801a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.189+0000 7f807d400640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d950 tx=0x7f806c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.190+0000 7f8076a00640 1 -- 172.21.15.59:0/2581758616 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c017030 con 0x7f8080075b40 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.190+0000 7f8076a00640 1 -- 172.21.15.59:0/2581758616 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806c004020 con 0x7f8080075b40 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.190+0000 7f8076a00640 1 -- 172.21.15.59:0/2581758616 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c003d60 con 0x7f8080075b40 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.190+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a6260 con 0x7f8080075b40 2024-04-02T17:27:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.190+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a6760 con 0x7f8080075b40 2024-04-02T17:27:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.195+0000 7f8087792740 1 Processor -- start 2024-04-02T17:27:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.195+0000 7f8087792740 1 -- start start 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e03350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f67d68d70 con 0x564f67e02f30 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f67d112a0 con 0x564f67e08a90 2024-04-02T17:27:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f8087792740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564f67d64d20 con 0x564f67e03920 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f807e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e03350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.196+0000 7f807de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55184/0 (socket says 172.21.15.59:55184) 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 -- 172.21.15.59:0/518490699 learned_addr learned my addr 172.21.15.59:0/518490699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f8076a00640 1 -- 172.21.15.59:0/2581758616 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f806c01a070 con 0x7f8080075b40 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 -- 172.21.15.59:0/518490699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 msgr2=0x564f67e0af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 -- 172.21.15.59:0/518490699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 msgr2=0x564f67e03350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e03350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.197+0000 7f807de00640 1 -- 172.21.15.59:0/518490699 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f67d25840 con 0x564f67e03920 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.198+0000 7f807de00640 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8070009a50 tx=0x7f8070009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5906a60322e1296e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.198+0000 7f806b000640 1 -- 172.21.15.59:0/518490699 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807000ec90 con 0x564f67e03920 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.198+0000 7f806b000640 1 -- 172.21.15.59:0/518490699 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f807000ee30 con 0x564f67e03920 2024-04-02T17:27:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.198+0000 7f806b000640 1 -- 172.21.15.59:0/518490699 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070010690 con 0x564f67e03920 2024-04-02T17:27:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f8076a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 0x7f80640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f8076a00640 1 -- 172.21.15.59:0/2581758616 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f806c0d00b0 con 0x7f8080075b40 2024-04-02T17:27:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f807e800640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 0x7f80640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f8087792740 1 -- 172.21.15.59:0/518490699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 msgr2=0x564f67e08420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f8087792740 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8070009a50 tx=0x7f8070009f10 comp rx=0 tx=0).stop 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.200+0000 7f8087792740 1 -- 172.21.15.59:0/518490699 shutdown_connections 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.201+0000 7f8087792740 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.201+0000 7f8087792740 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e08420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.201+0000 7f8087792740 1 --2- 172.21.15.59:0/518490699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e03350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.201+0000 7f8087792740 1 -- 172.21.15.59:0/518490699 >> 172.21.15.59:0/518490699 conn(0x564f67dfe4a0 msgr2=0x564f67dfe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.201+0000 7f807e800640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 0x7f80640a6300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8078006010 tx=0x7f807802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.204+0000 7f8087792740 1 -- 172.21.15.59:0/518490699 shutdown_connections 2024-04-02T17:27:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.204+0000 7f8087792740 1 -- 172.21.15.59:0/518490699 wait complete. 2024-04-02T17:27:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.204+0000 7f8087792740 1 -- start start 2024-04-02T17:27:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 Processor -- start 2024-04-02T17:27:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 0x564f67e0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f807de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f807d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 0x564f67e0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f807de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55186/0 (socket says 172.21.15.59:55186) 2024-04-02T17:27:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f807d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 0x564f67e0d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45536/0 (socket says 172.21.15.59:45536) 2024-04-02T17:27:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f807de00640 1 -- 172.21.15.59:0/347489376 learned_addr learned my addr 172.21.15.59:0/347489376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f67e0ba40 con 0x564f67e08a90 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f67d24660 con 0x564f67e03920 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.205+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564f67e0bbc0 con 0x564f67e02f30 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f8087792740 10 client.0 _ll_register_callbacks cb 0x564f67d24050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807e800640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 -- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 msgr2=0x564f67e0d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 0x564f67e0d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 -- 172.21.15.59:0/347489376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 msgr2=0x564f67e14a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e14a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f67d33e80 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f807de00640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f807000e7e0 tx=0x7f8070004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded4477c89f51631 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.206+0000 7f805f400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070026020 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8070004fc0 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807000ee30 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8070031050 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f8087792740 10 client.27103 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.207+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564f67a7d210 con 0x564f67e02f30 2024-04-02T17:27:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.208+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f807000ee30 con 0x564f67e02f30 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.208+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f8070010430 con 0x564f67e02f30 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 10 client.27103 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 10 client.27103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 20 client.27103 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e18fc0 0x564f67e1b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 4 client.27103 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f67d18cd0 con 0x564f67e18fc0 2024-04-02T17:27:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.209+0000 7f807d400640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e18fc0 0x564f67e1b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.211+0000 7f807d400640 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e18fc0 0x564f67e1b3b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c268f022e53a6960 server_cookie=4ad3bd837b3065dd in_seq=0 out_seq=0 2024-04-02T17:27:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.211+0000 7f805f400640 10 client.27103 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.212+0000 7f805f400640 1 -- 172.21.15.59:0/347489376 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f67d18cd0 con 0x564f67e18fc0 2024-04-02T17:27:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.212+0000 7f805f400640 10 client.27103 handle_command_reply: tid=0 2024-04-02T17:27:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.212+0000 7f8087792740 1 client.27103 shutdown 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 msgr2=0x564f67e0d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f807000e7e0 tx=0x7f8070004f50 comp rx=0 tx=0).stop 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 shutdown_connections 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e18fc0 0x564f67e1b3b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e03920 0x564f67e0d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 --2- 172.21.15.59:0/347489376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e02f30 0x564f67e0d4a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 >> 172.21.15.59:0/347489376 conn(0x564f67dfe4a0 msgr2=0x564f67dfd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f805f400640 0 client.27103 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 shutdown_connections 2024-04-02T17:27:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.213+0000 7f8087792740 1 -- 172.21.15.59:0/347489376 wait complete. 2024-04-02T17:27:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.214+0000 7f8087792740 20 client.27103 trim_cache size 0 max 16384 2024-04-02T17:27:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 Processor -- start 2024-04-02T17:27:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 -- start start 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e0cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80801af830 con 0x564f67e02f30 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80801b14f0 con 0x564f67e08a90 2024-04-02T17:27:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.239+0000 7f8087792740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80801aff30 con 0x564f67e03920 2024-04-02T17:27:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.240+0000 7f807de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e0cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.240+0000 7f807d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.240+0000 7f807d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55196/0 (socket says 172.21.15.59:55196) 2024-04-02T17:27:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.240+0000 7f807d400640 1 -- 172.21.15.59:0/287342843 learned_addr learned my addr 172.21.15.59:0/287342843 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.240+0000 7f807e800640 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 -- 172.21.15.59:0/287342843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 msgr2=0x564f67e149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 -- 172.21.15.59:0/287342843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 msgr2=0x564f67e0cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e0cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 -- 172.21.15.59:0/287342843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f67b49620 con 0x564f67e08a90 2024-04-02T17:27:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807d400640 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.242+0000 7f807e800640 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0d000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f807802c420 tx=0x7f807803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a5a364f08153c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.243+0000 7f806a600640 1 -- 172.21.15.59:0/287342843 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078003c30 con 0x564f67e08a90 2024-04-02T17:27:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.243+0000 7f806a600640 1 -- 172.21.15.59:0/287342843 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078052070 con 0x564f67e08a90 2024-04-02T17:27:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.243+0000 7f806a600640 1 -- 172.21.15.59:0/287342843 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078004790 con 0x564f67e08a90 2024-04-02T17:27:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.243+0000 7f8087792740 1 -- 172.21.15.59:0/287342843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 msgr2=0x564f67e0d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.243+0000 7f8087792740 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0d000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f807802c420 tx=0x7f807803f000 comp rx=0 tx=0).stop 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 -- 172.21.15.59:0/287342843 shutdown_connections 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e08a90 0x564f67e0d000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 --2- 172.21.15.59:0/287342843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e02f30 0x564f67e0cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 -- 172.21.15.59:0/287342843 >> 172.21.15.59:0/287342843 conn(0x564f67dfe4a0 msgr2=0x564f67e01700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 -- 172.21.15.59:0/287342843 shutdown_connections 2024-04-02T17:27:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.244+0000 7f8087792740 1 -- 172.21.15.59:0/287342843 wait complete. 2024-04-02T17:27:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.245+0000 7f8087792740 1 -- start start 2024-04-02T17:27:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.245+0000 7f8087792740 1 Processor -- start 2024-04-02T17:27:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.246+0000 7f8087792740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.246+0000 7f8087792740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.246+0000 7f807de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.246+0000 7f807de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:45548/0 (socket says 172.21.15.59:45548) 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f807de00640 1 -- 172.21.15.59:0/4148551691 learned_addr learned my addr 172.21.15.59:0/4148551691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80801b0d60 con 0x564f67e08a90 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f807d400640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80801b0630 con 0x564f67e02f30 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f807e800640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ad810 con 0x564f67e03920 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.247+0000 7f8087792740 10 client.0 _ll_register_callbacks cb 0x7f8070004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.248+0000 7f807de00640 1 -- 172.21.15.59:0/4148551691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 msgr2=0x564f67e10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.248+0000 7f807de00640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.248+0000 7f807de00640 1 -- 172.21.15.59:0/4148551691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 msgr2=0x564f67e13240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.248+0000 7f807de00640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e13240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.248+0000 7f807de00640 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f67e13830 con 0x564f67e02f30 2024-04-02T17:27:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f807de00640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8070001eb0 tx=0x7f80700048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7092f002bb4c63f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f805f400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:27:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070026070 con 0x564f67e02f30 2024-04-02T17:27:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8070010a70 con 0x564f67e02f30 2024-04-02T17:27:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8078052070 con 0x564f67e02f30 2024-04-02T17:27:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f807001d040 con 0x564f67e02f30 2024-04-02T17:27:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f8087792740 10 client.27095 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.249+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564f67e14b20 con 0x564f67e02f30 2024-04-02T17:27:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078052070 con 0x564f67e02f30 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f807003f570 con 0x564f67e02f30 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 10 client.27095 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 10 client.27095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 20 client.27095 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e90990 0x564f67e92db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 4 client.27095 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.250+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f67e04930 con 0x564f67e90990 2024-04-02T17:27:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.251+0000 7f807d400640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e90990 0x564f67e92db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.253+0000 7f807d400640 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e90990 0x564f67e92db0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41c772b01392f8d server_cookie=406bb94d3ef73fef in_seq=0 out_seq=0 2024-04-02T17:27:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.253+0000 7f805f400640 10 client.27095 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.527+0000 7f805f400640 1 -- 172.21.15.59:0/4148551691 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f67e04930 con 0x564f67e90990 2024-04-02T17:27:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.527+0000 7f805f400640 10 client.27095 handle_command_reply: tid=0 2024-04-02T17:27:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.527+0000 7f8087792740 1 client.27095 shutdown 2024-04-02T17:27:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.527+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 msgr2=0x564f67e0fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.527+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8070001eb0 tx=0x7f80700048d0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 shutdown_connections 2024-04-02T17:27:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564f67e90990 0x564f67e92db0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f67e08a90 0x564f67e13240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564f67e03920 0x564f67e10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 --2- 172.21.15.59:0/4148551691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f67e02f30 0x564f67e0fce0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 >> 172.21.15.59:0/4148551691 conn(0x564f67dfe4a0 msgr2=0x564f67dfce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f805f400640 0 client.27095 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 shutdown_connections 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 1 -- 172.21.15.59:0/4148551691 wait complete. 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.528+0000 7f8087792740 20 client.27095 trim_cache size 0 max 16384 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:27:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:27:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 msgr2=0x7f80640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 0x7f80640a6300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8078006010 tx=0x7f807802e040 comp rx=0 tx=0).stop 2024-04-02T17:27:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 msgr2=0x7f808019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d950 tx=0x7f806c00de10 comp rx=0 tx=0).stop 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 shutdown_connections 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f80640a3e40 0x7f80640a6300 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f808010e100 0x7f80801a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8080075b40 0x7f808019e890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 --2- 172.21.15.59:0/2581758616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8080074560 0x7f808019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 >> 172.21.15.59:0/2581758616 conn(0x7f80800fd0c0 msgr2=0x7f80801014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 shutdown_connections 2024-04-02T17:27:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:38.530+0000 7f8084a00640 1 -- 172.21.15.59:0/2581758616 wait complete. 2024-04-02T17:27:38.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 53394 46.86 MB/sec warmup 65 sec latency 143.465 ms 2024-04-02T17:27:39.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 54128 46.67 MB/sec warmup 66 sec latency 162.313 ms 2024-04-02T17:27:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:39 smithi059 ceph-mon[31664]: pgmap v825: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-04-02T17:27:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:39 smithi164 ceph-mon[36499]: pgmap v825: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-04-02T17:27:39.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:39 smithi154 ceph-mon[36479]: pgmap v825: 97 pgs: 97 active+clean; 509 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 886 op/s 2024-04-02T17:27:40.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 55249 47.05 MB/sec warmup 67 sec latency 62.043 ms 2024-04-02T17:27:41.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 56004 47.04 MB/sec warmup 68 sec latency 60.086 ms 2024-04-02T17:27:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:41 smithi059 ceph-mon[31664]: pgmap v826: 97 pgs: 97 active+clean; 509 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-04-02T17:27:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:41 smithi164 ceph-mon[36499]: pgmap v826: 97 pgs: 97 active+clean; 509 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-04-02T17:27:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:41 smithi154 ceph-mon[36479]: pgmap v826: 97 pgs: 97 active+clean; 509 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-04-02T17:27:42.590 INFO:tasks.workunit.client.0.smithi059.stdout: 10 56788 46.83 MB/sec warmup 69 sec latency 45.174 ms 2024-04-02T17:27:43.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 57812 46.91 MB/sec warmup 70 sec latency 32.805 ms 2024-04-02T17:27:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:43 smithi059 ceph-mon[31664]: pgmap v827: 97 pgs: 97 active+clean; 496 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-04-02T17:27:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:43 smithi164 ceph-mon[36499]: pgmap v827: 97 pgs: 97 active+clean; 496 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-04-02T17:27:43.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:43 smithi154 ceph-mon[36479]: pgmap v827: 97 pgs: 97 active+clean; 496 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-04-02T17:27:44.591 INFO:tasks.workunit.client.0.smithi059.stdout: 10 58787 47.17 MB/sec warmup 71 sec latency 72.024 ms 2024-04-02T17:27:45.591 INFO:tasks.workunit.client.0.smithi059.stdout: 10 59555 46.98 MB/sec warmup 72 sec latency 43.108 ms 2024-04-02T17:27:45.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:45 smithi059 ceph-mon[31664]: pgmap v828: 97 pgs: 97 active+clean; 453 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-04-02T17:27:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:45 smithi164 ceph-mon[36499]: pgmap v828: 97 pgs: 97 active+clean; 453 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-04-02T17:27:45.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:45 smithi154 ceph-mon[36479]: pgmap v828: 97 pgs: 97 active+clean; 453 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-04-02T17:27:46.591 INFO:tasks.workunit.client.0.smithi059.stdout: 10 60492 46.95 MB/sec warmup 73 sec latency 47.995 ms 2024-04-02T17:27:47.591 INFO:tasks.workunit.client.0.smithi059.stdout: 10 61590 47.18 MB/sec warmup 74 sec latency 48.783 ms 2024-04-02T17:27:47.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:47 smithi059 ceph-mon[31664]: pgmap v829: 97 pgs: 97 active+clean; 457 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-04-02T17:27:47.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:47 smithi164 ceph-mon[36499]: pgmap v829: 97 pgs: 97 active+clean; 457 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-04-02T17:27:47.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:47 smithi154 ceph-mon[36479]: pgmap v829: 97 pgs: 97 active+clean; 457 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.17k op/s 2024-04-02T17:27:48.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 62497 47.33 MB/sec warmup 75 sec latency 48.354 ms 2024-04-02T17:27:49.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 63379 47.21 MB/sec warmup 76 sec latency 37.275 ms 2024-04-02T17:27:49.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:49 smithi059 ceph-mon[31664]: pgmap v830: 97 pgs: 97 active+clean; 466 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:27:49.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:49 smithi164 ceph-mon[36499]: pgmap v830: 97 pgs: 97 active+clean; 466 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:27:49.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:49 smithi154 ceph-mon[36479]: pgmap v830: 97 pgs: 97 active+clean; 466 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:27:50.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 64311 47.23 MB/sec warmup 77 sec latency 46.190 ms 2024-04-02T17:27:51.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 65357 47.32 MB/sec warmup 78 sec latency 52.810 ms 2024-04-02T17:27:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 -- 172.21.15.59:0/2100238942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 msgr2=0x7f74541031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 --2- 172.21.15.59:0/2100238942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541031a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7440009900 tx=0x7f744002bc30 comp rx=0 tx=0).stop 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 -- 172.21.15.59:0/2100238942 shutdown_connections 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 --2- 172.21.15.59:0/2100238942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f7454109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 --2- 172.21.15.59:0/2100238942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541031a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 --2- 172.21.15.59:0/2100238942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 -- 172.21.15.59:0/2100238942 >> 172.21.15.59:0/2100238942 conn(0x7f74540fd0c0 msgr2=0x7f74540ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 -- 172.21.15.59:0/2100238942 shutdown_connections 2024-04-02T17:27:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.737+0000 7f745a200640 1 -- 172.21.15.59:0/2100238942 wait complete. 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.738+0000 7f745a200640 1 Processor -- start 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 -- start start 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f745419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f745410b620 con 0x7f74541036e0 2024-04-02T17:27:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f745410b4a0 con 0x7f7454101b30 2024-04-02T17:27:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f745a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f745410b7a0 con 0x7f7454102d20 2024-04-02T17:27:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f7453400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f7453400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59904/0 (socket says 172.21.15.59:59904) 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f7453400640 1 -- 172.21.15.59:0/4177403430 learned_addr learned my addr 172.21.15.59:0/4177403430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.739+0000 7f7452a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7453e00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f745419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 -- 172.21.15.59:0/4177403430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 msgr2=0x7f7454195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 -- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 msgr2=0x7f745419d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f745419d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 -- 172.21.15.59:0/4177403430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745419d7d0 con 0x7f7454102d20 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7453e00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f745419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:27:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.740+0000 7f7452a00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541960f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7440036040 tx=0x7f7440005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.741+0000 7f744be00640 1 -- 172.21.15.59:0/4177403430 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440043020 con 0x7f7454102d20 2024-04-02T17:27:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.741+0000 7f744be00640 1 -- 172.21.15.59:0/4177403430 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7440004410 con 0x7f7454102d20 2024-04-02T17:27:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.741+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745419da60 con 0x7f7454102d20 2024-04-02T17:27:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.741+0000 7f744be00640 1 -- 172.21.15.59:0/4177403430 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440003df0 con 0x7f7454102d20 2024-04-02T17:27:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.741+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745419df80 con 0x7f7454102d20 2024-04-02T17:27:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.746+0000 7f744be00640 1 -- 172.21.15.59:0/4177403430 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7440046020 con 0x7f7454102d20 2024-04-02T17:27:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.748+0000 7f744be00640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 0x7f74380aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.749+0000 7f744be00640 1 -- 172.21.15.59:0/4177403430 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f74400f2620 con 0x7f7454102d20 2024-04-02T17:27:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.749+0000 7f7453400640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 0x7f74380aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.749+0000 7f745ce3b740 1 Processor -- start 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 -- start start 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd78bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd79240 0x5606ebd7b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5606ebcd0d70 con 0x5606ebd79240 2024-04-02T17:27:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5606ebc792a0 con 0x5606ebd740d0 2024-04-02T17:27:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f745ce3b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606ebcccd20 con 0x5606ebd736e0 2024-04-02T17:27:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f7453400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f7453400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd73b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58440/0 (socket says 172.21.15.59:58440) 2024-04-02T17:27:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.750+0000 7f7453400640 1 -- 172.21.15.59:0/3517809902 learned_addr learned my addr 172.21.15.59:0/3517809902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd78bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7453400640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 0x7f74380aa520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f744c009fa0 tx=0x7f744c0096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 -- 172.21.15.59:0/3517809902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 msgr2=0x5606ebd73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 -- 172.21.15.59:0/3517809902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd79240 msgr2=0x5606ebd7b6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd79240 0x5606ebd7b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 -- 172.21.15.59:0/3517809902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ebc8d840 con 0x5606ebd740d0 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.751+0000 7f7452a00640 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd78bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7440009900 tx=0x7f74400056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95c7a9bc77bbe5c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.752+0000 7f743fe00640 1 -- 172.21.15.59:0/3517809902 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400be840 con 0x5606ebd740d0 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.752+0000 7f743fe00640 1 -- 172.21.15.59:0/3517809902 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74400c2bf0 con 0x5606ebd740d0 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.752+0000 7f743fe00640 1 -- 172.21.15.59:0/3517809902 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400baeb0 con 0x5606ebd740d0 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.753+0000 7f745ce3b740 1 -- 172.21.15.59:0/3517809902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 msgr2=0x5606ebd78bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.753+0000 7f745ce3b740 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd78bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7440009900 tx=0x7f74400056c0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 -- 172.21.15.59:0/3517809902 shutdown_connections 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd79240 0x5606ebd7b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd78bd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 --2- 172.21.15.59:0/3517809902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 -- 172.21.15.59:0/3517809902 >> 172.21.15.59:0/3517809902 conn(0x5606ebd6ec50 msgr2=0x5606ebd6f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 -- 172.21.15.59:0/3517809902 shutdown_connections 2024-04-02T17:27:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.754+0000 7f745ce3b740 1 -- 172.21.15.59:0/3517809902 wait complete. 2024-04-02T17:27:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.755+0000 7f745ce3b740 1 -- start start 2024-04-02T17:27:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.756+0000 7f745ce3b740 1 Processor -- start 2024-04-02T17:27:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd79240 0x5606ebd95e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5606ebd8cfc0 con 0x5606ebd740d0 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5606ebc8c660 con 0x5606ebd736e0 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5606ebd8d140 con 0x5606ebd79240 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f745ce3b740 10 client.0 _ll_register_callbacks cb 0x5606ebc8c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f7452a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f7452a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33376/0 (socket says 172.21.15.59:33376) 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.757+0000 7f7452a00640 1 -- 172.21.15.59:0/187922922 learned_addr learned my addr 172.21.15.59:0/187922922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7453400640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7452a00640 1 -- 172.21.15.59:0/187922922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd79240 msgr2=0x5606ebd95e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7452a00640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd79240 0x5606ebd95e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7452a00640 1 -- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 msgr2=0x5606ebd8e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7452a00640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd8e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.758+0000 7f7452a00640 1 -- 172.21.15.59:0/187922922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ebc9be80 con 0x5606ebd740d0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f7452a00640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74400be540 tx=0x7f744002c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b7f4cceb73ba3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f743cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400db020 con 0x5606ebd740d0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f74400c7020 con 0x5606ebd740d0 2024-04-02T17:27:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74400c2bf0 con 0x5606ebd740d0 2024-04-02T17:27:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f74400c74b0 con 0x5606ebd740d0 2024-04-02T17:27:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.759+0000 7f745ce3b740 10 client.17004 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.760+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5606eb9e5210 con 0x5606ebd740d0 2024-04-02T17:27:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.760+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74400c2bf0 con 0x5606ebd740d0 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f74400bfc90 con 0x5606ebd740d0 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 10 client.17004 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 20 client.17004 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd9a550 0x5606ebd9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 4 client.17004 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606ebc80cd0 con 0x5606ebd9a550 2024-04-02T17:27:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.761+0000 7f7453400640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd9a550 0x5606ebd9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.762+0000 7f7453400640 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd9a550 0x5606ebd9c980 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28ff84de9b0c6f6 server_cookie=c3f763ed3da88302 in_seq=0 out_seq=0 2024-04-02T17:27:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.762+0000 7f743cc00640 10 client.17004 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.763+0000 7f743cc00640 1 -- 172.21.15.59:0/187922922 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606ebc80cd0 con 0x5606ebd9a550 2024-04-02T17:27:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.763+0000 7f743cc00640 10 client.17004 handle_command_reply: tid=0 2024-04-02T17:27:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.763+0000 7f745ce3b740 1 client.17004 shutdown 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 msgr2=0x5606ebd8ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74400be540 tx=0x7f744002c450 comp rx=0 tx=0).stop 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 shutdown_connections 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd9a550 0x5606ebd9c980 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebd8ee10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd79240 0x5606ebd95e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 --2- 172.21.15.59:0/187922922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 >> 172.21.15.59:0/187922922 conn(0x5606ebd6ec50 msgr2=0x5606ebd65640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f743cc00640 0 client.17004 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 shutdown_connections 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 1 -- 172.21.15.59:0/187922922 wait complete. 2024-04-02T17:27:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.764+0000 7f745ce3b740 20 client.17004 trim_cache size 0 max 16384 2024-04-02T17:27:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 Processor -- start 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 -- start start 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 0x5606ebd64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f745419e790 con 0x5606ebd6ec50 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f74541a0450 con 0x5606ebd740d0 2024-04-02T17:27:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f745ce3b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f745419ee90 con 0x5606ebd736e0 2024-04-02T17:27:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.795+0000 7f7453e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7453e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59920/0 (socket says 172.21.15.59:59920) 2024-04-02T17:27:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7453e00640 1 -- 172.21.15.59:0/1104733148 learned_addr learned my addr 172.21.15.59:0/1104733148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7453400640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 0x5606ebd64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 -- 172.21.15.59:0/1104733148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 msgr2=0x5606ebd652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 -- 172.21.15.59:0/1104733148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 msgr2=0x5606ebd64db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 0x5606ebd64db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7452a00640 1 -- 172.21.15.59:0/1104733148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ebab1620 con 0x5606ebd736e0 2024-04-02T17:27:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.796+0000 7f7453e00640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.797+0000 7f7453400640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 0x5606ebd64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.797+0000 7f7452a00640 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd95e10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7440005ab0 tx=0x7f7440000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20edb62ba8ef9026 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.797+0000 7f743f400640 1 -- 172.21.15.59:0/1104733148 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400db070 con 0x5606ebd736e0 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.797+0000 7f743f400640 1 -- 172.21.15.59:0/1104733148 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74400bcc00 con 0x5606ebd736e0 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.797+0000 7f743f400640 1 -- 172.21.15.59:0/1104733148 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400db070 con 0x5606ebd736e0 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.798+0000 7f745ce3b740 1 -- 172.21.15.59:0/1104733148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 msgr2=0x5606ebd95e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.798+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd95e10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7440005ab0 tx=0x7f7440000bc0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:51 smithi059 ceph-mon[31664]: pgmap v831: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-04-02T17:27:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.799+0000 7f745ce3b740 1 -- 172.21.15.59:0/1104733148 shutdown_connections 2024-04-02T17:27:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.799+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd740d0 0x5606ebd652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.799+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd736e0 0x5606ebd95e10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.799+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1104733148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd6ec50 0x5606ebd64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.799+0000 7f745ce3b740 1 -- 172.21.15.59:0/1104733148 >> 172.21.15.59:0/1104733148 conn(0x5606ebd8d7a0 msgr2=0x5606ebd8e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.801+0000 7f745ce3b740 1 -- 172.21.15.59:0/1104733148 shutdown_connections 2024-04-02T17:27:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.801+0000 7f745ce3b740 1 -- 172.21.15.59:0/1104733148 wait complete. 2024-04-02T17:27:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.802+0000 7f745ce3b740 1 -- start start 2024-04-02T17:27:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.803+0000 7f745ce3b740 1 Processor -- start 2024-04-02T17:27:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebdfeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f745419fcc0 con 0x5606ebd740d0 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f745419f590 con 0x5606ebd736e0 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7454072230 con 0x5606ebd6ec50 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f745ce3b740 10 client.0 _ll_register_callbacks cb 0x5606ebc8c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f7453400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.804+0000 7f7453400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58460/0 (socket says 172.21.15.59:58460) 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.805+0000 7f7453400640 1 -- 172.21.15.59:0/1887354133 learned_addr learned my addr 172.21.15.59:0/1887354133 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.805+0000 7f7453e00640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebdfeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.805+0000 7f7452a00640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 -- 172.21.15.59:0/1887354133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 msgr2=0x5606ebd92db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 -- 172.21.15.59:0/1887354133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 msgr2=0x5606ebdfeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebdfeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 -- 172.21.15.59:0/1887354133 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ebdff0b0 con 0x5606ebd736e0 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7453400640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f7452a00640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd932f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74400d7000 tx=0x7f744002c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d118fb0439af0220 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.806+0000 7f743cc00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.807+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400a3060 con 0x5606ebd736e0 2024-04-02T17:27:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.807+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f74400c7220 con 0x5606ebd736e0 2024-04-02T17:27:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.807+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5606ebd750e0 con 0x5606ebd736e0 2024-04-02T17:27:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.808+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f74400db070 con 0x5606ebd736e0 2024-04-02T17:27:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.808+0000 7f745ce3b740 10 client.27113 fetch_fsmap learned FSMap version 45 2024-04-02T17:27:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.808+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5606ebdff0b0 con 0x5606ebd736e0 2024-04-02T17:27:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5606ebd750e0 con 0x5606ebd736e0 2024-04-02T17:27:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f74400bfc00 con 0x5606ebd736e0 2024-04-02T17:27:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 10 client.27113 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:27:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 10 client.27113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:27:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 20 client.27113 populate_metadata read hostname 'smithi059' 2024-04-02T17:27:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd79240 0x5606ebe029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:27:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 4 client.27113 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:27:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.809+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606ebcde2b0 con 0x5606ebd79240 2024-04-02T17:27:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.811+0000 7f7453400640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd79240 0x5606ebe029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:27:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.812+0000 7f7453400640 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd79240 0x5606ebe029b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed20e2b55f882e56 server_cookie=59eb0f6746d5cd40 in_seq=0 out_seq=0 2024-04-02T17:27:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.812+0000 7f743cc00640 10 client.27113 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:51.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:51 smithi164 ceph-mon[36499]: pgmap v831: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-04-02T17:27:51.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:51 smithi154 ceph-mon[36479]: pgmap v831: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-04-02T17:27:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.991+0000 7f743cc00640 1 -- 172.21.15.59:0/1887354133 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606ebcde2b0 con 0x5606ebd79240 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.991+0000 7f743cc00640 10 client.27113 handle_command_reply: tid=0 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.991+0000 7f745ce3b740 1 client.27113 shutdown 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.991+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 msgr2=0x5606ebd932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.991+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd932f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74400d7000 tx=0x7f744002c2b0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 shutdown_connections 2024-04-02T17:27:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5606ebd79240 0x5606ebe029b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606ebd740d0 0x5606ebdfeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5606ebd736e0 0x5606ebd932f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 --2- 172.21.15.59:0/1887354133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5606ebd6ec50 0x5606ebd92db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 >> 172.21.15.59:0/1887354133 conn(0x5606ebd8d7a0 msgr2=0x5606ebd64db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f743cc00640 0 client.27113 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 shutdown_connections 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 1 -- 172.21.15.59:0/1887354133 wait complete. 2024-04-02T17:27:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.992+0000 7f745ce3b740 20 client.27113 trim_cache size 0 max 16384 2024-04-02T17:27:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:27:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:27:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:27:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:27:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 msgr2=0x7f74380aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 0x7f74380aa520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f744c009fa0 tx=0x7f744c0096d0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 msgr2=0x7f74541960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541960f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7440036040 tx=0x7f7440005dd0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 shutdown_connections 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f74380a8060 0x7f74380aa520 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74541036e0 0x7f745419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7454102d20 0x7f74541960f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 --2- 172.21.15.59:0/4177403430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7454101b30 0x7f7454195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:27:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.995+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 >> 172.21.15.59:0/4177403430 conn(0x7f74540fd0c0 msgr2=0x7f7454105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:27:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.996+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 shutdown_connections 2024-04-02T17:27:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:27:51.996+0000 7f745a200640 1 -- 172.21.15.59:0/4177403430 wait complete. 2024-04-02T17:27:52.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 66131 47.44 MB/sec warmup 79 sec latency 116.921 ms 2024-04-02T17:27:53.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 66902 47.29 MB/sec warmup 80 sec latency 53.345 ms 2024-04-02T17:27:53.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:53 smithi059 ceph-mon[31664]: pgmap v832: 97 pgs: 97 active+clean; 481 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 818 op/s 2024-04-02T17:27:53.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:53 smithi164 ceph-mon[36499]: pgmap v832: 97 pgs: 97 active+clean; 481 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 818 op/s 2024-04-02T17:27:53.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:53 smithi154 ceph-mon[36479]: pgmap v832: 97 pgs: 97 active+clean; 481 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 818 op/s 2024-04-02T17:27:54.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 67665 47.17 MB/sec warmup 81 sec latency 79.153 ms 2024-04-02T17:27:55.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 68621 47.09 MB/sec warmup 82 sec latency 34.230 ms 2024-04-02T17:27:55.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:55 smithi059 ceph-mon[31664]: pgmap v833: 97 pgs: 97 active+clean; 484 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 815 op/s 2024-04-02T17:27:55.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:55 smithi164 ceph-mon[36499]: pgmap v833: 97 pgs: 97 active+clean; 484 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 815 op/s 2024-04-02T17:27:55.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:55 smithi154 ceph-mon[36479]: pgmap v833: 97 pgs: 97 active+clean; 484 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 815 op/s 2024-04-02T17:27:56.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 69726 47.42 MB/sec warmup 83 sec latency 49.610 ms 2024-04-02T17:27:57.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 70478 47.33 MB/sec warmup 84 sec latency 38.794 ms 2024-04-02T17:27:57.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:57 smithi059 ceph-mon[31664]: pgmap v834: 97 pgs: 97 active+clean; 492 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 686 op/s 2024-04-02T17:27:57.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:57 smithi164 ceph-mon[36499]: pgmap v834: 97 pgs: 97 active+clean; 492 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 686 op/s 2024-04-02T17:27:57.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:57 smithi154 ceph-mon[36479]: pgmap v834: 97 pgs: 97 active+clean; 492 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 686 op/s 2024-04-02T17:27:58.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 71229 47.21 MB/sec warmup 85 sec latency 83.296 ms 2024-04-02T17:27:59.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 72230 47.21 MB/sec warmup 86 sec latency 70.978 ms 2024-04-02T17:27:59.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:27:59 smithi059 ceph-mon[31664]: pgmap v835: 97 pgs: 97 active+clean; 496 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-04-02T17:27:59.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:27:59 smithi164 ceph-mon[36499]: pgmap v835: 97 pgs: 97 active+clean; 496 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-04-02T17:27:59.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:27:59 smithi154 ceph-mon[36479]: pgmap v835: 97 pgs: 97 active+clean; 496 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 686 op/s 2024-04-02T17:28:00.592 INFO:tasks.workunit.client.0.smithi059.stdout: 10 73219 47.38 MB/sec warmup 87 sec latency 36.432 ms 2024-04-02T17:28:01.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 74036 47.35 MB/sec warmup 88 sec latency 135.452 ms 2024-04-02T17:28:01.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:01 smithi059 ceph-mon[31664]: pgmap v836: 97 pgs: 97 active+clean; 509 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 707 op/s 2024-04-02T17:28:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:01 smithi164 ceph-mon[36499]: pgmap v836: 97 pgs: 97 active+clean; 509 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 707 op/s 2024-04-02T17:28:01.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:01 smithi154 ceph-mon[36479]: pgmap v836: 97 pgs: 97 active+clean; 509 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 707 op/s 2024-04-02T17:28:02.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 74525 47.12 MB/sec warmup 89 sec latency 262.762 ms 2024-04-02T17:28:03.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:02 smithi059 ceph-mon[31664]: pgmap v837: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 682 op/s 2024-04-02T17:28:03.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:02 smithi164 ceph-mon[36499]: pgmap v837: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 682 op/s 2024-04-02T17:28:03.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:02 smithi154 ceph-mon[36479]: pgmap v837: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 682 op/s 2024-04-02T17:28:03.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 75073 46.82 MB/sec warmup 90 sec latency 389.588 ms 2024-04-02T17:28:04.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 76111 46.95 MB/sec warmup 91 sec latency 121.360 ms 2024-04-02T17:28:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.180+0000 7f8cd3400640 1 -- 172.21.15.59:0/2030474616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 msgr2=0x7f8ccc1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.180+0000 7f8cd3400640 1 --2- 172.21.15.59:0/2030474616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc1027d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009860 tx=0x7f8cbc02bb70 comp rx=0 tx=0).stop 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 -- 172.21.15.59:0/2030474616 shutdown_connections 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 --2- 172.21.15.59:0/2030474616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 --2- 172.21.15.59:0/2030474616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 --2- 172.21.15.59:0/2030474616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc1027d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 -- 172.21.15.59:0/2030474616 >> 172.21.15.59:0/2030474616 conn(0x7f8ccc0fbfc0 msgr2=0x7f8ccc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 -- 172.21.15.59:0/2030474616 shutdown_connections 2024-04-02T17:28:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.181+0000 7f8cd3400640 1 -- 172.21.15.59:0/2030474616 wait complete. 2024-04-02T17:28:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.182+0000 7f8cd3400640 1 Processor -- start 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 -- start start 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc1a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113da0 con 0x7f8ccc1003a0 2024-04-02T17:28:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113c20 con 0x7f8ccc102d10 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd3400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113f20 con 0x7f8ccc1056c0 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc1a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56638/0 (socket says 172.21.15.59:56638) 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3502692871 learned_addr learned my addr 172.21.15.59:0/3502692871 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.183+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3502692871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 msgr2=0x7f8ccc1a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc1a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 msgr2=0x7f8ccc19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3502692871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc1a5f50 con 0x7f8ccc102d10 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400cce0 tx=0x7f8cc400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8cc9600640 1 -- 172.21.15.59:0/3502692871 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400f650 con 0x7f8ccc102d10 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1a6240 con 0x7f8ccc102d10 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8cc9600640 1 -- 172.21.15.59:0/3502692871 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc400fcd0 con 0x7f8ccc102d10 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.184+0000 7f8cc9600640 1 -- 172.21.15.59:0/3502692871 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400ec50 con 0x7f8ccc102d10 2024-04-02T17:28:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.185+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a67e0 con 0x7f8ccc102d10 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.189+0000 7f8cd61be740 1 Processor -- start 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cc9600640 1 -- 172.21.15.59:0/3502692871 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8cc400f7f0 con 0x7f8ccc102d10 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 -- start start 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae43a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae43cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556cae39ad70 con 0x556cae43aa30 2024-04-02T17:28:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556cae3432a0 con 0x556cae4358c0 2024-04-02T17:28:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd61be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556cae396d20 con 0x556cae434ed0 2024-04-02T17:28:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae43a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45210/0 (socket says 172.21.15.59:45210) 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.190+0000 7f8cd1400640 1 -- 172.21.15.59:0/934246292 learned_addr learned my addr 172.21.15.59:0/934246292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.191+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae43cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.191+0000 7f8cd1400640 1 -- 172.21.15.59:0/934246292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 msgr2=0x556cae43a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.191+0000 7f8cd1400640 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae43a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.191+0000 7f8cd1400640 1 -- 172.21.15.59:0/934246292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 msgr2=0x556cae43cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cc9600640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 0x7f8cb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cd1400640 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae43cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cd1400640 1 -- 172.21.15.59:0/934246292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cae357840 con 0x556cae434ed0 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cc9600640 1 -- 172.21.15.59:0/3502692871 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8cc40d0960 con 0x7f8ccc102d10 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae43cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 0x7f8cb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cd1400640 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009c40 tx=0x7f8cb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99d76c6db4f89d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.192+0000 7f8cc1c00640 1 -- 172.21.15.59:0/934246292 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8004230 con 0x556cae434ed0 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.193+0000 7f8cc1c00640 1 -- 172.21.15.59:0/934246292 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb80043d0 con 0x556cae434ed0 2024-04-02T17:28:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.193+0000 7f8cc1c00640 1 -- 172.21.15.59:0/934246292 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8019790 con 0x556cae434ed0 2024-04-02T17:28:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.193+0000 7f8cd61be740 1 -- 172.21.15.59:0/934246292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 msgr2=0x556cae4352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.193+0000 7f8cd61be740 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009c40 tx=0x7f8cb8011810 comp rx=0 tx=0).stop 2024-04-02T17:28:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 -- 172.21.15.59:0/934246292 shutdown_connections 2024-04-02T17:28:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae43cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae43a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 --2- 172.21.15.59:0/934246292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae4352f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 -- 172.21.15.59:0/934246292 >> 172.21.15.59:0/934246292 conn(0x556cae430440 msgr2=0x556cae430840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 -- 172.21.15.59:0/934246292 shutdown_connections 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 0x7f8cb00a6190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009830 tx=0x7f8cbc034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.194+0000 7f8cd61be740 1 -- 172.21.15.59:0/934246292 wait complete. 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.195+0000 7f8cd61be740 1 -- start start 2024-04-02T17:28:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.195+0000 7f8cd61be740 1 Processor -- start 2024-04-02T17:28:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae434ed0 0x556cae43f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae43aa30 0x556cae446940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556cae43d9e0 con 0x556cae434ed0 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556cae356660 con 0x556cae43aa30 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556cae43db60 con 0x556cae4358c0 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8cd61be740 10 client.0 _ll_register_callbacks cb 0x556cae356050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45216/0 (socket says 172.21.15.59:45216) 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.196+0000 7f8ccbe00640 1 -- 172.21.15.59:0/1547411813 learned_addr learned my addr 172.21.15.59:0/1547411813 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 -- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae43aa30 msgr2=0x556cae446940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae43aa30 0x556cae446940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 -- 172.21.15.59:0/1547411813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae434ed0 msgr2=0x556cae43f3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:28:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae434ed0 0x556cae43f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 -- 172.21.15.59:0/1547411813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cae365e80 con 0x556cae4358c0 2024-04-02T17:28:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40a5320 tx=0x7f8cc4005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40dec6fac062c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.197+0000 7f8cb6000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4087040 con 0x556cae4358c0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8cc400e3b0 con 0x556cae4358c0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb80043d0 con 0x556cae4358c0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8cc409c720 con 0x556cae4358c0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cd61be740 10 client.27136 fetch_fsmap learned FSMap version 45 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.198+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556cae0af210 con 0x556cae4358c0 2024-04-02T17:28:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb80043d0 con 0x556cae4358c0 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f8cc40bd070 con 0x556cae4358c0 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 10 client.27136 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 10 client.27136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 20 client.27136 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae45c2b0 0x556cae45e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 4 client.27136 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cae34acd0 con 0x556cae45c2b0 2024-04-02T17:28:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.199+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae45c2b0 0x556cae45e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.200+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae45c2b0 0x556cae45e750 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3b414b198d53ac server_cookie=fef6c571fa62409e in_seq=0 out_seq=0 2024-04-02T17:28:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.200+0000 7f8cb6000640 10 client.27136 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.201+0000 7f8cb6000640 1 -- 172.21.15.59:0/1547411813 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cae34acd0 con 0x556cae45c2b0 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.201+0000 7f8cb6000640 10 client.27136 handle_command_reply: tid=0 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.201+0000 7f8cd61be740 1 client.27136 shutdown 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 msgr2=0x556cae43f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40a5320 tx=0x7f8cc4005c50 comp rx=0 tx=0).stop 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 shutdown_connections 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae45c2b0 0x556cae45e750 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae43aa30 0x556cae446940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae4358c0 0x556cae43f900 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cb6000640 0 client.27136 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 --2- 172.21.15.59:0/1547411813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae434ed0 0x556cae43f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 >> 172.21.15.59:0/1547411813 conn(0x556cae430440 msgr2=0x556cae42f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 shutdown_connections 2024-04-02T17:28:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 1 -- 172.21.15.59:0/1547411813 wait complete. 2024-04-02T17:28:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.202+0000 7f8cd61be740 20 client.27136 trim_cache size 0 max 16384 2024-04-02T17:28:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.226+0000 7f8cd61be740 1 Processor -- start 2024-04-02T17:28:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.226+0000 7f8cd61be740 1 -- start start 2024-04-02T17:28:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.226+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae42edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd61be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1a7030 con 0x556cae43aa30 2024-04-02T17:28:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd61be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1a8cf0 con 0x556cae4358c0 2024-04-02T17:28:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd61be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1a7730 con 0x556cae434ed0 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae42edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45226/0 (socket says 172.21.15.59:45226) 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56672/0 (socket says 172.21.15.59:56672) 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd0a00640 1 -- 172.21.15.59:0/3666605682 learned_addr learned my addr 172.21.15.59:0/3666605682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3666605682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 msgr2=0x556cae43edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3666605682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 msgr2=0x556cae42edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae42edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8ccbe00640 1 -- 172.21.15.59:0/3666605682 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cae17b620 con 0x556cae4358c0 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd1400640 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae42edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.227+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.228+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4099560 tx=0x7f8cc4098f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5cc55697d000a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.228+0000 7f8cc1200640 1 -- 172.21.15.59:0/3666605682 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4087050 con 0x556cae4358c0 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.228+0000 7f8cc1200640 1 -- 172.21.15.59:0/3666605682 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc409d2e0 con 0x556cae4358c0 2024-04-02T17:28:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.228+0000 7f8cc1200640 1 -- 172.21.15.59:0/3666605682 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc409c540 con 0x556cae4358c0 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 -- 172.21.15.59:0/3666605682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 msgr2=0x556cae446900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4099560 tx=0x7f8cc4098f80 comp rx=0 tx=0).stop 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 -- 172.21.15.59:0/3666605682 shutdown_connections 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae43aa30 0x556cae42edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae4358c0 0x556cae446900 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4099560 tx=0x7f8cc4098f80 comp rx=0 tx=0).stop 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3666605682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae434ed0 0x556cae43edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.229+0000 7f8cd61be740 1 -- 172.21.15.59:0/3666605682 >> 172.21.15.59:0/3666605682 conn(0x556cae430440 msgr2=0x556cae43e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.230+0000 7f8cd61be740 1 -- 172.21.15.59:0/3666605682 shutdown_connections 2024-04-02T17:28:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.230+0000 7f8cd61be740 1 -- 172.21.15.59:0/3666605682 wait complete. 2024-04-02T17:28:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.230+0000 7f8cd61be740 1 -- start start 2024-04-02T17:28:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 Processor -- start 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae43aa30 0x556cae444340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae445350 0x556cae4457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1a8560 con 0x556cae445350 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1a7e30 con 0x556cae434ed0 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc072290 con 0x556cae43aa30 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.231+0000 7f8cd61be740 10 client.0 _ll_register_callbacks cb 0x7f8cc40d6240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56684/0 (socket says 172.21.15.59:56684) 2024-04-02T17:28:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 -- 172.21.15.59:0/3527211143 learned_addr learned my addr 172.21.15.59:0/3527211143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd1400640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae445350 0x556cae4457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae43aa30 0x556cae444340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 -- 172.21.15.59:0/3527211143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae43aa30 msgr2=0x556cae444340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae43aa30 0x556cae444340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 -- 172.21.15.59:0/3527211143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae445350 msgr2=0x556cae4457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae445350 0x556cae4457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 -- 172.21.15.59:0/3527211143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cae445d10 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.232+0000 7f8cd0a00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0349e0 tx=0x7f8cbc02bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a16c56ef1d43d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cb6000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc039040 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f8cbc005040 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc409d2e0 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc005240 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cd61be740 10 client.27125 fetch_fsmap learned FSMap version 45 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.233+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556cae4c2b40 con 0x556cae434ed0 2024-04-02T17:28:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.234+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc409d2e0 con 0x556cae434ed0 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8390+0+0 (secure 0 0 0) 0x7f8cbc049040 con 0x556cae434ed0 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 10 client.27125 fetch_fsmap finished waiting for FSMap version 45 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 10 client.27125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 20 client.27125 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae4c3560 0x556cae4c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 4 client.27125 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:28:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cae3a82b0 con 0x556cae4c3560 2024-04-02T17:28:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.235+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae4c3560 0x556cae4c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.236+0000 7f8ccbe00640 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae4c3560 0x556cae4c5980 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e700067c495712ed server_cookie=a4ed1d7b82a36e90 in_seq=0 out_seq=0 2024-04-02T17:28:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.236+0000 7f8cb6000640 10 client.27125 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:05.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:05 smithi059 ceph-mon[31664]: pgmap v838: 97 pgs: 97 active+clean; 536 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 776 op/s 2024-04-02T17:28:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:05 smithi164 ceph-mon[36499]: pgmap v838: 97 pgs: 97 active+clean; 536 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 776 op/s 2024-04-02T17:28:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:05 smithi154 ceph-mon[36479]: pgmap v838: 97 pgs: 97 active+clean; 536 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 776 op/s 2024-04-02T17:28:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.419+0000 7f8cb6000640 1 -- 172.21.15.59:0/3527211143 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cae3a82b0 con 0x556cae4c3560 2024-04-02T17:28:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.419+0000 7f8cb6000640 10 client.27125 handle_command_reply: tid=0 2024-04-02T17:28:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.419+0000 7f8cd61be740 1 client.27125 shutdown 2024-04-02T17:28:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 msgr2=0x556cae443e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0349e0 tx=0x7f8cbc02bb70 comp rx=0 tx=0).stop 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 shutdown_connections 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556cae4c3560 0x556cae4c5980 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556cae445350 0x556cae4457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae43aa30 0x556cae444340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 --2- 172.21.15.59:0/3527211143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556cae434ed0 0x556cae443e00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cb6000640 0 client.27125 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 >> 172.21.15.59:0/3527211143 conn(0x556cae430440 msgr2=0x556cae4336a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 shutdown_connections 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 1 -- 172.21.15.59:0/3527211143 wait complete. 2024-04-02T17:28:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.420+0000 7f8cd61be740 20 client.27125 trim_cache size 0 max 16384 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.421+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 msgr2=0x7f8cb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.421+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 0x7f8cb00a6190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009830 tx=0x7f8cbc034ab0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.421+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 msgr2=0x7f8ccc19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.421+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400cce0 tx=0x7f8cc400a620 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 shutdown_connections 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc1a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc102d10 0x7f8ccc19e870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8cb00a3cd0 0x7f8cb00a6190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 --2- 172.21.15.59:0/3502692871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ccc1003a0 0x7f8ccc19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 >> 172.21.15.59:0/3502692871 conn(0x7f8ccc0fbfc0 msgr2=0x7f8ccc1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.422+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 shutdown_connections 2024-04-02T17:28:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:05.424+0000 7f8cd3400640 1 -- 172.21.15.59:0/3502692871 wait complete. 2024-04-02T17:28:05.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 76893 46.92 MB/sec warmup 92 sec latency 110.548 ms 2024-04-02T17:28:06.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 77619 46.88 MB/sec warmup 93 sec latency 191.534 ms 2024-04-02T17:28:07.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:07 smithi059 ceph-mon[31664]: pgmap v839: 97 pgs: 97 active+clean; 539 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-04-02T17:28:07.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:07 smithi164 ceph-mon[36499]: pgmap v839: 97 pgs: 97 active+clean; 539 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-04-02T17:28:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:07 smithi154 ceph-mon[36479]: pgmap v839: 97 pgs: 97 active+clean; 539 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 727 op/s 2024-04-02T17:28:07.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 78451 46.78 MB/sec warmup 94 sec latency 51.717 ms 2024-04-02T17:28:08.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 78838 46.54 MB/sec warmup 95 sec latency 409.332 ms 2024-04-02T17:28:09.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:09 smithi164 ceph-mon[36499]: pgmap v840: 97 pgs: 97 active+clean; 534 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 789 op/s 2024-04-02T17:28:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:09 smithi154 ceph-mon[36479]: pgmap v840: 97 pgs: 97 active+clean; 534 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 789 op/s 2024-04-02T17:28:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:09 smithi059 ceph-mon[31664]: pgmap v840: 97 pgs: 97 active+clean; 534 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 789 op/s 2024-04-02T17:28:09.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 79075 46.15 MB/sec warmup 96 sec latency 713.018 ms 2024-04-02T17:28:10.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 79652 46.04 MB/sec warmup 97 sec latency 97.960 ms 2024-04-02T17:28:11.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:11 smithi164 ceph-mon[36499]: pgmap v841: 97 pgs: 97 active+clean; 574 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 908 op/s 2024-04-02T17:28:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:11 smithi154 ceph-mon[36479]: pgmap v841: 97 pgs: 97 active+clean; 574 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 908 op/s 2024-04-02T17:28:11.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:11 smithi059 ceph-mon[31664]: pgmap v841: 97 pgs: 97 active+clean; 574 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 908 op/s 2024-04-02T17:28:11.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 80481 46.07 MB/sec warmup 98 sec latency 54.807 ms 2024-04-02T17:28:12.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 81219 46.02 MB/sec warmup 99 sec latency 94.413 ms 2024-04-02T17:28:13.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:13 smithi164 ceph-mon[36499]: pgmap v842: 97 pgs: 97 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 939 op/s 2024-04-02T17:28:13.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:13 smithi154 ceph-mon[36479]: pgmap v842: 97 pgs: 97 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 939 op/s 2024-04-02T17:28:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:13 smithi059 ceph-mon[31664]: pgmap v842: 97 pgs: 97 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 939 op/s 2024-04-02T17:28:13.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 82069 45.96 MB/sec warmup 100 sec latency 52.186 ms 2024-04-02T17:28:14.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 83115 46.06 MB/sec warmup 101 sec latency 31.631 ms 2024-04-02T17:28:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:15 smithi154 ceph-mon[36479]: pgmap v843: 97 pgs: 97 active+clean; 577 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-04-02T17:28:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:15 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:15 smithi059 ceph-mon[31664]: pgmap v843: 97 pgs: 97 active+clean; 577 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-04-02T17:28:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:15 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:15 smithi164 ceph-mon[36499]: pgmap v843: 97 pgs: 97 active+clean; 577 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.18k op/s 2024-04-02T17:28:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:15 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:15.593 INFO:tasks.workunit.client.0.smithi059.stdout: 10 83966 46.09 MB/sec warmup 102 sec latency 56.061 ms 2024-04-02T17:28:16.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 84690 46.04 MB/sec warmup 103 sec latency 97.370 ms 2024-04-02T17:28:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:17 smithi154 ceph-mon[36479]: pgmap v844: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:28:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:17 smithi059 ceph-mon[31664]: pgmap v844: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:28:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:17 smithi164 ceph-mon[36499]: pgmap v844: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-04-02T17:28:17.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 85615 46.05 MB/sec warmup 104 sec latency 88.524 ms 2024-04-02T17:28:18.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 86691 46.20 MB/sec warmup 105 sec latency 43.652 ms 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.624+0000 7f5b4e600640 1 -- 172.21.15.59:0/308678072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48104f70 msgr2=0x7f5b48107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.624+0000 7f5b4e600640 1 --2- 172.21.15.59:0/308678072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48104f70 0x7f5b48107360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402bc40 comp rx=0 tx=0).stop 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 -- 172.21.15.59:0/308678072 shutdown_connections 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 --2- 172.21.15.59:0/308678072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 0x7f5b48109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 --2- 172.21.15.59:0/308678072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48104f70 0x7f5b48107360 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 --2- 172.21.15.59:0/308678072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48102640 0x7f5b48104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 -- 172.21.15.59:0/308678072 >> 172.21.15.59:0/308678072 conn(0x7f5b480fbf90 msgr2=0x7f5b480fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 -- 172.21.15.59:0/308678072 shutdown_connections 2024-04-02T17:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.625+0000 7f5b4e600640 1 -- 172.21.15.59:0/308678072 wait complete. 2024-04-02T17:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.626+0000 7f5b4e600640 1 Processor -- start 2024-04-02T17:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.626+0000 7f5b4e600640 1 -- start start 2024-04-02T17:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.626+0000 7f5b4e600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 0x7f5b48199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b4e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b4e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 0x7f5b481a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b4e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d790 con 0x7f5b481078a0 2024-04-02T17:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b4e600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d610 con 0x7f5b48102640 2024-04-02T17:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b4e600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810d910 con 0x7f5b48104f70 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 0x7f5b48199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41228/0 (socket says 172.21.15.59:41228) 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b46a00640 1 -- 172.21.15.59:0/628131118 learned_addr learned my addr 172.21.15.59:0/628131118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47e00640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 0x7f5b481a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47400640 1 -- 172.21.15.59:0/628131118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 msgr2=0x7f5b4819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47400640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47400640 1 -- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 msgr2=0x7f5b481a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.627+0000 7f5b47400640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 0x7f5b481a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b47400640 1 -- 172.21.15.59:0/628131118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b481a1c00 con 0x7f5b48102640 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b46a00640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b47400640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 0x7f5b48199fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e9f0 tx=0x7f5b3800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b3fe00640 1 -- 172.21.15.59:0/628131118 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800ccc0 con 0x7f5b48102640 2024-04-02T17:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b3fe00640 1 -- 172.21.15.59:0/628131118 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b38005020 con 0x7f5b48102640 2024-04-02T17:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b3fe00640 1 -- 172.21.15.59:0/628131118 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004a00 con 0x7f5b48102640 2024-04-02T17:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b481a1ef0 con 0x7f5b48102640 2024-04-02T17:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.628+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a22a0 con 0x7f5b48102640 2024-04-02T17:28:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.631+0000 7f5b513fd740 1 Processor -- start 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 -- start start 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a808c0 0x55b945a853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b9459e5d70 con 0x55b945a7fed0 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b94598e2a0 con 0x55b945a85a30 2024-04-02T17:28:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b513fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b9459e1d20 con 0x55b945a808c0 2024-04-02T17:28:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b3fe00640 1 -- 172.21.15.59:0/628131118 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b38004ba0 con 0x7f5b48102640 2024-04-02T17:28:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37376/0 (socket says 172.21.15.59:37376) 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b46a00640 1 -- 172.21.15.59:0/1797985250 learned_addr learned my addr 172.21.15.59:0/1797985250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.632+0000 7f5b47400640 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a808c0 0x55b945a853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 -- 172.21.15.59:0/1797985250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a808c0 msgr2=0x55b945a853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a808c0 0x55b945a853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 -- 172.21.15.59:0/1797985250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 msgr2=0x55b945a87ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 -- 172.21.15.59:0/1797985250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9459a2840 con 0x55b945a7fed0 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b46a00640 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a802f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009f10 tx=0x7f5b40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dbc9aef3c0360e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b3fe00640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 0x7f5b2c0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.633+0000 7f5b3cc00640 1 -- 172.21.15.59:0/1797985250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004d20 con 0x55b945a7fed0 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b3fe00640 1 -- 172.21.15.59:0/628131118 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b380d03f0 con 0x7f5b48102640 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b3cc00640 1 -- 172.21.15.59:0/1797985250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40004030 con 0x55b945a7fed0 2024-04-02T17:28:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b3cc00640 1 -- 172.21.15.59:0/1797985250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40005260 con 0x55b945a7fed0 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b47400640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 0x7f5b2c0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b513fd740 1 -- 172.21.15.59:0/1797985250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 msgr2=0x55b945a802f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a802f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009f10 tx=0x7f5b40011810 comp rx=0 tx=0).stop 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.634+0000 7f5b47400640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 0x7f5b2c0a6110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b38098a70 tx=0x7f5b380d3e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 -- 172.21.15.59:0/1797985250 shutdown_connections 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945a87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a808c0 0x55b945a853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1797985250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a802f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 -- 172.21.15.59:0/1797985250 >> 172.21.15.59:0/1797985250 conn(0x55b945a7b440 msgr2=0x55b945a7b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 -- 172.21.15.59:0/1797985250 shutdown_connections 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 -- 172.21.15.59:0/1797985250 wait complete. 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 -- start start 2024-04-02T17:28:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.635+0000 7f5b513fd740 1 Processor -- start 2024-04-02T17:28:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a85a30 0x55b945aa2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b945a999f0 con 0x55b945a808c0 2024-04-02T17:28:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b9459a1660 con 0x55b945a7fed0 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b945a99b70 con 0x55b945a85a30 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b513fd740 10 client.0 _ll_register_callbacks cb 0x55b9459a1050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36822/0 (socket says 172.21.15.59:36822) 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 -- 172.21.15.59:0/1849738051 learned_addr learned my addr 172.21.15.59:0/1849738051 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a85a30 0x55b945aa2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 -- 172.21.15.59:0/1849738051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a85a30 msgr2=0x55b945aa2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a85a30 0x55b945aa2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 -- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 msgr2=0x55b945a9b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b46a00640 1 -- 172.21.15.59:0/1849738051 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9459b0e80 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.636+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.637+0000 7f5b46a00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005ab0 tx=0x7f5b34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63319e2b108213c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.637+0000 7f5b31600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.637+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34005d50 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.637+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b40004030 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.637+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b340048c0 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.638+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b34004ac0 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.639+0000 7f5b513fd740 10 client.17034 fetch_fsmap learned FSMap version 46 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.639+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b9456fa210 con 0x55b945a808c0 2024-04-02T17:28:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.639+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40004030 con 0x55b945a808c0 2024-04-02T17:28:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f5b340037f0 con 0x55b945a808c0 2024-04-02T17:28:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 10 client.17034 fetch_fsmap finished waiting for FSMap version 46 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 20 client.17034 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945aa73c0 0x55b945aa9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 4 client.17034 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b945995cd0 con 0x55b945aa73c0 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.640+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945aa73c0 0x55b945aa9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.642+0000 7f5b47e00640 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945aa73c0 0x55b945aa9820 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3219c47daec6c4 server_cookie=ff92aae2eddb66e8 in_seq=0 out_seq=0 2024-04-02T17:28:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.642+0000 7f5b31600640 10 client.17034 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.643+0000 7f5b31600640 1 -- 172.21.15.59:0/1849738051 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b945995cd0 con 0x55b945aa73c0 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.643+0000 7f5b31600640 10 client.17034 handle_command_reply: tid=0 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.643+0000 7f5b513fd740 1 client.17034 shutdown 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 msgr2=0x55b945a9b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005ab0 tx=0x7f5b34004270 comp rx=0 tx=0).stop 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 shutdown_connections 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945aa73c0 0x55b945aa9820 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a808c0 0x55b945a9b9d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a85a30 0x55b945aa2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 --2- 172.21.15.59:0/1849738051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 >> 172.21.15.59:0/1849738051 conn(0x55b945a7b440 msgr2=0x55b945a7a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b31600640 0 client.17034 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 shutdown_connections 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.644+0000 7f5b513fd740 1 -- 172.21.15.59:0/1849738051 wait complete. 2024-04-02T17:28:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.645+0000 7f5b513fd740 20 client.17034 trim_cache size 0 max 16384 2024-04-02T17:28:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 Processor -- start 2024-04-02T17:28:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 -- start start 2024-04-02T17:28:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945aa29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945a9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4806fed0 con 0x55b945a7fed0 2024-04-02T17:28:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48071b60 con 0x55b945a85a30 2024-04-02T17:28:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b513fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b480705a0 con 0x55b945a99f10 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.671+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36832/0 (socket says 172.21.15.59:36832) 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 -- 172.21.15.59:0/3678705092 learned_addr learned my addr 172.21.15.59:0/3678705092 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b46a00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945aa29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945a9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 -- 172.21.15.59:0/3678705092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 msgr2=0x55b945a9a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945a9a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 -- 172.21.15.59:0/3678705092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 msgr2=0x55b945aa29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945aa29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b47e00640 1 -- 172.21.15.59:0/3678705092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9457c6620 con 0x55b945a7fed0 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.672+0000 7f5b46a00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945aa29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.673+0000 7f5b47e00640 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b40011850 tx=0x7f5b40003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fde0e9f97d7b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.673+0000 7f5b33e00640 1 -- 172.21.15.59:0/3678705092 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4001e040 con 0x55b945a7fed0 2024-04-02T17:28:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.673+0000 7f5b33e00640 1 -- 172.21.15.59:0/3678705092 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40003d10 con 0x55b945a7fed0 2024-04-02T17:28:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.673+0000 7f5b33e00640 1 -- 172.21.15.59:0/3678705092 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4002c710 con 0x55b945a7fed0 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 -- 172.21.15.59:0/3678705092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 msgr2=0x55b945a7b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b40011850 tx=0x7f5b40003b50 comp rx=0 tx=0).stop 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 -- 172.21.15.59:0/3678705092 shutdown_connections 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945a9a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a85a30 0x55b945aa29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 --2- 172.21.15.59:0/3678705092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a7fed0 0x55b945a7b820 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 -- 172.21.15.59:0/3678705092 >> 172.21.15.59:0/3678705092 conn(0x55b945a7b440 msgr2=0x55b945a81360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.674+0000 7f5b513fd740 1 -- 172.21.15.59:0/3678705092 shutdown_connections 2024-04-02T17:28:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.675+0000 7f5b513fd740 1 -- 172.21.15.59:0/3678705092 wait complete. 2024-04-02T17:28:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.675+0000 7f5b513fd740 1 -- start start 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.675+0000 7f5b513fd740 1 Processor -- start 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 0x55b945aa0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945aa1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b480713d0 con 0x55b945a85a30 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48070ca0 con 0x55b945a7fed0 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4806deb0 con 0x55b945a99f10 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b513fd740 10 client.0 _ll_register_callbacks cb 0x7f5b3402c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37400/0 (socket says 172.21.15.59:37400) 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b47e00640 1 -- 172.21.15.59:0/2221256979 learned_addr learned my addr 172.21.15.59:0/2221256979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b47e00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945aa1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.676+0000 7f5b46a00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 0x55b945aa0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 -- 172.21.15.59:0/2221256979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 msgr2=0x55b945aa1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945aa1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 -- 172.21.15.59:0/2221256979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 msgr2=0x55b945aa0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 0x55b945aa0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 -- 172.21.15.59:0/2221256979 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b945aa1900 con 0x55b945a7fed0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b46a00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 0x55b945aa0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b47e00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b40005c40 tx=0x7f5b40005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2bb550606c02fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.677+0000 7f5b31600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4002c490 con 0x55b945a7fed0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b40034070 con 0x55b945a7fed0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b40003d10 con 0x55b945a7fed0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b4002c630 con 0x55b945a7fed0 2024-04-02T17:28:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b513fd740 10 client.27143 fetch_fsmap learned FSMap version 46 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b945aa2c10 con 0x55b945a7fed0 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40003d10 con 0x55b945a7fed0 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.678+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f5b40024040 con 0x55b945a7fed0 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 10 client.27143 fetch_fsmap finished waiting for FSMap version 46 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 10 client.27143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 20 client.27143 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945b0ddd0 0x55b945b101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 4 client.27143 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:28:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.679+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9459f32b0 con 0x55b945b0ddd0 2024-04-02T17:28:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.680+0000 7f5b46a00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945b0ddd0 0x55b945b101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.682+0000 7f5b46a00640 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945b0ddd0 0x55b945b101f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6b1afa5bc5b257 server_cookie=8b77520e5f7d7e4b in_seq=0 out_seq=0 2024-04-02T17:28:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.682+0000 7f5b31600640 10 client.27143 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.873+0000 7f5b31600640 1 -- 172.21.15.59:0/2221256979 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9459f32b0 con 0x55b945b0ddd0 2024-04-02T17:28:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.873+0000 7f5b31600640 10 client.27143 handle_command_reply: tid=0 2024-04-02T17:28:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.873+0000 7f5b513fd740 1 client.27143 shutdown 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 msgr2=0x55b945a9fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b40005c40 tx=0x7f5b40005cf0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 shutdown_connections 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b945b0ddd0 0x55b945b101f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b945a85a30 0x55b945aa0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b945a99f10 0x55b945aa1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 --2- 172.21.15.59:0/2221256979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b945a7fed0 0x55b945a9fdc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b31600640 0 client.27143 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 >> 172.21.15.59:0/2221256979 conn(0x55b945a7b440 msgr2=0x55b945a7b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 shutdown_connections 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 1 -- 172.21.15.59:0/2221256979 wait complete. 2024-04-02T17:28:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.874+0000 7f5b513fd740 20 client.27143 trim_cache size 0 max 16384 2024-04-02T17:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.875+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 msgr2=0x7f5b2c0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.875+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 0x7f5b2c0a6110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b38098a70 tx=0x7f5b380d3e30 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.875+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 msgr2=0x7f5b48199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.875+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 0x7f5b48199fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e9f0 tx=0x7f5b3800edf0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 shutdown_connections 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b2c0a3c50 0x7f5b2c0a6110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b481078a0 0x7f5b481a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b48104f70 0x7f5b4819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 --2- 172.21.15.59:0/628131118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b48102640 0x7f5b48199fe0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 >> 172.21.15.59:0/628131118 conn(0x7f5b480fbf90 msgr2=0x7f5b48101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.876+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 shutdown_connections 2024-04-02T17:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:18.877+0000 7f5b4e600640 1 -- 172.21.15.59:0/628131118 wait complete. 2024-04-02T17:28:19.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:19 smithi154 ceph-mon[36479]: pgmap v845: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-04-02T17:28:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:19 smithi059 ceph-mon[31664]: pgmap v845: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-04-02T17:28:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:19 smithi164 ceph-mon[36499]: pgmap v845: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.11k op/s 2024-04-02T17:28:19.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 87358 46.08 MB/sec warmup 106 sec latency 129.762 ms 2024-04-02T17:28:20.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 88083 45.99 MB/sec warmup 107 sec latency 64.832 ms 2024-04-02T17:28:21.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:21 smithi154 ceph-mon[36479]: pgmap v846: 97 pgs: 97 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-04-02T17:28:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:21 smithi059 ceph-mon[31664]: pgmap v846: 97 pgs: 97 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-04-02T17:28:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:21 smithi164 ceph-mon[36499]: pgmap v846: 97 pgs: 97 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-04-02T17:28:21.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 88991 46.00 MB/sec warmup 108 sec latency 89.852 ms 2024-04-02T17:28:22.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 89699 45.89 MB/sec warmup 109 sec latency 269.079 ms 2024-04-02T17:28:23.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:23 smithi154 ceph-mon[36479]: pgmap v847: 97 pgs: 97 active+clean; 538 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 958 op/s 2024-04-02T17:28:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:23 smithi059 ceph-mon[31664]: pgmap v847: 97 pgs: 97 active+clean; 538 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 958 op/s 2024-04-02T17:28:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:23 smithi164 ceph-mon[36499]: pgmap v847: 97 pgs: 97 active+clean; 538 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 958 op/s 2024-04-02T17:28:23.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 90692 46.08 MB/sec warmup 110 sec latency 30.351 ms 2024-04-02T17:28:24.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 91369 45.96 MB/sec warmup 111 sec latency 35.398 ms 2024-04-02T17:28:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:25 smithi154 ceph-mon[36479]: pgmap v848: 97 pgs: 97 active+clean; 543 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 942 op/s 2024-04-02T17:28:25.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:25 smithi059 ceph-mon[31664]: pgmap v848: 97 pgs: 97 active+clean; 543 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 942 op/s 2024-04-02T17:28:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:25 smithi164 ceph-mon[36499]: pgmap v848: 97 pgs: 97 active+clean; 543 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 942 op/s 2024-04-02T17:28:25.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 92194 45.93 MB/sec warmup 112 sec latency 65.613 ms 2024-04-02T17:28:26.595 INFO:tasks.workunit.client.0.smithi059.stdout: 10 93343 46.00 MB/sec warmup 113 sec latency 34.314 ms 2024-04-02T17:28:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:27 smithi154 ceph-mon[36479]: pgmap v849: 97 pgs: 97 active+clean; 550 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 715 op/s 2024-04-02T17:28:27.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:27 smithi059 ceph-mon[31664]: pgmap v849: 97 pgs: 97 active+clean; 550 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 715 op/s 2024-04-02T17:28:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:27 smithi164 ceph-mon[36499]: pgmap v849: 97 pgs: 97 active+clean; 550 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.7 MiB/s wr, 715 op/s 2024-04-02T17:28:27.594 INFO:tasks.workunit.client.0.smithi059.stdout: 10 94249 46.14 MB/sec warmup 114 sec latency 48.277 ms 2024-04-02T17:28:28.595 INFO:tasks.workunit.client.0.smithi059.stdout: 10 94943 46.04 MB/sec warmup 115 sec latency 69.334 ms 2024-04-02T17:28:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:29 smithi154 ceph-mon[36479]: pgmap v850: 97 pgs: 97 active+clean; 558 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 644 op/s 2024-04-02T17:28:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:29 smithi059 ceph-mon[31664]: pgmap v850: 97 pgs: 97 active+clean; 558 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 644 op/s 2024-04-02T17:28:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:29 smithi164 ceph-mon[36499]: pgmap v850: 97 pgs: 97 active+clean; 558 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 644 op/s 2024-04-02T17:28:29.595 INFO:tasks.workunit.client.0.smithi059.stdout: 10 95712 45.96 MB/sec warmup 116 sec latency 128.208 ms 2024-04-02T17:28:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:30 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:30 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:28:30.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:30 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:30.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:30 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:28:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:30 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:30 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:28:30.595 INFO:tasks.workunit.client.0.smithi059.stdout: 10 96316 45.85 MB/sec warmup 117 sec latency 163.136 ms 2024-04-02T17:28:31.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:31 smithi154 ceph-mon[36479]: pgmap v851: 97 pgs: 97 active+clean; 572 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 715 op/s 2024-04-02T17:28:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:31 smithi059 ceph-mon[31664]: pgmap v851: 97 pgs: 97 active+clean; 572 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 715 op/s 2024-04-02T17:28:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:31 smithi164 ceph-mon[36499]: pgmap v851: 97 pgs: 97 active+clean; 572 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 715 op/s 2024-04-02T17:28:31.595 INFO:tasks.workunit.client.0.smithi059.stdout: 10 97278 45.87 MB/sec warmup 118 sec latency 54.213 ms 2024-04-02T17:28:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 -- 172.21.15.59:0/596028806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 msgr2=0x7f27341097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 --2- 172.21.15.59:0/596028806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f27341097d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f271c009900 tx=0x7f271c02bc60 comp rx=0 tx=0).stop 2024-04-02T17:28:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 -- 172.21.15.59:0/596028806 shutdown_connections 2024-04-02T17:28:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 --2- 172.21.15.59:0/596028806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2734109d10 0x7f27341105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 --2- 172.21.15.59:0/596028806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f27341097d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 --2- 172.21.15.59:0/596028806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 0x7f2734108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 -- 172.21.15.59:0/596028806 >> 172.21.15.59:0/596028806 conn(0x7f2734077110 msgr2=0x7f2734079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.067+0000 7f273b200640 1 -- 172.21.15.59:0/596028806 shutdown_connections 2024-04-02T17:28:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.068+0000 7f273b200640 1 -- 172.21.15.59:0/596028806 wait complete. 2024-04-02T17:28:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.070+0000 7f273b200640 1 Processor -- start 2024-04-02T17:28:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.070+0000 7f273b200640 1 -- start start 2024-04-02T17:28:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 0x7f273419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2734109d10 0x7f27341a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2734113e80 con 0x7f2734108160 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2734113d00 con 0x7f2734109d10 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.071+0000 7f273b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2734114000 con 0x7f2734109350 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51510/0 (socket says 172.21.15.59:51510) 2024-04-02T17:28:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 -- 172.21.15.59:0/3042289504 learned_addr learned my addr 172.21.15.59:0/3042289504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 -- 172.21.15.59:0/3042289504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2734109d10 msgr2=0x7f27341a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2734109d10 0x7f27341a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.072+0000 7f2733400640 1 -- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 msgr2=0x7f273419e430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.073+0000 7f2733e00640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 0x7f273419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.073+0000 7f2733400640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 0x7f273419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.073+0000 7f2733400640 1 -- 172.21.15.59:0/3042289504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341a6050 con 0x7f2734109350 2024-04-02T17:28:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.073+0000 7f2733400640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f271c000c00 tx=0x7f271c02eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.074+0000 7f2730c00640 1 -- 172.21.15.59:0/3042289504 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c041020 con 0x7f2734109350 2024-04-02T17:28:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.075+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27341a62e0 con 0x7f2734109350 2024-04-02T17:28:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.075+0000 7f2730c00640 1 -- 172.21.15.59:0/3042289504 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c004d90 con 0x7f2734109350 2024-04-02T17:28:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.075+0000 7f2730c00640 1 -- 172.21.15.59:0/3042289504 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c041020 con 0x7f2734109350 2024-04-02T17:28:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.075+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341a6820 con 0x7f2734109350 2024-04-02T17:28:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.081+0000 7f2730c00640 1 -- 172.21.15.59:0/3042289504 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f271c044020 con 0x7f2734109350 2024-04-02T17:28:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.082+0000 7f273dfc5740 1 Processor -- start 2024-04-02T17:28:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.082+0000 7f273dfc5740 1 -- start start 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e3e946d70 con 0x558e3e9e18c0 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558e3e8ef2a0 con 0x558e3e9e0ed0 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f273dfc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e3e942d20 con 0x558e3e9e6a30 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f2739200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f2733e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f2739200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39034/0 (socket says 172.21.15.59:39034) 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.083+0000 7f2739200640 1 -- 172.21.15.59:0/4062468197 learned_addr learned my addr 172.21.15.59:0/4062468197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2733400640 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2739200640 1 -- 172.21.15.59:0/4062468197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 msgr2=0x558e3e9e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2739200640 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2739200640 1 -- 172.21.15.59:0/4062468197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 msgr2=0x558e3e9e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2739200640 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9e63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2730c00640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 0x7f27180a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2739200640 1 -- 172.21.15.59:0/4062468197 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3e903840 con 0x558e3e9e0ed0 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.084+0000 7f2733400640 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.085+0000 7f2730c00640 1 -- 172.21.15.59:0/3042289504 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f271c0f1ac0 con 0x7f2734109350 2024-04-02T17:28:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.085+0000 7f2739200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 0x7f27180a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.085+0000 7f2739200640 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f272c009790 tx=0x7f272c009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b01deb0e393609 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.087+0000 7f2729600640 1 -- 172.21.15.59:0/4062468197 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c017030 con 0x558e3e9e0ed0 2024-04-02T17:28:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.087+0000 7f2729600640 1 -- 172.21.15.59:0/4062468197 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272c00b9f0 con 0x558e3e9e0ed0 2024-04-02T17:28:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.087+0000 7f2729600640 1 -- 172.21.15.59:0/4062468197 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c010380 con 0x558e3e9e0ed0 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 -- 172.21.15.59:0/4062468197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 msgr2=0x558e3e9e12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f272c009790 tx=0x7f272c009c50 comp rx=0 tx=0).stop 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f2739200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 0x7f27180a63a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f272c01b000 tx=0x7f272c01c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 -- 172.21.15.59:0/4062468197 shutdown_connections 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 --2- 172.21.15.59:0/4062468197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e12f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 -- 172.21.15.59:0/4062468197 >> 172.21.15.59:0/4062468197 conn(0x558e3e9dc440 msgr2=0x558e3e9dc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.088+0000 7f273dfc5740 1 -- 172.21.15.59:0/4062468197 shutdown_connections 2024-04-02T17:28:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.089+0000 7f273dfc5740 1 -- 172.21.15.59:0/4062468197 wait complete. 2024-04-02T17:28:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.089+0000 7f273dfc5740 1 -- start start 2024-04-02T17:28:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.090+0000 7f273dfc5740 1 Processor -- start 2024-04-02T17:28:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.090+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 0x558e3e9eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.090+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f273dfc5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e3e9e99e0 con 0x558e3e9e18c0 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f273dfc5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558e3e902660 con 0x558e3e9e6a30 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f273dfc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e3e9e9b60 con 0x558e3e9e0ed0 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f273dfc5740 10 client.0 _ll_register_callbacks cb 0x558e3e902050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f2739200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f2733e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 0x558e3e9eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f2733e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 0x558e3e9eb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51534/0 (socket says 172.21.15.59:51534) 2024-04-02T17:28:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.091+0000 7f2733e00640 1 -- 172.21.15.59:0/1887293310 learned_addr learned my addr 172.21.15.59:0/1887293310 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.092+0000 7f2739200640 1 -- 172.21.15.59:0/1887293310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 msgr2=0x558e3e9eb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.092+0000 7f2739200640 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 0x558e3e9eb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.092+0000 7f2739200640 1 -- 172.21.15.59:0/1887293310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 msgr2=0x558e3e9eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.092+0000 7f2739200640 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.092+0000 7f2739200640 1 -- 172.21.15.59:0/1887293310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3e911e80 con 0x558e3e9e6a30 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.093+0000 7f2739200640 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9f29f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f272c009790 tx=0x7f272c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21b82c2efe00731 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.093+0000 7f2722000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:28:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.094+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c029020 con 0x558e3e9e6a30 2024-04-02T17:28:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.094+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f272c01c3d0 con 0x558e3e9e6a30 2024-04-02T17:28:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.094+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272c00b9f0 con 0x558e3e9e6a30 2024-04-02T17:28:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.095+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f272c021040 con 0x558e3e9e6a30 2024-04-02T17:28:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.095+0000 7f273dfc5740 10 client.27149 fetch_fsmap learned FSMap version 47 2024-04-02T17:28:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.095+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558e3e65b210 con 0x558e3e9e6a30 2024-04-02T17:28:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.096+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272c00b9f0 con 0x558e3e9e6a30 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.096+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7f272c024070 con 0x558e3e9e6a30 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.096+0000 7f273dfc5740 10 client.27149 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.096+0000 7f273dfc5740 10 client.27149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.097+0000 7f273dfc5740 20 client.27149 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.097+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3e9f71b0 0x558e3e9f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.097+0000 7f273dfc5740 4 client.27149 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:28:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.097+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e3e8f6cd0 con 0x558e3e9f71b0 2024-04-02T17:28:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.098+0000 7f2733e00640 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3e9f71b0 0x558e3e9f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.099+0000 7f2733e00640 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3e9f71b0 0x558e3e9f95f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acbed95718651688 server_cookie=b78847b6ca1a2959 in_seq=0 out_seq=0 2024-04-02T17:28:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.099+0000 7f2722000640 10 client.27149 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.104+0000 7f2722000640 1 -- 172.21.15.59:0/1887293310 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e3e8f6cd0 con 0x558e3e9f71b0 2024-04-02T17:28:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.104+0000 7f2722000640 10 client.27149 handle_command_reply: tid=0 2024-04-02T17:28:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.104+0000 7f273dfc5740 1 client.27149 shutdown 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.104+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 msgr2=0x558e3e9f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.104+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9f29f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f272c009790 tx=0x7f272c01c900 comp rx=0 tx=0).stop 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 shutdown_connections 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3e9f71b0 0x558e3e9f95f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e18c0 0x558e3e9eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9f29f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 --2- 172.21.15.59:0/1887293310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e0ed0 0x558e3e9eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 >> 172.21.15.59:0/1887293310 conn(0x558e3e9dc440 msgr2=0x558e3e9db640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f2722000640 0 client.27149 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 shutdown_connections 2024-04-02T17:28:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 1 -- 172.21.15.59:0/1887293310 wait complete. 2024-04-02T17:28:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.105+0000 7f273dfc5740 20 client.27149 trim_cache size 0 max 16384 2024-04-02T17:28:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.142+0000 7f273dfc5740 1 Processor -- start 2024-04-02T17:28:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.143+0000 7f273dfc5740 1 -- start start 2024-04-02T17:28:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.143+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.143+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f273dfc5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27341af890 con 0x558e3e9e0ed0 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f273dfc5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27341b14f0 con 0x558e3e9e6a30 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51548/0 (socket says 172.21.15.59:51548) 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f2733400640 1 -- 172.21.15.59:0/49451191 learned_addr learned my addr 172.21.15.59:0/49451191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27341aff60 con 0x558e3e9e18c0 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f2739200640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.144+0000 7f2733e00640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 -- 172.21.15.59:0/49451191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 msgr2=0x558e3e9dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 -- 172.21.15.59:0/49451191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 msgr2=0x558e3e9e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 -- 172.21.15.59:0/49451191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3e727620 con 0x558e3e9e18c0 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733e00640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.145+0000 7f2733400640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ba1a0 tx=0x7f271c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a7d8eed28ace27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.146+0000 7f2739200640 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.146+0000 7f2728c00640 1 -- 172.21.15.59:0/49451191 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0bd880 con 0x558e3e9e18c0 2024-04-02T17:28:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.146+0000 7f2728c00640 1 -- 172.21.15.59:0/49451191 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c0be440 con 0x558e3e9e18c0 2024-04-02T17:28:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.146+0000 7f2728c00640 1 -- 172.21.15.59:0/49451191 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0bd6a0 con 0x558e3e9e18c0 2024-04-02T17:28:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.148+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 msgr2=0x558e3e9f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.148+0000 7f273dfc5740 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ba1a0 tx=0x7f271c009260 comp rx=0 tx=0).stop 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 shutdown_connections 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e6a30 0x558e3e9dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e18c0 0x558e3e9f29b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 --2- 172.21.15.59:0/49451191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 >> 172.21.15.59:0/49451191 conn(0x558e3e9dc440 msgr2=0x558e3e9e9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.149+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 shutdown_connections 2024-04-02T17:28:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.150+0000 7f273dfc5740 1 -- 172.21.15.59:0/49451191 wait complete. 2024-04-02T17:28:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.150+0000 7f273dfc5740 1 -- start start 2024-04-02T17:28:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.150+0000 7f273dfc5740 1 Processor -- start 2024-04-02T17:28:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9edc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27341b0d60 con 0x558e3e9e0ed0 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27341b0660 con 0x558e3e9e18c0 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f273dfc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27341ad830 con 0x558e3e9e6a30 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.151+0000 7f2733400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.152+0000 7f2733400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39042/0 (socket says 172.21.15.59:39042) 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.152+0000 7f2733400640 1 -- 172.21.15.59:0/2857133618 learned_addr learned my addr 172.21.15.59:0/2857133618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.152+0000 7f273dfc5740 10 client.0 _ll_register_callbacks cb 0x7f272c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.152+0000 7f2733400640 1 -- 172.21.15.59:0/2857133618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 msgr2=0x558e3e9f1190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.152+0000 7f2733400640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9f1190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.153+0000 7f2733400640 1 -- 172.21.15.59:0/2857133618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 msgr2=0x558e3e9edc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.153+0000 7f2733e00640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.153+0000 7f2733400640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f2733400640 1 -- 172.21.15.59:0/2857133618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3e9f1780 con 0x558e3e9e18c0 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f271c0be440 con 0x558e3e9e18c0 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f2733e00640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:28:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f2722000640 10 client.27155 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f2733400640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ef5b0 tx=0x7f271c02c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f677c81a8d60b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.154+0000 7f2722000640 1 -- 172.21.15.59:0/2857133618 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0bd6a0 con 0x558e3e9e18c0 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.155+0000 7f2722000640 1 -- 172.21.15.59:0/2857133618 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f271c005270 con 0x558e3e9e18c0 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.155+0000 7f2722000640 1 -- 172.21.15.59:0/2857133618 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f271c0bd880 con 0x558e3e9e18c0 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.155+0000 7f273dfc5740 10 client.27155 fetch_fsmap learned FSMap version 47 2024-04-02T17:28:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.155+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=47}) -- 0x558e3e9f2a20 con 0x558e3e9e18c0 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f2722000640 1 -- 172.21.15.59:0/2857133618 <== mon.2 v2:172.21.15.154:3300/0 4 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7f271c0d70a0 con 0x558e3e9e18c0 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 10 client.27155 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 10 client.27155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 20 client.27155 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3ea6eb90 0x558e3ea70fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 4 client.27155 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:28:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e3e9542b0 con 0x558e3ea6eb90 2024-04-02T17:28:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.156+0000 7f2733e00640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3ea6eb90 0x558e3ea70fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.157+0000 7f2733e00640 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3ea6eb90 0x558e3ea70fb0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6367451e95de51 server_cookie=941103f881f5b2c1 in_seq=0 out_seq=0 2024-04-02T17:28:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.158+0000 7f2722000640 10 client.27155 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.374+0000 7f2722000640 1 -- 172.21.15.59:0/2857133618 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e3e9542b0 con 0x558e3ea6eb90 2024-04-02T17:28:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.374+0000 7f2722000640 10 client.27155 handle_command_reply: tid=0 2024-04-02T17:28:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.374+0000 7f273dfc5740 1 client.27155 shutdown 2024-04-02T17:28:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 msgr2=0x558e3e9ee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ef5b0 tx=0x7f271c02c220 comp rx=0 tx=0).stop 2024-04-02T17:28:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 shutdown_connections 2024-04-02T17:28:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558e3ea6eb90 0x558e3ea70fb0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e3e9e6a30 0x558e3e9f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558e3e9e18c0 0x558e3e9ee1c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f273dfc5740 1 --2- 172.21.15.59:0/2857133618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e3e9e0ed0 0x558e3e9edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.376+0000 7f2722000640 0 client.27155 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.377+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 >> 172.21.15.59:0/2857133618 conn(0x558e3e9dc440 msgr2=0x558e3e9df6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.377+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 shutdown_connections 2024-04-02T17:28:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.377+0000 7f273dfc5740 1 -- 172.21.15.59:0/2857133618 wait complete. 2024-04-02T17:28:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.377+0000 7f273dfc5740 20 client.27155 trim_cache size 0 max 16384 2024-04-02T17:28:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:28:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:28:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:28:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:28:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 msgr2=0x7f27180a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 0x7f27180a63a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f272c01b000 tx=0x7f272c01c000 comp rx=0 tx=0).stop 2024-04-02T17:28:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 msgr2=0x7f273419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f271c000c00 tx=0x7f271c02eb70 comp rx=0 tx=0).stop 2024-04-02T17:28:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 shutdown_connections 2024-04-02T17:28:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2734109d10 0x7f27341a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734109350 0x7f273419e970 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f27180a3ee0 0x7f27180a63a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 --2- 172.21.15.59:0/3042289504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2734108160 0x7f273419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.378+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 >> 172.21.15.59:0/3042289504 conn(0x7f2734077110 msgr2=0x7f273410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.379+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 shutdown_connections 2024-04-02T17:28:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:32.379+0000 7f273b200640 1 -- 172.21.15.59:0/3042289504 wait complete. 2024-04-02T17:28:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:28:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:28:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:28:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:28:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:28:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:28:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:28:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:28:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' 2024-04-02T17:28:32.596 INFO:tasks.workunit.client.0.smithi059.stdout: 10 97944 45.82 MB/sec warmup 119 sec latency 139.537 ms 2024-04-02T17:28:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:33 smithi059 ceph-mon[31664]: pgmap v852: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-02T17:28:33.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:33 smithi164 ceph-mon[36499]: pgmap v852: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-02T17:28:33.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:33 smithi154 ceph-mon[36479]: pgmap v852: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-02T17:28:34.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 99814 59.00 MB/sec execute 1 sec latency 58.958 ms 2024-04-02T17:28:35.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 99844 29.75 MB/sec execute 2 sec latency 702.344 ms 2024-04-02T17:28:35.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:35 smithi059 ceph-mon[31664]: pgmap v853: 97 pgs: 97 active+clean; 583 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 708 op/s 2024-04-02T17:28:35.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:35 smithi164 ceph-mon[36499]: pgmap v853: 97 pgs: 97 active+clean; 583 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 708 op/s 2024-04-02T17:28:35.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:35 smithi154 ceph-mon[36479]: pgmap v853: 97 pgs: 97 active+clean; 583 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 708 op/s 2024-04-02T17:28:36.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 100266 28.29 MB/sec execute 3 sec latency 704.912 ms 2024-04-02T17:28:37.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 100787 28.01 MB/sec execute 4 sec latency 65.536 ms 2024-04-02T17:28:37.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:37 smithi059 ceph-mon[31664]: pgmap v854: 97 pgs: 97 active+clean; 611 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 767 op/s 2024-04-02T17:28:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:37 smithi164 ceph-mon[36499]: pgmap v854: 97 pgs: 97 active+clean; 611 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 767 op/s 2024-04-02T17:28:37.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:37 smithi154 ceph-mon[36479]: pgmap v854: 97 pgs: 97 active+clean; 611 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 767 op/s 2024-04-02T17:28:38.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 101538 30.60 MB/sec execute 5 sec latency 48.105 ms 2024-04-02T17:28:39.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 102498 33.27 MB/sec execute 6 sec latency 76.576 ms 2024-04-02T17:28:39.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:39 smithi059 ceph-mon[31664]: pgmap v855: 97 pgs: 97 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 14 MiB/s wr, 898 op/s 2024-04-02T17:28:39.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:39 smithi164 ceph-mon[36499]: pgmap v855: 97 pgs: 97 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 14 MiB/s wr, 898 op/s 2024-04-02T17:28:39.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:39 smithi154 ceph-mon[36479]: pgmap v855: 97 pgs: 97 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 14 MiB/s wr, 898 op/s 2024-04-02T17:28:40.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 103492 37.29 MB/sec execute 7 sec latency 36.837 ms 2024-04-02T17:28:41.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 104317 37.76 MB/sec execute 8 sec latency 28.401 ms 2024-04-02T17:28:41.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:41 smithi059 ceph-mon[31664]: pgmap v856: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-04-02T17:28:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:41 smithi164 ceph-mon[36499]: pgmap v856: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-04-02T17:28:41.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:41 smithi154 ceph-mon[36479]: pgmap v856: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.00k op/s 2024-04-02T17:28:42.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 105103 38.16 MB/sec execute 9 sec latency 48.306 ms 2024-04-02T17:28:43.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 106231 39.70 MB/sec execute 10 sec latency 68.956 ms 2024-04-02T17:28:43.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:43 smithi059 ceph-mon[31664]: pgmap v857: 97 pgs: 97 active+clean; 566 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-02T17:28:43.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:43 smithi164 ceph-mon[36499]: pgmap v857: 97 pgs: 97 active+clean; 566 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-02T17:28:43.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:43 smithi154 ceph-mon[36479]: pgmap v857: 97 pgs: 97 active+clean; 566 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-02T17:28:44.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 107093 41.38 MB/sec execute 11 sec latency 39.300 ms 2024-04-02T17:28:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 -- 172.21.15.59:0/1716724582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738103d60 msgr2=0x7ff738104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 --2- 172.21.15.59:0/1716724582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738103d60 0x7ff738104180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009900 tx=0x7ff72c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 -- 172.21.15.59:0/1716724582 shutdown_connections 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 --2- 172.21.15.59:0/1716724582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 0x7ff73810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 --2- 172.21.15.59:0/1716724582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738104f50 0x7ff7381053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 --2- 172.21.15.59:0/1716724582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738103d60 0x7ff738104180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.558+0000 7ff73fa00640 1 -- 172.21.15.59:0/1716724582 >> 172.21.15.59:0/1716724582 conn(0x7ff7380ff2f0 msgr2=0x7ff738101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.559+0000 7ff73fa00640 1 -- 172.21.15.59:0/1716724582 shutdown_connections 2024-04-02T17:28:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.559+0000 7ff73fa00640 1 -- 172.21.15.59:0/1716724582 wait complete. 2024-04-02T17:28:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.560+0000 7ff73fa00640 1 Processor -- start 2024-04-02T17:28:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.560+0000 7ff73fa00640 1 -- start start 2024-04-02T17:28:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 0x7ff738199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 0x7ff7381a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810fa30 con 0x7ff738104f50 2024-04-02T17:28:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810f8b0 con 0x7ff738103d60 2024-04-02T17:28:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.561+0000 7ff73fa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810fbb0 con 0x7ff738105910 2024-04-02T17:28:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.562+0000 7ff737e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.562+0000 7ff737e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56184/0 (socket says 172.21.15.59:56184) 2024-04-02T17:28:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.562+0000 7ff737e00640 1 -- 172.21.15.59:0/170455564 learned_addr learned my addr 172.21.15.59:0/170455564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.562+0000 7ff73da00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 0x7ff7381a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff73d000640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 0x7ff738199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff737e00640 1 -- 172.21.15.59:0/170455564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 msgr2=0x7ff7381a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff737e00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 0x7ff7381a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff737e00640 1 -- 172.21.15.59:0/170455564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 msgr2=0x7ff738199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff737e00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 0x7ff738199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff737e00640 1 -- 172.21.15.59:0/170455564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7381a1bd0 con 0x7ff738104f50 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.563+0000 7ff73d000640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 0x7ff738199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.564+0000 7ff737e00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ed60 tx=0x7ff72400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.565+0000 7ff735600640 1 -- 172.21.15.59:0/170455564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724009e10 con 0x7ff738104f50 2024-04-02T17:28:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.565+0000 7ff735600640 1 -- 172.21.15.59:0/170455564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff724010910 con 0x7ff738104f50 2024-04-02T17:28:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.565+0000 7ff735600640 1 -- 172.21.15.59:0/170455564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724018a90 con 0x7ff738104f50 2024-04-02T17:28:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.565+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7381a1ec0 con 0x7ff738104f50 2024-04-02T17:28:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.565+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7381a2330 con 0x7ff738104f50 2024-04-02T17:28:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.571+0000 7ff735600640 1 -- 172.21.15.59:0/170455564 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff724010ac0 con 0x7ff738104f50 2024-04-02T17:28:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.572+0000 7ff735600640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 0x7ff7200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.573+0000 7ff735600640 1 -- 172.21.15.59:0/170455564 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff7240d07c0 con 0x7ff738104f50 2024-04-02T17:28:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.573+0000 7ff73d000640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 0x7ff7200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.573+0000 7ff7425f9740 1 Processor -- start 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.573+0000 7ff7425f9740 1 -- start start 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ace6e0 0x564fd4aceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ad66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564fd4a2bd70 con 0x564fd4ace6e0 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564fd49d42a0 con 0x564fd4ad4240 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff7425f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564fd4a27d20 con 0x564fd4acf0d0 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff737e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff737e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33182/0 (socket says 172.21.15.59:33182) 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff737e00640 1 -- 172.21.15.59:0/506760713 learned_addr learned my addr 172.21.15.59:0/506760713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff73da00640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ad66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.574+0000 7ff73d000640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ace6e0 0x564fd4aceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 -- 172.21.15.59:0/506760713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 msgr2=0x564fd4ad3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 -- 172.21.15.59:0/506760713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ace6e0 msgr2=0x564fd4aceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ace6e0 0x564fd4aceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73d000640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 0x7ff7200a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009850 tx=0x7ff72c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 -- 172.21.15.59:0/506760713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd49e8840 con 0x564fd4ad4240 2024-04-02T17:28:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff737e00640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.575+0000 7ff73da00640 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ad66d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff730009f00 tx=0x7ff730011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6a5cce023fb334 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.576+0000 7ff72a000640 1 -- 172.21.15.59:0/506760713 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730004d20 con 0x564fd4ad4240 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.576+0000 7ff72a000640 1 -- 172.21.15.59:0/506760713 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730004030 con 0x564fd4ad4240 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.576+0000 7ff72a000640 1 -- 172.21.15.59:0/506760713 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730005260 con 0x564fd4ad4240 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 -- 172.21.15.59:0/506760713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 msgr2=0x564fd4ad66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ad66d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff730009f00 tx=0x7ff730011810 comp rx=0 tx=0).stop 2024-04-02T17:28:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 -- 172.21.15.59:0/506760713 shutdown_connections 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ad66d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 --2- 172.21.15.59:0/506760713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ace6e0 0x564fd4aceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 -- 172.21.15.59:0/506760713 >> 172.21.15.59:0/506760713 conn(0x564fd4ac9c50 msgr2=0x564fd4aca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.577+0000 7ff7425f9740 1 -- 172.21.15.59:0/506760713 shutdown_connections 2024-04-02T17:28:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.578+0000 7ff7425f9740 1 -- 172.21.15.59:0/506760713 wait complete. 2024-04-02T17:28:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 -- start start 2024-04-02T17:28:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 Processor -- start 2024-04-02T17:28:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 0x564fd4ad8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 0x564fd4ae0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564fd4ad71f0 con 0x564fd4ad4240 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564fd49e7660 con 0x564fd4ace6e0 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564fd4ad7370 con 0x564fd4acf0d0 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.580+0000 7ff7425f9740 10 client.0 _ll_register_callbacks cb 0x564fd49e7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.581+0000 7ff73da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 0x564fd4ae0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.581+0000 7ff73da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 0x564fd4ae0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56204/0 (socket says 172.21.15.59:56204) 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.581+0000 7ff73da00640 1 -- 172.21.15.59:0/571437972 learned_addr learned my addr 172.21.15.59:0/571437972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.581+0000 7ff737e00640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.581+0000 7ff73d000640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 0x564fd4ad8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff73d000640 1 -- 172.21.15.59:0/571437972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 msgr2=0x564fd4ad9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff73d000640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff73d000640 1 -- 172.21.15.59:0/571437972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 msgr2=0x564fd4ae0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff73d000640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 0x564fd4ae0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff73d000640 1 -- 172.21.15.59:0/571437972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd49f6e80 con 0x564fd4ace6e0 2024-04-02T17:28:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.582+0000 7ff737e00640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.583+0000 7ff73d000640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 0x564fd4ad8c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009230 tx=0x7ff72c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906b47d4a4c6e09b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.583+0000 7ff71ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.583+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c050070 con 0x564fd4ace6e0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.583+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff730004030 con 0x564fd4ace6e0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.583+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff72c004e50 con 0x564fd4ace6e0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.584+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff72c005050 con 0x564fd4ace6e0 2024-04-02T17:28:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.584+0000 7ff7425f9740 10 client.27167 fetch_fsmap learned FSMap version 47 2024-04-02T17:28:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.584+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564fd4740210 con 0x564fd4ace6e0 2024-04-02T17:28:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.584+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730004030 con 0x564fd4ace6e0 2024-04-02T17:28:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.584+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7ff72c04b040 con 0x564fd4ace6e0 2024-04-02T17:28:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 10 client.27167 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 10 client.27167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 20 client.27167 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4ae4a70 0x564fd4ae6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 4 client.27167 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.585+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd49dbcd0 con 0x564fd4ae4a70 2024-04-02T17:28:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.586+0000 7ff737e00640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4ae4a70 0x564fd4ae6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.587+0000 7ff737e00640 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4ae4a70 0x564fd4ae6f30 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc30075f19ab2065 server_cookie=fa1eebd79fa38723 in_seq=0 out_seq=0 2024-04-02T17:28:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.587+0000 7ff71ea00640 10 client.27167 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.589+0000 7ff71ea00640 1 -- 172.21.15.59:0/571437972 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fd49dbcd0 con 0x564fd4ae4a70 2024-04-02T17:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.589+0000 7ff71ea00640 10 client.27167 handle_command_reply: tid=0 2024-04-02T17:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.589+0000 7ff7425f9740 1 client.27167 shutdown 2024-04-02T17:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 msgr2=0x564fd4ad8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 0x564fd4ad8c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009230 tx=0x7ff72c009260 comp rx=0 tx=0).stop 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 shutdown_connections 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4ae4a70 0x564fd4ae6f30 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4ad4240 0x564fd4ae0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 --2- 172.21.15.59:0/571437972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ace6e0 0x564fd4ad8c00 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009230 tx=0x7ff72c009260 comp rx=0 tx=0).stop 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.590+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 >> 172.21.15.59:0/571437972 conn(0x564fd4ac9c50 msgr2=0x564fd4ac0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.591+0000 7ff71ea00640 0 client.27167 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.591+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 shutdown_connections 2024-04-02T17:28:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.591+0000 7ff7425f9740 1 -- 172.21.15.59:0/571437972 wait complete. 2024-04-02T17:28:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.592+0000 7ff7425f9740 20 client.27167 trim_cache size 0 max 16384 2024-04-02T17:28:45.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 107847 40.87 MB/sec execute 12 sec latency 32.805 ms 2024-04-02T17:28:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.624+0000 7ff7425f9740 1 Processor -- start 2024-04-02T17:28:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.624+0000 7ff7425f9740 1 -- start start 2024-04-02T17:28:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ae0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a6f10 con 0x564fd4acff30 2024-04-02T17:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a8b70 con 0x564fd4ad4240 2024-04-02T17:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.625+0000 7ff7425f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a75e0 con 0x564fd4acf0d0 2024-04-02T17:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.626+0000 7ff73da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.626+0000 7ff737e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ae0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.626+0000 7ff73da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56214/0 (socket says 172.21.15.59:56214) 2024-04-02T17:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.626+0000 7ff73da00640 1 -- 172.21.15.59:0/2495559125 learned_addr learned my addr 172.21.15.59:0/2495559125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.626+0000 7ff73d000640 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff73da00640 1 -- 172.21.15.59:0/2495559125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 msgr2=0x564fd4ad8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff73da00640 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff73da00640 1 -- 172.21.15.59:0/2495559125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 msgr2=0x564fd4ae0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff73da00640 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ae0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff73da00640 1 -- 172.21.15.59:0/2495559125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd480c620 con 0x564fd4acff30 2024-04-02T17:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.627+0000 7ff737e00640 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ae0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.628+0000 7ff73da00640 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff730003a60 tx=0x7ff730003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5edf4fb9b28ef20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.628+0000 7ff729600640 1 -- 172.21.15.59:0/2495559125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730028020 con 0x564fd4acff30 2024-04-02T17:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.628+0000 7ff729600640 1 -- 172.21.15.59:0/2495559125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730004dc0 con 0x564fd4acff30 2024-04-02T17:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.628+0000 7ff729600640 1 -- 172.21.15.59:0/2495559125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730005040 con 0x564fd4acff30 2024-04-02T17:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.629+0000 7ff7425f9740 1 -- 172.21.15.59:0/2495559125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 msgr2=0x564fd4ad8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff730003a60 tx=0x7ff730003a90 comp rx=0 tx=0).stop 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 -- 172.21.15.59:0/2495559125 shutdown_connections 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4acff30 0x564fd4ad8d50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4ae0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 --2- 172.21.15.59:0/2495559125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4ad8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 -- 172.21.15.59:0/2495559125 >> 172.21.15.59:0/2495559125 conn(0x564fd4ac9c50 msgr2=0x564fd4ad7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 -- 172.21.15.59:0/2495559125 shutdown_connections 2024-04-02T17:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.630+0000 7ff7425f9740 1 -- 172.21.15.59:0/2495559125 wait complete. 2024-04-02T17:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.631+0000 7ff7425f9740 1 -- start start 2024-04-02T17:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.631+0000 7ff7425f9740 1 Processor -- start 2024-04-02T17:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4add4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 0x564fd4b49480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a83e0 con 0x564fd4b47090 2024-04-02T17:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a7ce0 con 0x564fd4ad4240 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a4ef0 con 0x564fd4acf0d0 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff7425f9740 10 client.0 _ll_register_callbacks cb 0x7ff72c004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.633+0000 7ff73d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.634+0000 7ff73d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33208/0 (socket says 172.21.15.59:33208) 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.634+0000 7ff73d000640 1 -- 172.21.15.59:0/4218818972 learned_addr learned my addr 172.21.15.59:0/4218818972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.634+0000 7ff737e00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4add4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.634+0000 7ff73da00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 0x564fd4b49480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73d000640 1 -- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 msgr2=0x564fd4add4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73d000640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4add4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73d000640 1 -- 172.21.15.59:0/4218818972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 msgr2=0x564fd4b49480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73d000640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 0x564fd4b49480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73d000640 1 -- 172.21.15.59:0/4218818972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd4b499c0 con 0x564fd4acf0d0 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff737e00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4add4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.635+0000 7ff73da00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 0x564fd4b49480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.636+0000 7ff73d000640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff72c003ce0 tx=0x7ff72c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4d785906fc0e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.636+0000 7ff71ea00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.637+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c050040 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.638+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff730004dc0 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.638+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7ff72c004e50 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.638+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff72c0048f0 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.638+0000 7ff7425f9740 10 client.27190 fetch_fsmap learned FSMap version 47 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.638+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564fd4b4abe0 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.639+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730004dc0 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.639+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7ff72c05fc60 con 0x564fd4acf0d0 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.639+0000 7ff7425f9740 10 client.27190 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T17:28:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.639+0000 7ff7425f9740 10 client.27190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.639+0000 7ff7425f9740 20 client.27190 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.640+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4b4b5c0 0x564fd4b4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.640+0000 7ff7425f9740 4 client.27190 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.640+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fd4a392b0 con 0x564fd4b4b5c0 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.640+0000 7ff737e00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4b4b5c0 0x564fd4b4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.641+0000 7ff737e00640 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4b4b5c0 0x564fd4b4d9e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288d7eb5a0be2ddc server_cookie=b3035b9a76db4ef9 in_seq=0 out_seq=0 2024-04-02T17:28:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.641+0000 7ff71ea00640 10 client.27190 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:45.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:45 smithi059 ceph-mon[31664]: pgmap v858: 97 pgs: 97 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.17k op/s 2024-04-02T17:28:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.811+0000 7ff71ea00640 1 -- 172.21.15.59:0/4218818972 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fd4a392b0 con 0x564fd4b4b5c0 2024-04-02T17:28:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff71ea00640 10 client.27190 handle_command_reply: tid=0 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 client.27190 shutdown 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 msgr2=0x564fd4adcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff72c003ce0 tx=0x7ff72c005420 comp rx=0 tx=0).stop 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 shutdown_connections 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564fd4b4b5c0 0x564fd4b4d9e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564fd4b47090 0x564fd4b49480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564fd4ad4240 0x564fd4add4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 --2- 172.21.15.59:0/4218818972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564fd4acf0d0 0x564fd4adcf60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff71ea00640 0 client.27190 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 >> 172.21.15.59:0/4218818972 conn(0x564fd4ac9c50 msgr2=0x564fd4ad8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 shutdown_connections 2024-04-02T17:28:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 1 -- 172.21.15.59:0/4218818972 wait complete. 2024-04-02T17:28:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.812+0000 7ff7425f9740 20 client.27190 trim_cache size 0 max 16384 2024-04-02T17:28:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.813+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 msgr2=0x7ff7200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.813+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 0x7ff7200a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009850 tx=0x7ff72c02df90 comp rx=0 tx=0).stop 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.813+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 msgr2=0x7ff73819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.813+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff72400ed60 tx=0x7ff72400c6a0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 shutdown_connections 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff7200a3e40 0x7ff7200a6300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff738104f50 0x7ff73819a4f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff738105910 0x7ff7381a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 --2- 172.21.15.59:0/170455564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff738103d60 0x7ff738199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 >> 172.21.15.59:0/170455564 conn(0x7ff7380ff2f0 msgr2=0x7ff738108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 shutdown_connections 2024-04-02T17:28:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:45.817+0000 7ff73fa00640 1 -- 172.21.15.59:0/170455564 wait complete. 2024-04-02T17:28:45.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:45 smithi164 ceph-mon[36499]: pgmap v858: 97 pgs: 97 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.17k op/s 2024-04-02T17:28:45.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:45 smithi154 ceph-mon[36479]: pgmap v858: 97 pgs: 97 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.17k op/s 2024-04-02T17:28:46.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 108543 40.27 MB/sec execute 13 sec latency 104.340 ms 2024-04-02T17:28:46.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:46 smithi164 ceph-mon[36499]: pgmap v859: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-04-02T17:28:46.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:46 smithi154 ceph-mon[36479]: pgmap v859: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-04-02T17:28:47.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:46 smithi059 ceph-mon[31664]: pgmap v859: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.05k op/s 2024-04-02T17:28:47.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 109540 41.01 MB/sec execute 14 sec latency 163.055 ms 2024-04-02T17:28:48.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 110530 42.67 MB/sec execute 15 sec latency 37.154 ms 2024-04-02T17:28:49.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:49 smithi164 ceph-mon[36499]: pgmap v860: 97 pgs: 97 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-02T17:28:49.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:49 smithi154 ceph-mon[36479]: pgmap v860: 97 pgs: 97 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-02T17:28:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:49 smithi059 ceph-mon[31664]: pgmap v860: 97 pgs: 97 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-02T17:28:49.599 INFO:tasks.workunit.client.0.smithi059.stdout: 10 111269 42.20 MB/sec execute 16 sec latency 47.865 ms 2024-04-02T17:28:50.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 112120 42.32 MB/sec execute 17 sec latency 38.358 ms 2024-04-02T17:28:51.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:51 smithi164 ceph-mon[36499]: pgmap v861: 97 pgs: 97 active+clean; 594 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 885 op/s 2024-04-02T17:28:51.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:51 smithi154 ceph-mon[36479]: pgmap v861: 97 pgs: 97 active+clean; 594 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 885 op/s 2024-04-02T17:28:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:51 smithi059 ceph-mon[31664]: pgmap v861: 97 pgs: 97 active+clean; 594 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 885 op/s 2024-04-02T17:28:51.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 113199 43.00 MB/sec execute 18 sec latency 40.413 ms 2024-04-02T17:28:52.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 114108 44.13 MB/sec execute 19 sec latency 34.330 ms 2024-04-02T17:28:53.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:53 smithi164 ceph-mon[36499]: pgmap v862: 97 pgs: 97 active+clean; 602 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 793 op/s 2024-04-02T17:28:53.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:53 smithi154 ceph-mon[36479]: pgmap v862: 97 pgs: 97 active+clean; 602 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 793 op/s 2024-04-02T17:28:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:53 smithi059 ceph-mon[31664]: pgmap v862: 97 pgs: 97 active+clean; 602 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 793 op/s 2024-04-02T17:28:53.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 114900 43.47 MB/sec execute 20 sec latency 28.072 ms 2024-04-02T17:28:54.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 115875 43.89 MB/sec execute 21 sec latency 36.973 ms 2024-04-02T17:28:55.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:55 smithi154 ceph-mon[36479]: pgmap v863: 97 pgs: 97 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 802 op/s 2024-04-02T17:28:55.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:55 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:55 smithi059 ceph-mon[31664]: pgmap v863: 97 pgs: 97 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 802 op/s 2024-04-02T17:28:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:55 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:55 smithi164 ceph-mon[36499]: pgmap v863: 97 pgs: 97 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.8 MiB/s wr, 802 op/s 2024-04-02T17:28:55.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:55 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:55.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 116838 44.22 MB/sec execute 22 sec latency 137.956 ms 2024-04-02T17:28:56.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 117688 44.66 MB/sec execute 23 sec latency 48.415 ms 2024-04-02T17:28:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:57 smithi154 ceph-mon[36479]: pgmap v864: 97 pgs: 97 active+clean; 621 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 674 op/s 2024-04-02T17:28:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:57 smithi059 ceph-mon[31664]: pgmap v864: 97 pgs: 97 active+clean; 621 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 674 op/s 2024-04-02T17:28:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:57 smithi164 ceph-mon[36499]: pgmap v864: 97 pgs: 97 active+clean; 621 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 674 op/s 2024-04-02T17:28:57.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 118482 44.15 MB/sec execute 24 sec latency 46.370 ms 2024-04-02T17:28:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:58 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:58 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:58.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:58 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:28:58.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 119523 44.63 MB/sec execute 25 sec latency 38.815 ms 2024-04-02T17:28:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 -- 172.21.15.59:0/3976425029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0104f50 msgr2=0x7f5ba01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/3976425029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0104f50 0x7f5ba01053d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009860 tx=0x7f5b8c02bb70 comp rx=0 tx=0).stop 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 -- 172.21.15.59:0/3976425029 shutdown_connections 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/3976425029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0105910 0x7f5ba010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/3976425029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0104f50 0x7f5ba01053d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/3976425029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 0x7f5ba0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 -- 172.21.15.59:0/3976425029 >> 172.21.15.59:0/3976425029 conn(0x7f5ba00ff2f0 msgr2=0x7f5ba0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 -- 172.21.15.59:0/3976425029 shutdown_connections 2024-04-02T17:28:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.003+0000 7f5ba7c00640 1 -- 172.21.15.59:0/3976425029 wait complete. 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.004+0000 7f5ba7c00640 1 Processor -- start 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 -- start start 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 0x7f5ba0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 0x7f5ba01a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010f9e0 con 0x7f5ba0103d60 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010f860 con 0x7f5ba0104f50 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba7c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba010fb60 con 0x7f5ba0105910 2024-04-02T17:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 0x7f5ba0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53258/0 (socket says 172.21.15.59:53258) 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5ba5c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 0x7f5ba01a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.005+0000 7f5b9fe00640 1 -- 172.21.15.59:0/1886850369 learned_addr learned my addr 172.21.15.59:0/1886850369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 -- 172.21.15.59:0/1886850369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 msgr2=0x7f5ba01a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 0x7f5ba01a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 -- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 msgr2=0x7f5ba0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 0x7f5ba0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 -- 172.21.15.59:0/1886850369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba01a1b90 con 0x7f5ba0104f50 2024-04-02T17:28:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 0x7f5ba01a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005940 tx=0x7f5b8c032470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.006+0000 7f5b9d600640 1 -- 172.21.15.59:0/1886850369 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c041020 con 0x7f5ba0104f50 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.007+0000 7f5b9d600640 1 -- 172.21.15.59:0/1886850369 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c02c9f0 con 0x7f5ba0104f50 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.007+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba01a1e20 con 0x7f5ba0104f50 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.007+0000 7f5b9d600640 1 -- 172.21.15.59:0/1886850369 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c039040 con 0x7f5ba0104f50 2024-04-02T17:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.007+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba01a2290 con 0x7f5ba0104f50 2024-04-02T17:28:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.012+0000 7f5baa83f740 1 Processor -- start 2024-04-02T17:28:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.012+0000 7f5b9d600640 1 -- 172.21.15.59:0/1886850369 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b8c044020 con 0x7f5ba0104f50 2024-04-02T17:28:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.013+0000 7f5baa83f740 1 -- start start 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x5617639f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 0x5617639fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56176395ad70 con 0x5617639f4ed0 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5617639032a0 con 0x5617639faa30 2024-04-02T17:28:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5baa83f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561763956d20 con 0x5617639f58c0 2024-04-02T17:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:38328/0 (socket says 172.21.15.59:38328) 2024-04-02T17:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.014+0000 7f5ba5200640 1 -- 172.21.15.59:0/3506736317 learned_addr learned my addr 172.21.15.59:0/3506736317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5b9d600640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 0x7f5b880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5ba5200640 1 -- 172.21.15.59:0/3506736317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 msgr2=0x5617639fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x5617639f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 0x5617639fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5b9d600640 1 -- 172.21.15.59:0/1886850369 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b8c0f1d30 con 0x7f5ba0104f50 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.015+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 0x5617639fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5200640 1 -- 172.21.15.59:0/3506736317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 msgr2=0x5617639f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x5617639f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5200640 1 -- 172.21.15.59:0/3506736317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561763917840 con 0x5617639f58c0 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 0x5617639fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x5617639f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 0x7f5b880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009c80 tx=0x7f5b94011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcce0a4482d2d428 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5b92000640 1 -- 172.21.15.59:0/3506736317 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94005220 con 0x5617639f58c0 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5b92000640 1 -- 172.21.15.59:0/3506736317 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94009e30 con 0x5617639f58c0 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.016+0000 7f5b92000640 1 -- 172.21.15.59:0/3506736317 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94019830 con 0x5617639f58c0 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 -- 172.21.15.59:0/3506736317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 msgr2=0x5617639fa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 0x7f5b880a6190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5b980099e0 tx=0x7f5b98004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009c80 tx=0x7f5b94011810 comp rx=0 tx=0).stop 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 -- 172.21.15.59:0/3506736317 shutdown_connections 2024-04-02T17:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639faa30 0x5617639fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f58c0 0x5617639fa3c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3506736317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x5617639f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.018+0000 7f5baa83f740 1 -- 172.21.15.59:0/3506736317 >> 172.21.15.59:0/3506736317 conn(0x5617639f0440 msgr2=0x5617639f0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.019+0000 7f5baa83f740 1 -- 172.21.15.59:0/3506736317 shutdown_connections 2024-04-02T17:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.019+0000 7f5baa83f740 1 -- 172.21.15.59:0/3506736317 wait complete. 2024-04-02T17:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.020+0000 7f5baa83f740 1 -- start start 2024-04-02T17:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.020+0000 7f5baa83f740 1 Processor -- start 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x561763a068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5617639fd9e0 con 0x5617639faa30 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561763916660 con 0x5617639f58c0 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5617639fdb60 con 0x5617639f4ed0 2024-04-02T17:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5baa83f740 10 client.0 _ll_register_callbacks cb 0x561763916050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5ba5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x561763a068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53284/0 (socket says 172.21.15.59:53284) 2024-04-02T17:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.022+0000 7f5b9fe00640 1 -- 172.21.15.59:0/2943280786 learned_addr learned my addr 172.21.15.59:0/2943280786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5ba5c00640 1 -- 172.21.15.59:0/2943280786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 msgr2=0x5617639ff320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639ff320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5ba5c00640 1 -- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 msgr2=0x5617639ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5ba5c00640 1 -- 172.21.15.59:0/2943280786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561763925e80 con 0x5617639faa30 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.023+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.024+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x561763a068a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5617639f6120 tx=0x7f5b98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52828d3f28056c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.024+0000 7f5b86a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:28:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.024+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9802c060 con 0x5617639faa30 2024-04-02T17:28:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.025+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b98017030 con 0x5617639faa30 2024-04-02T17:28:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.025+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b94009e30 con 0x5617639faa30 2024-04-02T17:28:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.026+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5b9801fe10 con 0x5617639faa30 2024-04-02T17:28:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.026+0000 7f5baa83f740 10 client.17076 fetch_fsmap learned FSMap version 49 2024-04-02T17:28:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.026+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56176366f210 con 0x5617639faa30 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.026+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94009e30 con 0x5617639faa30 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7f5b98027630 con 0x5617639faa30 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 10 client.17076 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 10 client.17076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 20 client.17076 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a0b290 0x561763a0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 4 client.17076 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:28:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176390acd0 con 0x561763a0b290 2024-04-02T17:28:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.027+0000 7f5ba5200640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a0b290 0x561763a0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.028+0000 7f5ba5200640 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a0b290 0x561763a0d730 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920c62af0e1674f0 server_cookie=b4f80c95be272c2e in_seq=0 out_seq=0 2024-04-02T17:28:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.028+0000 7f5b86a00640 10 client.17076 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.029+0000 7f5b86a00640 1 -- 172.21.15.59:0/2943280786 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56176390acd0 con 0x561763a0b290 2024-04-02T17:28:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.029+0000 7f5b86a00640 10 client.17076 handle_command_reply: tid=0 2024-04-02T17:28:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 client.17076 shutdown 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 msgr2=0x561763a068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x561763a068a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5617639f6120 tx=0x7f5b98004ab0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 shutdown_connections 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a0b290 0x561763a0d730 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x561763a068a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x5617639ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2943280786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 >> 172.21.15.59:0/2943280786 conn(0x5617639f0440 msgr2=0x5617639ef640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5b86a00640 0 client.17076 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 shutdown_connections 2024-04-02T17:28:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.030+0000 7f5baa83f740 1 -- 172.21.15.59:0/2943280786 wait complete. 2024-04-02T17:28:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.031+0000 7f5baa83f740 20 client.17076 trim_cache size 0 max 16384 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 Processor -- start 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 -- start start 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x5617639fdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01ab310 con 0x5617639faa30 2024-04-02T17:28:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01acf40 con 0x5617639f58c0 2024-04-02T17:28:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.057+0000 7f5baa83f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01ab9e0 con 0x5617639f4ed0 2024-04-02T17:28:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.058+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.058+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:38338/0 (socket says 172.21.15.59:38338) 2024-04-02T17:28:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.058+0000 7f5ba5200640 1 -- 172.21.15.59:0/2372475519 learned_addr learned my addr 172.21.15.59:0/2372475519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.059+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.059+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x5617639fdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 -- 172.21.15.59:0/2372475519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 msgr2=0x561763a06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 -- 172.21.15.59:0/2372475519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 msgr2=0x5617639fdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x5617639fdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 -- 172.21.15.59:0/2372475519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176373b620 con 0x5617639f4ed0 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x5617639fdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:28:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5ba5200640 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401b040 tx=0x7f5b94011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be2a43c8990da6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5b91600640 1 -- 172.21.15.59:0/2372475519 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94019430 con 0x5617639f4ed0 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5b91600640 1 -- 172.21.15.59:0/2372475519 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94009870 con 0x5617639f4ed0 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.061+0000 7f5b91600640 1 -- 172.21.15.59:0/2372475519 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94025910 con 0x5617639f4ed0 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 -- 172.21.15.59:0/2372475519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 msgr2=0x5617639f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401b040 tx=0x7f5b94011dc0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 -- 172.21.15.59:0/2372475519 shutdown_connections 2024-04-02T17:28:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639faa30 0x5617639fdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 --2- 172.21.15.59:0/2372475519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639f4ed0 0x5617639f3db0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.062+0000 7f5baa83f740 1 -- 172.21.15.59:0/2372475519 >> 172.21.15.59:0/2372475519 conn(0x5617639f0440 msgr2=0x5617639f36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.063+0000 7f5baa83f740 1 -- 172.21.15.59:0/2372475519 shutdown_connections 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.063+0000 7f5baa83f740 1 -- 172.21.15.59:0/2372475519 wait complete. 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.063+0000 7f5baa83f740 1 -- start start 2024-04-02T17:28:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.063+0000 7f5baa83f740 1 Processor -- start 2024-04-02T17:28:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639faa30 0x561763a050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01ac7b0 con 0x5617639f4ed0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01ac0e0 con 0x5617639f58c0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba01a92f0 con 0x5617639faa30 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5baa83f740 10 client.0 _ll_register_callbacks cb 0x7f5b9800fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5ba5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41838/0 (socket says 172.21.15.59:41838) 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5ba5200640 1 -- 172.21.15.59:0/3435423748 learned_addr learned my addr 172.21.15.59:0/3435423748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.064+0000 7f5ba5c00640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639faa30 0x561763a050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 -- 172.21.15.59:0/3435423748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639faa30 msgr2=0x561763a050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639faa30 0x561763a050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 -- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 msgr2=0x561763a040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 -- 172.21.15.59:0/3435423748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561763a056d0 con 0x5617639f4ed0 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5ba5200640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000c00 tx=0x7f5b94009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fb68a10f3901cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5b86a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.065+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94009e30 con 0x5617639f4ed0 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5b940196d0 con 0x5617639f4ed0 2024-04-02T17:28:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b94009870 con 0x5617639f4ed0 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5b940198d0 con 0x5617639f4ed0 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5baa83f740 10 client.17079 fetch_fsmap learned FSMap version 49 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561763a06950 con 0x5617639f4ed0 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.066+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94009870 con 0x5617639f4ed0 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.067+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7f5b94023400 con 0x5617639f4ed0 2024-04-02T17:28:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 10 client.17079 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 20 client.17079 populate_metadata read hostname 'smithi059' 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a71cb0 0x561763a740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 4 client.17079 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617639682b0 con 0x561763a71cb0 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.068+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a71cb0 0x561763a740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.069+0000 7f5b86a00640 10 client.17079 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.069+0000 7f5b9fe00640 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a71cb0 0x561763a740d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aceb2f16553a83c server_cookie=598ddefdfc1fd1e1 in_seq=0 out_seq=0 2024-04-02T17:28:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.328+0000 7f5b86a00640 1 -- 172.21.15.59:0/3435423748 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617639682b0 con 0x561763a71cb0 2024-04-02T17:28:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.328+0000 7f5b86a00640 10 client.17079 handle_command_reply: tid=0 2024-04-02T17:28:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.329+0000 7f5baa83f740 1 client.17079 shutdown 2024-04-02T17:28:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.329+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 msgr2=0x561763a03b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.329+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000c00 tx=0x7f5b94009510 comp rx=0 tx=0).stop 2024-04-02T17:28:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 shutdown_connections 2024-04-02T17:28:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561763a71cb0 0x561763a740d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617639faa30 0x561763a050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5617639f58c0 0x561763a040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 --2- 172.21.15.59:0/3435423748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5617639f4ed0 0x561763a03b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5b86a00640 0 client.17079 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 >> 172.21.15.59:0/3435423748 conn(0x5617639f0440 msgr2=0x5617639f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 shutdown_connections 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 1 -- 172.21.15.59:0/3435423748 wait complete. 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.331+0000 7f5baa83f740 20 client.17079 trim_cache size 0 max 16384 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:28:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:28:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.334+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 msgr2=0x7f5b880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.334+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 0x7f5b880a6190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5b980099e0 tx=0x7f5b98004a40 comp rx=0 tx=0).stop 2024-04-02T17:28:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.334+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 msgr2=0x7f5ba019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:28:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.334+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005940 tx=0x7f5b8c032470 comp rx=0 tx=0).stop 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 shutdown_connections 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ba0105910 0x7f5ba01a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba0104f50 0x7f5ba019a4b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f5b880a3cd0 0x7f5b880a6190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 --2- 172.21.15.59:0/1886850369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ba0103d60 0x7f5ba0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:28:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.335+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 >> 172.21.15.59:0/1886850369 conn(0x7f5ba00ff2f0 msgr2=0x7f5ba0108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:28:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.339+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 shutdown_connections 2024-04-02T17:28:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:28:59.340+0000 7f5ba7c00640 1 -- 172.21.15.59:0/1886850369 wait complete. 2024-04-02T17:28:59.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:28:59 smithi154 ceph-mon[36479]: pgmap v865: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.5 MiB/s wr, 756 op/s 2024-04-02T17:28:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:28:59 smithi059 ceph-mon[31664]: pgmap v865: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.5 MiB/s wr, 756 op/s 2024-04-02T17:28:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:28:59 smithi164 ceph-mon[36499]: pgmap v865: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.5 MiB/s wr, 756 op/s 2024-04-02T17:28:59.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 120304 44.66 MB/sec execute 26 sec latency 195.260 ms 2024-04-02T17:29:00.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 120912 44.57 MB/sec execute 27 sec latency 308.866 ms 2024-04-02T17:29:01.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:01 smithi154 ceph-mon[36479]: pgmap v866: 97 pgs: 97 active+clean; 635 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-04-02T17:29:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:01 smithi059 ceph-mon[31664]: pgmap v866: 97 pgs: 97 active+clean; 635 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-04-02T17:29:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:01 smithi164 ceph-mon[36499]: pgmap v866: 97 pgs: 97 active+clean; 635 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 604 op/s 2024-04-02T17:29:01.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 121137 43.45 MB/sec execute 28 sec latency 436.166 ms 2024-04-02T17:29:02.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 121589 42.50 MB/sec execute 29 sec latency 187.992 ms 2024-04-02T17:29:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:03 smithi154 ceph-mon[36479]: pgmap v867: 97 pgs: 97 active+clean; 645 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-04-02T17:29:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:03 smithi059 ceph-mon[31664]: pgmap v867: 97 pgs: 97 active+clean; 645 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-04-02T17:29:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:03 smithi164 ceph-mon[36499]: pgmap v867: 97 pgs: 97 active+clean; 645 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-04-02T17:29:03.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 122436 42.35 MB/sec execute 30 sec latency 60.590 ms 2024-04-02T17:29:04.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 123630 43.07 MB/sec execute 31 sec latency 35.100 ms 2024-04-02T17:29:05.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:05 smithi154 ceph-mon[36479]: pgmap v868: 97 pgs: 97 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 782 op/s 2024-04-02T17:29:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:05 smithi059 ceph-mon[31664]: pgmap v868: 97 pgs: 97 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 782 op/s 2024-04-02T17:29:05.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:05 smithi164 ceph-mon[36499]: pgmap v868: 97 pgs: 97 active+clean; 643 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 782 op/s 2024-04-02T17:29:05.600 INFO:tasks.workunit.client.0.smithi059.stdout: 10 124461 43.64 MB/sec execute 32 sec latency 44.696 ms 2024-04-02T17:29:06.403 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:06 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:06.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:06 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:06.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:06 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:06.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 124927 42.91 MB/sec execute 33 sec latency 246.339 ms 2024-04-02T17:29:07.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:07 smithi154 ceph-mon[36479]: pgmap v869: 97 pgs: 97 active+clean; 648 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 861 op/s 2024-04-02T17:29:07.449 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:07 smithi059 ceph-mon[31664]: pgmap v869: 97 pgs: 97 active+clean; 648 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 861 op/s 2024-04-02T17:29:07.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:07 smithi164 ceph-mon[36499]: pgmap v869: 97 pgs: 97 active+clean; 648 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 861 op/s 2024-04-02T17:29:07.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 125001 41.70 MB/sec execute 34 sec latency 608.875 ms 2024-04-02T17:29:08.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 125452 41.05 MB/sec execute 35 sec latency 111.005 ms 2024-04-02T17:29:09.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:09 smithi154 ceph-mon[36479]: pgmap v870: 97 pgs: 97 active+clean; 665 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 895 op/s 2024-04-02T17:29:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:09 smithi059 ceph-mon[31664]: pgmap v870: 97 pgs: 97 active+clean; 665 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 895 op/s 2024-04-02T17:29:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:09 smithi164 ceph-mon[36499]: pgmap v870: 97 pgs: 97 active+clean; 665 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 895 op/s 2024-04-02T17:29:09.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 126362 41.30 MB/sec execute 36 sec latency 134.467 ms 2024-04-02T17:29:10.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 127424 41.70 MB/sec execute 37 sec latency 50.531 ms 2024-04-02T17:29:11.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:11 smithi154 ceph-mon[36479]: pgmap v871: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-04-02T17:29:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:11 smithi059 ceph-mon[31664]: pgmap v871: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-04-02T17:29:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:11 smithi164 ceph-mon[36499]: pgmap v871: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 959 op/s 2024-04-02T17:29:11.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 128280 42.03 MB/sec execute 38 sec latency 39.638 ms 2024-04-02T17:29:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 -- 172.21.15.59:0/1086940561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454104ed0 msgr2=0x7f54541072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 --2- 172.21.15.59:0/1086940561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454104ed0 0x7f54541072c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f544000dd10 tx=0x7f54400341c0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 -- 172.21.15.59:0/1086940561 shutdown_connections 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 --2- 172.21.15.59:0/1086940561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454107800 0x7f5454109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 --2- 172.21.15.59:0/1086940561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454104ed0 0x7f54541072c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 --2- 172.21.15.59:0/1086940561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54541025a0 0x7f5454104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 -- 172.21.15.59:0/1086940561 >> 172.21.15.59:0/1086940561 conn(0x7f54540fbf90 msgr2=0x7f54540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.546+0000 7f545b600640 1 -- 172.21.15.59:0/1086940561 shutdown_connections 2024-04-02T17:29:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.547+0000 7f545b600640 1 -- 172.21.15.59:0/1086940561 wait complete. 2024-04-02T17:29:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.548+0000 7f545b600640 1 Processor -- start 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 -- start start 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454104ed0 0x7f5454196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5454107800 0x7f545419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f545410b590 con 0x7f5454107800 2024-04-02T17:29:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f545410b410 con 0x7f5454104ed0 2024-04-02T17:29:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.550+0000 7f545b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f545410b710 con 0x7f54541025a0 2024-04-02T17:29:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5458c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5458c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46780/0 (socket says 172.21.15.59:46780) 2024-04-02T17:29:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5458c00640 1 -- 172.21.15.59:0/821275417 learned_addr learned my addr 172.21.15.59:0/821275417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5453e00640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454104ed0 0x7f5454196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5458c00640 1 -- 172.21.15.59:0/821275417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454104ed0 msgr2=0x7f5454196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.551+0000 7f5458c00640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454104ed0 0x7f5454196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5458c00640 1 -- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5454107800 msgr2=0x7f545419d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5458c00640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5454107800 0x7f545419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5458c00640 1 -- 172.21.15.59:0/821275417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545419d720 con 0x7f54541025a0 2024-04-02T17:29:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5458c00640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f544800ea60 tx=0x7f544800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5451600640 1 -- 172.21.15.59:0/821275417 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800cde0 con 0x7f54541025a0 2024-04-02T17:29:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5451600640 1 -- 172.21.15.59:0/821275417 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54480035f0 con 0x7f54541025a0 2024-04-02T17:29:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.552+0000 7f5451600640 1 -- 172.21.15.59:0/821275417 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448010670 con 0x7f54541025a0 2024-04-02T17:29:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.553+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545419da10 con 0x7f54541025a0 2024-04-02T17:29:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.553+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545419de80 con 0x7f54541025a0 2024-04-02T17:29:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.559+0000 7f5451600640 1 -- 172.21.15.59:0/821275417 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5448004020 con 0x7f54541025a0 2024-04-02T17:29:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.560+0000 7f545e26e740 1 Processor -- start 2024-04-02T17:29:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.560+0000 7f545e26e740 1 -- start start 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 0x561aedf8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 0x561aedf96ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561aedef4d70 con 0x561aedf94a30 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561aede9d2a0 con 0x561aedf8f8c0 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f545e26e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aedef0d20 con 0x561aedf8eed0 2024-04-02T17:29:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.561+0000 7f5451600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 0x7f543c0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.562+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 0x561aedf8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.562+0000 7f5451600640 1 -- 172.21.15.59:0/821275417 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f54480d0830 con 0x7f54541025a0 2024-04-02T17:29:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.562+0000 7f5453e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 0x561aedf96ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.563+0000 7f5459600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 0x7f543c0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.563+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 0x561aedf8f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46788/0 (socket says 172.21.15.59:46788) 2024-04-02T17:29:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.563+0000 7f5459600640 1 -- 172.21.15.59:0/1117119429 learned_addr learned my addr 172.21.15.59:0/1117119429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.564+0000 7f5453e00640 1 -- 172.21.15.59:0/1117119429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 msgr2=0x561aedf8f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.564+0000 7f5453e00640 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 0x561aedf8f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.565+0000 7f5458c00640 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.564+0000 7f5453e00640 1 -- 172.21.15.59:0/1117119429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 msgr2=0x561aedf943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.565+0000 7f5453e00640 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf943c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.565+0000 7f5453e00640 1 -- 172.21.15.59:0/1117119429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aedeb1840 con 0x561aedf94a30 2024-04-02T17:29:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.565+0000 7f5458c00640 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.566+0000 7f5453e00640 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 0x561aedf96ec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f54400347d0 tx=0x7f544003c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3bd04930e02e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.567+0000 7f5445c00640 1 -- 172.21.15.59:0/1117119429 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544004c070 con 0x561aedf94a30 2024-04-02T17:29:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.567+0000 7f5445c00640 1 -- 172.21.15.59:0/1117119429 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544003cb40 con 0x561aedf94a30 2024-04-02T17:29:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.567+0000 7f5445c00640 1 -- 172.21.15.59:0/1117119429 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440005290 con 0x561aedf94a30 2024-04-02T17:29:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.567+0000 7f5459600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 0x7f543c0a6110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x561aedf90120 tx=0x7f544c005e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.568+0000 7f545e26e740 1 -- 172.21.15.59:0/1117119429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 msgr2=0x561aedf96ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 0x561aedf96ec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f54400347d0 tx=0x7f544003c460 comp rx=0 tx=0).stop 2024-04-02T17:29:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 -- 172.21.15.59:0/1117119429 shutdown_connections 2024-04-02T17:29:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf94a30 0x561aedf96ec0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 --2- 172.21.15.59:0/1117119429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8eed0 0x561aedf8f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.569+0000 7f545e26e740 1 -- 172.21.15.59:0/1117119429 >> 172.21.15.59:0/1117119429 conn(0x561aedf8a440 msgr2=0x561aedf8a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.570+0000 7f545e26e740 1 -- 172.21.15.59:0/1117119429 shutdown_connections 2024-04-02T17:29:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.570+0000 7f545e26e740 1 -- 172.21.15.59:0/1117119429 wait complete. 2024-04-02T17:29:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.571+0000 7f545e26e740 1 -- start start 2024-04-02T17:29:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.571+0000 7f545e26e740 1 Processor -- start 2024-04-02T17:29:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.573+0000 7f545e26e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf99420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.573+0000 7f545e26e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.573+0000 7f545e26e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f545e26e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561aedf97980 con 0x561aedf8eed0 2024-04-02T17:29:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f545e26e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561aedeb0660 con 0x561aedf8f8c0 2024-04-02T17:29:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f545e26e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aedf97b00 con 0x561aedf94a30 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f545e26e740 10 client.0 _ll_register_callbacks cb 0x561aedeb0050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46804/0 (socket says 172.21.15.59:46804) 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.574+0000 7f5459600640 1 -- 172.21.15.59:0/1649735605 learned_addr learned my addr 172.21.15.59:0/1649735605 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5453e00640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5459600640 1 -- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 msgr2=0x561aedf99960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5459600640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf99960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5459600640 1 -- 172.21.15.59:0/1649735605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 msgr2=0x561aedf99420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5459600640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf99420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.575+0000 7f5459600640 1 -- 172.21.15.59:0/1649735605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aedebfe80 con 0x561aedf94a30 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.576+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544003cb40 con 0x561aedf94a30 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.576+0000 7f5459600640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f544c010ed0 tx=0x7f544c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294780f185a787f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.576+0000 7f543a000640 10 client.27217 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.577+0000 7f543a000640 1 -- 172.21.15.59:0/1649735605 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c022020 con 0x561aedf94a30 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.577+0000 7f543a000640 1 -- 172.21.15.59:0/1649735605 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f544c01e640 con 0x561aedf94a30 2024-04-02T17:29:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.577+0000 7f543a000640 1 -- 172.21.15.59:0/1649735605 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f544c01e880 con 0x561aedf94a30 2024-04-02T17:29:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.580+0000 7f545e26e740 10 client.27217 fetch_fsmap learned FSMap version 50 2024-04-02T17:29:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.580+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=50}) -- 0x561aedc09210 con 0x561aedf94a30 2024-04-02T17:29:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f543a000640 1 -- 172.21.15.59:0/1649735605 <== mon.1 v2:172.21.15.164:3300/0 4 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f544c00f040 con 0x561aedf94a30 2024-04-02T17:29:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f545e26e740 10 client.27217 fetch_fsmap finished waiting for FSMap version 50 2024-04-02T17:29:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f545e26e740 10 client.27217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f545e26e740 20 client.27217 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aedfa5160 0x561aedfa75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.581+0000 7f545e26e740 4 client.27217 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:29:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.582+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aedea4cd0 con 0x561aedfa5160 2024-04-02T17:29:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.582+0000 7f5458c00640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aedfa5160 0x561aedfa75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.583+0000 7f5458c00640 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aedfa5160 0x561aedfa75f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed53c47677d5aa1 server_cookie=e7c0effcb8975458 in_seq=0 out_seq=0 2024-04-02T17:29:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.583+0000 7f543a000640 10 client.27217 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.585+0000 7f543a000640 1 -- 172.21.15.59:0/1649735605 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561aedea4cd0 con 0x561aedfa5160 2024-04-02T17:29:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.585+0000 7f543a000640 10 client.27217 handle_command_reply: tid=0 2024-04-02T17:29:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.585+0000 7f545e26e740 1 client.27217 shutdown 2024-04-02T17:29:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 msgr2=0x561aedfa09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f544c010ed0 tx=0x7f544c004a40 comp rx=0 tx=0).stop 2024-04-02T17:29:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 shutdown_connections 2024-04-02T17:29:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aedfa5160 0x561aedfa75f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedfa09a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedf99960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 --2- 172.21.15.59:0/1649735605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 >> 172.21.15.59:0/1649735605 conn(0x561aedf8a440 msgr2=0x561aedf89640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.586+0000 7f543a000640 0 client.27217 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.588+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 shutdown_connections 2024-04-02T17:29:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.588+0000 7f545e26e740 1 -- 172.21.15.59:0/1649735605 wait complete. 2024-04-02T17:29:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.588+0000 7f545e26e740 20 client.27217 trim_cache size 0 max 16384 2024-04-02T17:29:12.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 129035 41.89 MB/sec execute 39 sec latency 72.648 ms 2024-04-02T17:29:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.616+0000 7f545e26e740 1 Processor -- start 2024-04-02T17:29:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.616+0000 7f545e26e740 1 -- start start 2024-04-02T17:29:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedfa7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedfa0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a6f00 con 0x561aedf8eed0 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a8b00 con 0x561aedf8f8c0 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f545e26e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a75d0 con 0x561aedf94a30 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f5459600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:46818/0 (socket says 172.21.15.59:46818) 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f5459600640 1 -- 172.21.15.59:0/3858147325 learned_addr learned my addr 172.21.15.59:0/3858147325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f5453e00640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedfa0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.617+0000 7f5458c00640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedfa7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 -- 172.21.15.59:0/3858147325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 msgr2=0x561aedfa0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedfa0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 -- 172.21.15.59:0/3858147325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 msgr2=0x561aedfa7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedfa7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 -- 172.21.15.59:0/3858147325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aedcd5620 con 0x561aedf94a30 2024-04-02T17:29:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5458c00640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedfa7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.618+0000 7f5459600640 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f544c011250 tx=0x7f544c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38641890058186c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.619+0000 7f5445200640 1 -- 172.21.15.59:0/3858147325 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c022040 con 0x561aedf94a30 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.619+0000 7f5445200640 1 -- 172.21.15.59:0/3858147325 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544003cb40 con 0x561aedf94a30 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.619+0000 7f5445200640 1 -- 172.21.15.59:0/3858147325 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c039400 con 0x561aedf94a30 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 -- 172.21.15.59:0/3858147325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 msgr2=0x561aedf88db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f544c011250 tx=0x7f544c01d000 comp rx=0 tx=0).stop 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 -- 172.21.15.59:0/3858147325 shutdown_connections 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf94a30 0x561aedf88db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf8f8c0 0x561aedfa0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 --2- 172.21.15.59:0/3858147325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedfa7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.620+0000 7f545e26e740 1 -- 172.21.15.59:0/3858147325 >> 172.21.15.59:0/3858147325 conn(0x561aedf8a440 msgr2=0x561aedf97ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.621+0000 7f545e26e740 1 -- 172.21.15.59:0/3858147325 shutdown_connections 2024-04-02T17:29:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.621+0000 7f545e26e740 1 -- 172.21.15.59:0/3858147325 wait complete. 2024-04-02T17:29:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.621+0000 7f545e26e740 1 -- start start 2024-04-02T17:29:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.621+0000 7f545e26e740 1 Processor -- start 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf9d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 0x561aedf9de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 0x561aee009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a8370 con 0x561aedf8eed0 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a7ca0 con 0x561aedf94a30 2024-04-02T17:29:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f54541a4ee0 con 0x561aedf8f8c0 2024-04-02T17:29:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.622+0000 7f545e26e740 10 client.0 _ll_register_callbacks cb 0x7f544c010b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5459600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 0x561aee009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5459600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 0x561aee009760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50384/0 (socket says 172.21.15.59:50384) 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5458c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf9d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5458c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf9d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34584/0 (socket says 172.21.15.59:34584) 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5458c00640 1 -- 172.21.15.59:0/1154676193 learned_addr learned my addr 172.21.15.59:0/1154676193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.623+0000 7f5453e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 0x561aedf9de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.624+0000 7f5453e00640 1 -- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 msgr2=0x561aee009760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.624+0000 7f5453e00640 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 0x561aee009760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.624+0000 7f5453e00640 1 -- 172.21.15.59:0/1154676193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 msgr2=0x561aedf9d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.624+0000 7f5453e00640 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf9d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.624+0000 7f5453e00640 1 -- 172.21.15.59:0/1154676193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aee009d20 con 0x561aedf8f8c0 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.625+0000 7f5453e00640 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 0x561aedf9de60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f544000dd10 tx=0x7f544000d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf520870e838caf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.625+0000 7f543a000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:29:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.625+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440056070 con 0x561aedf8f8c0 2024-04-02T17:29:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.625+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f5440004040 con 0x561aedf8f8c0 2024-04-02T17:29:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.626+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544003cb40 con 0x561aedf8f8c0 2024-04-02T17:29:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.626+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f544004c040 con 0x561aedf8f8c0 2024-04-02T17:29:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.626+0000 7f545e26e740 10 client.27223 fetch_fsmap learned FSMap version 50 2024-04-02T17:29:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.627+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561aee00b010 con 0x561aedf8f8c0 2024-04-02T17:29:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.627+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544003cb40 con 0x561aedf8f8c0 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 50) ==== 8370+0+0 (secure 0 0 0) 0x7f5440059020 con 0x561aedf8f8c0 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 10 client.27223 fetch_fsmap finished waiting for FSMap version 50 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 10 client.27223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 20 client.27223 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aee00b570 0x561aee00d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 4 client.27223 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:29:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.628+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aedf022b0 con 0x561aee00b570 2024-04-02T17:29:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.630+0000 7f5458c00640 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aee00b570 0x561aee00d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.633+0000 7f5458c00640 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aee00b570 0x561aee00d990 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5d08d1c469c97d server_cookie=14580c7f6fd931e0 in_seq=0 out_seq=0 2024-04-02T17:29:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.633+0000 7f543a000640 10 client.27223 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.807+0000 7f543a000640 1 -- 172.21.15.59:0/1154676193 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aedf022b0 con 0x561aee00b570 2024-04-02T17:29:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.807+0000 7f543a000640 10 client.27223 handle_command_reply: tid=0 2024-04-02T17:29:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.807+0000 7f545e26e740 1 client.27223 shutdown 2024-04-02T17:29:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.808+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 msgr2=0x561aedf9de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.808+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 0x561aedf9de60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f544000dd10 tx=0x7f544000d710 comp rx=0 tx=0).stop 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 shutdown_connections 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561aee00b570 0x561aee00d990 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561aedf94a30 0x561aee009760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aedf8f8c0 0x561aedf9de60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 --2- 172.21.15.59:0/1154676193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561aedf8eed0 0x561aedf9d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f543a000640 0 client.27223 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 >> 172.21.15.59:0/1154676193 conn(0x561aedf8a440 msgr2=0x561aedf8d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 shutdown_connections 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 1 -- 172.21.15.59:0/1154676193 wait complete. 2024-04-02T17:29:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.809+0000 7f545e26e740 20 client.27223 trim_cache size 0 max 16384 2024-04-02T17:29:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:29:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:29:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:29:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:29:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 msgr2=0x7f543c0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 0x7f543c0a6110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x561aedf90120 tx=0x7f544c005e90 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 msgr2=0x7f5454195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f544800ea60 tx=0x7f544800ef20 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 shutdown_connections 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f543c0a3c50 0x7f543c0a6110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5454107800 0x7f545419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5454104ed0 0x7f5454196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 --2- 172.21.15.59:0/821275417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54541025a0 0x7f5454195b00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 >> 172.21.15.59:0/821275417 conn(0x7f54540fbf90 msgr2=0x7f5454102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 shutdown_connections 2024-04-02T17:29:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:12.811+0000 7f545b600640 1 -- 172.21.15.59:0/821275417 wait complete. 2024-04-02T17:29:13.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:13 smithi154 ceph-mon[36479]: pgmap v872: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-04-02T17:29:13.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:13 smithi059 ceph-mon[31664]: pgmap v872: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-04-02T17:29:13.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:13 smithi164 ceph-mon[36499]: pgmap v872: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-04-02T17:29:13.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 129716 41.76 MB/sec execute 40 sec latency 116.469 ms 2024-04-02T17:29:14.603 INFO:tasks.workunit.client.0.smithi059.stdout: 10 130667 41.91 MB/sec execute 41 sec latency 41.885 ms 2024-04-02T17:29:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:15 smithi154 ceph-mon[36479]: pgmap v873: 97 pgs: 97 active+clean; 642 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.12k op/s 2024-04-02T17:29:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:15 smithi059 ceph-mon[31664]: pgmap v873: 97 pgs: 97 active+clean; 642 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.12k op/s 2024-04-02T17:29:15.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:15 smithi164 ceph-mon[36499]: pgmap v873: 97 pgs: 97 active+clean; 642 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.12k op/s 2024-04-02T17:29:15.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 131560 42.11 MB/sec execute 42 sec latency 38.845 ms 2024-04-02T17:29:16.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 132303 42.20 MB/sec execute 43 sec latency 42.238 ms 2024-04-02T17:29:17.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:17 smithi154 ceph-mon[36479]: pgmap v874: 97 pgs: 97 active+clean; 648 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-02T17:29:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:17 smithi059 ceph-mon[31664]: pgmap v874: 97 pgs: 97 active+clean; 648 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-02T17:29:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:17 smithi164 ceph-mon[36499]: pgmap v874: 97 pgs: 97 active+clean; 648 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-02T17:29:17.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 133296 42.29 MB/sec execute 44 sec latency 52.819 ms 2024-04-02T17:29:18.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 134354 42.66 MB/sec execute 45 sec latency 44.734 ms 2024-04-02T17:29:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:19 smithi059 ceph-mon[31664]: pgmap v875: 97 pgs: 97 active+clean; 653 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 992 op/s 2024-04-02T17:29:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:19 smithi164 ceph-mon[36499]: pgmap v875: 97 pgs: 97 active+clean; 653 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 992 op/s 2024-04-02T17:29:19.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 135159 42.77 MB/sec execute 46 sec latency 50.715 ms 2024-04-02T17:29:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:19 smithi154 ceph-mon[36479]: pgmap v875: 97 pgs: 97 active+clean; 653 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 992 op/s 2024-04-02T17:29:20.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 135880 42.61 MB/sec execute 47 sec latency 67.815 ms 2024-04-02T17:29:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:21 smithi059 ceph-mon[31664]: pgmap v876: 97 pgs: 97 active+clean; 657 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 922 op/s 2024-04-02T17:29:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:21 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:21 smithi164 ceph-mon[36499]: pgmap v876: 97 pgs: 97 active+clean; 657 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 922 op/s 2024-04-02T17:29:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:21 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:21.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 136842 42.73 MB/sec execute 48 sec latency 31.199 ms 2024-04-02T17:29:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:21 smithi154 ceph-mon[36479]: pgmap v876: 97 pgs: 97 active+clean; 657 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 922 op/s 2024-04-02T17:29:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:21 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:22.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 137934 43.16 MB/sec execute 49 sec latency 34.795 ms 2024-04-02T17:29:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:23 smithi059 ceph-mon[31664]: pgmap v877: 97 pgs: 97 active+clean; 663 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 801 op/s 2024-04-02T17:29:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:23 smithi164 ceph-mon[36499]: pgmap v877: 97 pgs: 97 active+clean; 663 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 801 op/s 2024-04-02T17:29:23.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 138697 43.17 MB/sec execute 50 sec latency 49.296 ms 2024-04-02T17:29:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:23 smithi154 ceph-mon[36479]: pgmap v877: 97 pgs: 97 active+clean; 663 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 801 op/s 2024-04-02T17:29:24.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 139452 42.98 MB/sec execute 51 sec latency 34.452 ms 2024-04-02T17:29:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:25 smithi059 ceph-mon[31664]: pgmap v878: 97 pgs: 97 active+clean; 676 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 765 op/s 2024-04-02T17:29:25.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:25 smithi164 ceph-mon[36499]: pgmap v878: 97 pgs: 97 active+clean; 676 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 765 op/s 2024-04-02T17:29:25.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 140238 42.87 MB/sec execute 52 sec latency 240.267 ms 2024-04-02T17:29:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:25 smithi154 ceph-mon[36479]: pgmap v878: 97 pgs: 97 active+clean; 676 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 765 op/s 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 -- 172.21.15.59:0/2945668326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c101b60 msgr2=0x7f297c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 --2- 172.21.15.59:0/2945668326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c101b60 0x7f297c101f80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f296c009900 tx=0x7f296c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 -- 172.21.15.59:0/2945668326 shutdown_connections 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 --2- 172.21.15.59:0/2945668326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 --2- 172.21.15.59:0/2945668326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c103710 0x7f297c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 --2- 172.21.15.59:0/2945668326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c101b60 0x7f297c101f80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.001+0000 7f2983000640 1 -- 172.21.15.59:0/2945668326 >> 172.21.15.59:0/2945668326 conn(0x7f297c0fd0d0 msgr2=0x7f297c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.002+0000 7f2983000640 1 -- 172.21.15.59:0/2945668326 shutdown_connections 2024-04-02T17:29:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.002+0000 7f2983000640 1 -- 172.21.15.59:0/2945668326 wait complete. 2024-04-02T17:29:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.004+0000 7f2983000640 1 Processor -- start 2024-04-02T17:29:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.004+0000 7f2983000640 1 -- start start 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.004+0000 7f2983000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c101b60 0x7f297c195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.004+0000 7f2983000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2983000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 0x7f297c1ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2983000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f297c10b690 con 0x7f297c102d50 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2983000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f297c10b510 con 0x7f297c103710 2024-04-02T17:29:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2983000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f297c10b810 con 0x7f297c101b60 2024-04-02T17:29:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f297b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f297b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55208/0 (socket says 172.21.15.59:55208) 2024-04-02T17:29:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f297b400640 1 -- 172.21.15.59:0/553399920 learned_addr learned my addr 172.21.15.59:0/553399920 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2981000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 0x7f297c1ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f297be00640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c101b60 0x7f297c195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2981000640 1 -- 172.21.15.59:0/553399920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c101b60 msgr2=0x7f297c195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2981000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c101b60 0x7f297c195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2981000640 1 -- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 msgr2=0x7f297c1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.005+0000 7f2981000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2981000640 1 -- 172.21.15.59:0/553399920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c199300 con 0x7f297c103710 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2981000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 0x7f297c1ae5b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2974009fd0 tx=0x7f297400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2978c00640 1 -- 172.21.15.59:0/553399920 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297400ce20 con 0x7f297c103710 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2978c00640 1 -- 172.21.15.59:0/553399920 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2974005020 con 0x7f297c103710 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2978c00640 1 -- 172.21.15.59:0/553399920 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974010720 con 0x7f297c103710 2024-04-02T17:29:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.006+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1aebb0 con 0x7f297c103710 2024-04-02T17:29:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.007+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1af080 con 0x7f297c103710 2024-04-02T17:29:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.011+0000 7f2978c00640 1 -- 172.21.15.59:0/553399920 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2974005540 con 0x7f297c103710 2024-04-02T17:29:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.012+0000 7f2985e18740 1 Processor -- start 2024-04-02T17:29:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.013+0000 7f2985e18740 1 -- start start 2024-04-02T17:29:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2978c00640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 0x7f29600a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 0x55891a6e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 0x55891a6e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55891a642d70 con 0x55891a6e2a30 2024-04-02T17:29:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55891a5eb2a0 con 0x55891a6dd8c0 2024-04-02T17:29:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2985e18740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55891a63ed20 con 0x55891a6dced0 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.014+0000 7f2978c00640 1 -- 172.21.15.59:0/553399920 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f29740d0a20 con 0x7f297c103710 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.015+0000 7f297b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 0x55891a6e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.015+0000 7f297b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 0x55891a6e23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40022/0 (socket says 172.21.15.59:40022) 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.015+0000 7f297b400640 1 -- 172.21.15.59:0/4086009495 learned_addr learned my addr 172.21.15.59:0/4086009495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.015+0000 7f297b400640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 0x7f29600a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.016+0000 7f2981000640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 0x55891a6e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.016+0000 7f297be00640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.016+0000 7f2981000640 1 -- 172.21.15.59:0/4086009495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 msgr2=0x55891a6dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.017+0000 7f2981000640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6dd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.017+0000 7f2981000640 1 -- 172.21.15.59:0/4086009495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 msgr2=0x55891a6e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.017+0000 7f2981000640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 0x55891a6e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.017+0000 7f2981000640 1 -- 172.21.15.59:0/4086009495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891a5ff840 con 0x55891a6e2a30 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.017+0000 7f297be00640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.018+0000 7f2981000640 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 0x55891a6e4ec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f29740ce460 tx=0x7f297409c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a565937bc9d78b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.018+0000 7f2971600640 1 -- 172.21.15.59:0/4086009495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740a58b0 con 0x55891a6e2a30 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.018+0000 7f2971600640 1 -- 172.21.15.59:0/4086009495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297409af40 con 0x55891a6e2a30 2024-04-02T17:29:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.018+0000 7f2971600640 1 -- 172.21.15.59:0/4086009495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974099430 con 0x55891a6e2a30 2024-04-02T17:29:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.019+0000 7f2985e18740 1 -- 172.21.15.59:0/4086009495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 msgr2=0x55891a6e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.019+0000 7f2985e18740 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 0x55891a6e4ec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f29740ce460 tx=0x7f297409c690 comp rx=0 tx=0).stop 2024-04-02T17:29:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.020+0000 7f2985e18740 1 -- 172.21.15.59:0/4086009495 shutdown_connections 2024-04-02T17:29:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.020+0000 7f2985e18740 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6e2a30 0x55891a6e4ec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.020+0000 7f2985e18740 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dd8c0 0x55891a6e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.020+0000 7f2985e18740 1 --2- 172.21.15.59:0/4086009495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.020+0000 7f2985e18740 1 -- 172.21.15.59:0/4086009495 >> 172.21.15.59:0/4086009495 conn(0x55891a6d8440 msgr2=0x55891a6d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.023+0000 7f2985e18740 1 -- 172.21.15.59:0/4086009495 shutdown_connections 2024-04-02T17:29:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.023+0000 7f297b400640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 0x7f29600a61b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55891a6ddcc0 tx=0x7f2968006e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.023+0000 7f2985e18740 1 -- 172.21.15.59:0/4086009495 wait complete. 2024-04-02T17:29:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.025+0000 7f2985e18740 1 -- start start 2024-04-02T17:29:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.025+0000 7f2985e18740 1 Processor -- start 2024-04-02T17:29:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6e2a30 0x55891a6eea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55891a6e59e0 con 0x55891a6dd8c0 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55891a5fe660 con 0x55891a6dced0 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55891a6e5b60 con 0x55891a6e2a30 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f297b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f297be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f2985e18740 10 client.0 _ll_register_callbacks cb 0x55891a5fe050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f297be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40034/0 (socket says 172.21.15.59:40034) 2024-04-02T17:29:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.027+0000 7f297be00640 1 -- 172.21.15.59:0/1920228653 learned_addr learned my addr 172.21.15.59:0/1920228653 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f2981000640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6e2a30 0x55891a6eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297be00640 1 -- 172.21.15.59:0/1920228653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6e2a30 msgr2=0x55891a6eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297be00640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6e2a30 0x55891a6eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297be00640 1 -- 172.21.15.59:0/1920228653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 msgr2=0x55891a6e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297be00640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297be00640 1 -- 172.21.15.59:0/1920228653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891a60de80 con 0x55891a6dced0 2024-04-02T17:29:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.028+0000 7f297b400640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.029+0000 7f297be00640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b065894a729d912a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.029+0000 7f2966000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:29:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.029+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c005d50 con 0x55891a6dced0 2024-04-02T17:29:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.030+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f296c0046c0 con 0x55891a6dced0 2024-04-02T17:29:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.030+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f297409af40 con 0x55891a6dced0 2024-04-02T17:29:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.030+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f296c0048c0 con 0x55891a6dced0 2024-04-02T17:29:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.030+0000 7f2985e18740 10 client.27206 fetch_fsmap learned FSMap version 51 2024-04-02T17:29:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.030+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55891a357210 con 0x55891a6dced0 2024-04-02T17:29:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.031+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297409af40 con 0x55891a6dced0 2024-04-02T17:29:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f296c005000 con 0x55891a6dced0 2024-04-02T17:29:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 10 client.27206 fetch_fsmap finished waiting for FSMap version 51 2024-04-02T17:29:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 20 client.27206 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a6f31d0 0x55891a6f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 4 client.27206 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.032+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891a5f2cd0 con 0x55891a6f31d0 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.033+0000 7f297b400640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a6f31d0 0x55891a6f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.034+0000 7f2966000640 10 client.27206 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.034+0000 7f297b400640 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a6f31d0 0x55891a6f5600 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434706d84b09ce0b server_cookie=65e7d337961a2551 in_seq=0 out_seq=0 2024-04-02T17:29:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.035+0000 7f2966000640 1 -- 172.21.15.59:0/1920228653 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55891a5f2cd0 con 0x55891a6f31d0 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.035+0000 7f2966000640 10 client.27206 handle_command_reply: tid=0 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.035+0000 7f2985e18740 1 client.27206 shutdown 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 msgr2=0x55891a6e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c004270 comp rx=0 tx=0).stop 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 shutdown_connections 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a6f31d0 0x55891a6f5600 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6e2a30 0x55891a6eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 --2- 172.21.15.59:0/1920228653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6dced0 0x55891a6e74b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 >> 172.21.15.59:0/1920228653 conn(0x55891a6d8440 msgr2=0x55891a6d7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.036+0000 7f2966000640 0 client.27206 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.037+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 shutdown_connections 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.037+0000 7f2985e18740 1 -- 172.21.15.59:0/1920228653 wait complete. 2024-04-02T17:29:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.037+0000 7f2985e18740 20 client.27206 trim_cache size 0 max 16384 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 Processor -- start 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 -- start start 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 0x55891a6d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b7fa0 con 0x55891a6dced0 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b9ae0 con 0x55891a6e2a30 2024-04-02T17:29:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.066+0000 7f2985e18740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b8670 con 0x55891a6dd8c0 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 0x55891a6d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45520/0 (socket says 172.21.15.59:45520) 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 -- 172.21.15.59:0/1560235174 learned_addr learned my addr 172.21.15.59:0/1560235174 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f2981000640 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 -- 172.21.15.59:0/1560235174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 msgr2=0x55891a6ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 -- 172.21.15.59:0/1560235174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 msgr2=0x55891a6d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 0x55891a6d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297b400640 1 -- 172.21.15.59:0/1560235174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891a423620 con 0x55891a6dd8c0 2024-04-02T17:29:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.067+0000 7f297be00640 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 0x55891a6d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.068+0000 7f297b400640 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2968006ed0 tx=0x7f296800d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2a222169a55fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.068+0000 7f2970c00640 1 -- 172.21.15.59:0/1560235174 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968026170 con 0x55891a6dd8c0 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.068+0000 7f2970c00640 1 -- 172.21.15.59:0/1560235174 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296802c070 con 0x55891a6dd8c0 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.068+0000 7f2970c00640 1 -- 172.21.15.59:0/1560235174 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968023aa0 con 0x55891a6dd8c0 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 -- 172.21.15.59:0/1560235174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 msgr2=0x55891a6d7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2968006ed0 tx=0x7f296800d000 comp rx=0 tx=0).stop 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 -- 172.21.15.59:0/1560235174 shutdown_connections 2024-04-02T17:29:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dd8c0 0x55891a6d7a00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.069+0000 7f2985e18740 1 --2- 172.21.15.59:0/1560235174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dced0 0x55891a6d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.070+0000 7f2985e18740 1 -- 172.21.15.59:0/1560235174 >> 172.21.15.59:0/1560235174 conn(0x55891a6d8440 msgr2=0x55891a6d6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.070+0000 7f2985e18740 1 -- 172.21.15.59:0/1560235174 shutdown_connections 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.070+0000 7f2985e18740 1 -- 172.21.15.59:0/1560235174 wait complete. 2024-04-02T17:29:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.071+0000 7f2985e18740 1 -- start start 2024-04-02T17:29:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.071+0000 7f2985e18740 1 Processor -- start 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.073+0000 7f2985e18740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f297be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ed0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b9410 con 0x55891a6dd8c0 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b8d40 con 0x55891a6e2a30 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b5f80 con 0x55891a6dced0 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2985e18740 10 client.0 _ll_register_callbacks cb 0x7f296c02cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f297be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:45522/0 (socket says 172.21.15.59:45522) 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f297be00640 1 -- 172.21.15.59:0/2950141298 learned_addr learned my addr 172.21.15.59:0/2950141298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f2981000640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ed0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.074+0000 7f297b400640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 -- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 msgr2=0x55891a6ed0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ed0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 -- 172.21.15.59:0/2950141298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 msgr2=0x55891a6ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 -- 172.21.15.59:0/2950141298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891a6ed6c0 con 0x55891a6dced0 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297b400640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f297be00640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f296c005370 tx=0x7f296c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1f33300e371bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f2966000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:29:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.075+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c005d50 con 0x55891a6dced0 2024-04-02T17:29:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.076+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296802c070 con 0x55891a6dced0 2024-04-02T17:29:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.076+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8335+0+0 (secure 0 0 0) 0x7f296c0039e0 con 0x55891a6dced0 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.076+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f296c003be0 con 0x55891a6dced0 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.076+0000 7f2985e18740 10 client.27238 fetch_fsmap learned FSMap version 51 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.076+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55891a6eea70 con 0x55891a6dced0 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296802c070 con 0x55891a6dced0 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f296c034440 con 0x55891a6dced0 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 10 client.27238 fetch_fsmap finished waiting for FSMap version 51 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 10 client.27238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 20 client.27238 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a759d80 0x55891a75c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 4 client.27238 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:29:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.077+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55891a6502b0 con 0x55891a759d80 2024-04-02T17:29:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.078+0000 7f297b400640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a759d80 0x55891a75c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.079+0000 7f2966000640 10 client.27238 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.079+0000 7f297b400640 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a759d80 0x55891a75c1a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107a5b56601bfd0f server_cookie=41c7682c0df4f17 in_seq=0 out_seq=0 2024-04-02T17:29:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.262+0000 7f2966000640 1 -- 172.21.15.59:0/2950141298 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55891a6502b0 con 0x55891a759d80 2024-04-02T17:29:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.262+0000 7f2966000640 10 client.27238 handle_command_reply: tid=0 2024-04-02T17:29:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.262+0000 7f2985e18740 1 client.27238 shutdown 2024-04-02T17:29:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 msgr2=0x55891a6ebbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f296c005370 tx=0x7f296c005740 comp rx=0 tx=0).stop 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 shutdown_connections 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55891a759d80 0x55891a75c1a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55891a6dd8c0 0x55891a6ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55891a6e2a30 0x55891a6ed0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 --2- 172.21.15.59:0/2950141298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55891a6dced0 0x55891a6ebbd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2966000640 0 client.27238 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.263+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 >> 172.21.15.59:0/2950141298 conn(0x55891a6d8440 msgr2=0x55891a6d7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.264+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 shutdown_connections 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.264+0000 7f2985e18740 1 -- 172.21.15.59:0/2950141298 wait complete. 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.264+0000 7f2985e18740 20 client.27238 trim_cache size 0 max 16384 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:29:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:29:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 msgr2=0x7f29600a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 0x7f29600a61b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55891a6ddcc0 tx=0x7f2968006e60 comp rx=0 tx=0).stop 2024-04-02T17:29:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 msgr2=0x7f297c1ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 0x7f297c1ae5b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2974009fd0 tx=0x7f297400ef10 comp rx=0 tx=0).stop 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 shutdown_connections 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f29600a3cf0 0x7f29600a61b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f297c102d50 0x7f297c1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f297c103710 0x7f297c1ae5b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 --2- 172.21.15.59:0/553399920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f297c101b60 0x7f297c195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.266+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 >> 172.21.15.59:0/553399920 conn(0x7f297c0fd0d0 msgr2=0x7f297c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.267+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 shutdown_connections 2024-04-02T17:29:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:26.267+0000 7f2983000640 1 -- 172.21.15.59:0/553399920 wait complete. 2024-04-02T17:29:26.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 141018 42.91 MB/sec execute 53 sec latency 260.801 ms 2024-04-02T17:29:27.174 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:29:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setuser ceph since I am not root 2024-04-02T17:29:27.175 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:29:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setgroup ceph since I am not root 2024-04-02T17:29:27.175 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:29:26 smithi164 ceph-mds[66927]: -- 172.21.15.164:0/1619642654 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576ecd2da40 con 0x5576ecd00800 2024-04-02T17:29:27.175 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:29:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: starting mds.f at 2024-04-02T17:29:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:27 smithi059 ceph-mon[31664]: pgmap v879: 97 pgs: 97 active+clean; 681 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 653 op/s 2024-04-02T17:29:27.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:27 smithi059 ceph-mon[31664]: daemon mds.f restarted 2024-04-02T17:29:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:27 smithi164 ceph-mon[36499]: pgmap v879: 97 pgs: 97 active+clean; 681 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 653 op/s 2024-04-02T17:29:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:27 smithi164 ceph-mon[36499]: daemon mds.f restarted 2024-04-02T17:29:27.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 141829 43.19 MB/sec execute 54 sec latency 168.912 ms 2024-04-02T17:29:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:27 smithi154 ceph-mon[36479]: pgmap v879: 97 pgs: 97 active+clean; 681 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 653 op/s 2024-04-02T17:29:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:27 smithi154 ceph-mon[36479]: daemon mds.f restarted 2024-04-02T17:29:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:28 smithi059 ceph-mon[31664]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T17:29:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:28 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6834/1847771172,v1:172.21.15.164:6835/1847771172] up:boot 2024-04-02T17:29:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:28 smithi059 ceph-mon[31664]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:29:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:28 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:29:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:28 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:28 smithi164 ceph-mon[36499]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T17:29:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:28 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6834/1847771172,v1:172.21.15.164:6835/1847771172] up:boot 2024-04-02T17:29:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:28 smithi164 ceph-mon[36499]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:29:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:28 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:29:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:28 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:28.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 142344 42.87 MB/sec execute 55 sec latency 57.710 ms 2024-04-02T17:29:28.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:28 smithi154 ceph-mon[36479]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T17:29:28.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:28 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6834/1847771172,v1:172.21.15.164:6835/1847771172] up:boot 2024-04-02T17:29:28.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:28 smithi154 ceph-mon[36479]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:29:28.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:28 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:29:28.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:28 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:29:29.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:29 smithi059 ceph-mon[31664]: pgmap v881: 97 pgs: 97 active+clean; 688 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-04-02T17:29:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:29 smithi164 ceph-mon[36499]: pgmap v881: 97 pgs: 97 active+clean; 688 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-04-02T17:29:29.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 143081 42.68 MB/sec execute 56 sec latency 92.955 ms 2024-04-02T17:29:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:29 smithi154 ceph-mon[36479]: pgmap v881: 97 pgs: 97 active+clean; 688 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 632 op/s 2024-04-02T17:29:30.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 144162 42.79 MB/sec execute 57 sec latency 38.929 ms 2024-04-02T17:29:31.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:31 smithi059 ceph-mon[31664]: pgmap v882: 97 pgs: 97 active+clean; 694 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-04-02T17:29:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:31 smithi164 ceph-mon[36499]: pgmap v882: 97 pgs: 97 active+clean; 694 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-04-02T17:29:31.604 INFO:tasks.workunit.client.0.smithi059.stdout: 10 145250 43.36 MB/sec execute 58 sec latency 35.739 ms 2024-04-02T17:29:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:31 smithi154 ceph-mon[36479]: pgmap v882: 97 pgs: 97 active+clean; 694 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 683 op/s 2024-04-02T17:29:32.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:32 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:32 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:32.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 145781 43.10 MB/sec execute 59 sec latency 160.072 ms 2024-04-02T17:29:32.750 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:32 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:33.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:33 smithi059 ceph-mon[31664]: pgmap v883: 97 pgs: 97 active+clean; 696 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-04-02T17:29:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:33 smithi164 ceph-mon[36499]: pgmap v883: 97 pgs: 97 active+clean; 696 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-04-02T17:29:33.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 145800 42.40 MB/sec execute 60 sec latency 893.879 ms 2024-04-02T17:29:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:33 smithi154 ceph-mon[36479]: pgmap v883: 97 pgs: 97 active+clean; 696 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 777 op/s 2024-04-02T17:29:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:29:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:34 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:29:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:29:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:29:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:34.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:34 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:34.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 146062 41.87 MB/sec execute 61 sec latency 751.971 ms 2024-04-02T17:29:34.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:29:34.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:34.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:34.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:34 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:35.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 146450 41.45 MB/sec execute 62 sec latency 149.328 ms 2024-04-02T17:29:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:35 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:35 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:36.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:29:36.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:29:36.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:29:36.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:35 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:29:36.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 147179 41.45 MB/sec execute 63 sec latency 216.760 ms 2024-04-02T17:29:37.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 148057 41.34 MB/sec execute 64 sec latency 384.405 ms 2024-04-02T17:29:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:38 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:29:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:38 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:29:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:38 smithi154 ceph-mon[36479]: pgmap v884: 97 pgs: 97 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 755 op/s 2024-04-02T17:29:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:38 smithi154 ceph-mon[36479]: pgmap v885: 97 pgs: 97 active+clean; 769 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-02T17:29:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:38 smithi154 ceph-mon[36479]: pgmap v886: 97 pgs: 97 active+clean; 754 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-02T17:29:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:38 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:29:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:38 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:29:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:38 smithi059 ceph-mon[31664]: pgmap v884: 97 pgs: 97 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 755 op/s 2024-04-02T17:29:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:38 smithi059 ceph-mon[31664]: pgmap v885: 97 pgs: 97 active+clean; 769 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-02T17:29:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:38 smithi059 ceph-mon[31664]: pgmap v886: 97 pgs: 97 active+clean; 754 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-02T17:29:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:38 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi059 to 177.8M 2024-04-02T17:29:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:38 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi059 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-04-02T17:29:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:38 smithi164 ceph-mon[36499]: pgmap v884: 97 pgs: 97 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 755 op/s 2024-04-02T17:29:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:38 smithi164 ceph-mon[36499]: pgmap v885: 97 pgs: 97 active+clean; 769 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-02T17:29:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:38 smithi164 ceph-mon[36499]: pgmap v886: 97 pgs: 97 active+clean; 754 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-02T17:29:38.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 149053 41.81 MB/sec execute 65 sec latency 119.829 ms 2024-04-02T17:29:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 -- 172.21.15.59:0/4085446560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668108b00 msgr2=0x7f9668108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 --2- 172.21.15.59:0/4085446560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668108b00 0x7f9668108f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802bc40 comp rx=0 tx=0).stop 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 -- 172.21.15.59:0/4085446560 shutdown_connections 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 --2- 172.21.15.59:0/4085446560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668109460 0x7f966810fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 --2- 172.21.15.59:0/4085446560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668110380 0x7f9668112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 --2- 172.21.15.59:0/4085446560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668108b00 0x7f9668108f20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 -- 172.21.15.59:0/4085446560 >> 172.21.15.59:0/4085446560 conn(0x7f966806eb40 msgr2=0x7f9668070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 -- 172.21.15.59:0/4085446560 shutdown_connections 2024-04-02T17:29:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.452+0000 7f966e200640 1 -- 172.21.15.59:0/4085446560 wait complete. 2024-04-02T17:29:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.453+0000 7f966e200640 1 Processor -- start 2024-04-02T17:29:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.453+0000 7f966e200640 1 -- start start 2024-04-02T17:29:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.454+0000 7f966e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.454+0000 7f966e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668109460 0x7f966819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.454+0000 7f966e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 0x7f96681b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.454+0000 7f966e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9668113de0 con 0x7f9668110380 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.454+0000 7f966e200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9668113c60 con 0x7f9668108b00 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f966e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9668113f60 con 0x7f9668109460 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9666a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668109460 0x7f966819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39054/0 (socket says 172.21.15.59:39054) 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 -- 172.21.15.59:0/2652531204 learned_addr learned my addr 172.21.15.59:0/2652531204 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667e00640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 0x7f96681b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 -- 172.21.15.59:0/2652531204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668109460 msgr2=0x7f966819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668109460 0x7f966819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 -- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 msgr2=0x7f96681b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 0x7f96681b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667400640 1 -- 172.21.15.59:0/2652531204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96681a19c0 con 0x7f9668108b00 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.455+0000 7f9667e00640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 0x7f96681b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.456+0000 7f9667400640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f965802c250 tx=0x7f9658004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.456+0000 7f965fe00640 1 -- 172.21.15.59:0/2652531204 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658041070 con 0x7f9668108b00 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.456+0000 7f965fe00640 1 -- 172.21.15.59:0/2652531204 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96580054e0 con 0x7f9668108b00 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.456+0000 7f965fe00640 1 -- 172.21.15.59:0/2652531204 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658034440 con 0x7f9668108b00 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.456+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96681b7200 con 0x7f9668108b00 2024-04-02T17:29:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.457+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681b7670 con 0x7f9668108b00 2024-04-02T17:29:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.461+0000 7f9670f6c740 1 Processor -- start 2024-04-02T17:29:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.462+0000 7f9670f6c740 1 -- start start 2024-04-02T17:29:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.462+0000 7f965fe00640 1 -- 172.21.15.59:0/2652531204 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9658044070 con 0x7f9668108b00 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.462+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.462+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9670f6c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c6e010d70 con 0x557c6e0b0a30 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9670f6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c6dfb92a0 con 0x557c6e0aaed0 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9670f6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c6e00cd20 con 0x557c6e0ab8c0 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9667e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9667400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9667e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39064/0 (socket says 172.21.15.59:39064) 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9667e00640 1 -- 172.21.15.59:0/3479451501 learned_addr learned my addr 172.21.15.59:0/3479451501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.463+0000 7f9666a00640 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9667e00640 1 -- 172.21.15.59:0/3479451501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0ab8c0 msgr2=0x557c6e0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9667e00640 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9667e00640 1 -- 172.21.15.59:0/3479451501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 msgr2=0x557c6e0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9667e00640 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9667e00640 1 -- 172.21.15.59:0/3479451501 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6dfcd840 con 0x557c6e0aaed0 2024-04-02T17:29:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.464+0000 7f9666a00640 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.465+0000 7f9667e00640 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f966000c400 tx=0x7f966000d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae20e7c3ac1ec8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.465+0000 7f965fe00640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 0x7f964c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.465+0000 7f9653e00640 1 -- 172.21.15.59:0/3479451501 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660003970 con 0x557c6e0aaed0 2024-04-02T17:29:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.465+0000 7f9653e00640 1 -- 172.21.15.59:0/3479451501 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660004020 con 0x557c6e0aaed0 2024-04-02T17:29:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.465+0000 7f9653e00640 1 -- 172.21.15.59:0/3479451501 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966000a9a0 con 0x557c6e0aaed0 2024-04-02T17:29:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.466+0000 7f9667400640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 0x7f964c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.466+0000 7f965fe00640 1 -- 172.21.15.59:0/2652531204 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f96580f2040 con 0x7f9668108b00 2024-04-02T17:29:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 -- 172.21.15.59:0/3479451501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 msgr2=0x557c6e0ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f966000c400 tx=0x7f966000d910 comp rx=0 tx=0).stop 2024-04-02T17:29:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9667400640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 0x7f964c0a61e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96580efe50 tx=0x7f96580f5b20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 -- 172.21.15.59:0/3479451501 shutdown_connections 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 --2- 172.21.15.59:0/3479451501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0aaed0 0x557c6e0ab2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 -- 172.21.15.59:0/3479451501 >> 172.21.15.59:0/3479451501 conn(0x557c6e0a6440 msgr2=0x557c6e0a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.467+0000 7f9670f6c740 1 -- 172.21.15.59:0/3479451501 shutdown_connections 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.468+0000 7f9670f6c740 1 -- 172.21.15.59:0/3479451501 wait complete. 2024-04-02T17:29:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.468+0000 7f9670f6c740 1 -- start start 2024-04-02T17:29:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.469+0000 7f9670f6c740 1 Processor -- start 2024-04-02T17:29:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.469+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.469+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9670f6c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c6e0b39e0 con 0x557c6e0b0a30 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9670f6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c6dfcc660 con 0x557c6e0ab8c0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9670f6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557c6e0b3b60 con 0x557c6e0aaed0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56550/0 (socket says 172.21.15.59:56550) 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 -- 172.21.15.59:0/2975149242 learned_addr learned my addr 172.21.15.59:0/2975149242 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9666a00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9670f6c740 10 client.0 _ll_register_callbacks cb 0x557c6dfcc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 -- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 msgr2=0x557c6e0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 -- 172.21.15.59:0/2975149242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 msgr2=0x557c6e0bca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9667e00640 1 -- 172.21.15.59:0/2975149242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6dfdbe80 con 0x557c6e0aaed0 2024-04-02T17:29:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.470+0000 7f9666a00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.471+0000 7f9667e00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f966000cd50 tx=0x7f9660000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e27cb3794bb4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.471+0000 7f9650c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.471+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660027070 con 0x557c6e0aaed0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.471+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f96600041d0 con 0x557c6e0aaed0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.472+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9660004020 con 0x557c6e0aaed0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.473+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9660031050 con 0x557c6e0aaed0 2024-04-02T17:29:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.473+0000 7f9670f6c740 10 client.27256 fetch_fsmap learned FSMap version 53 2024-04-02T17:29:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.473+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557c6dd25210 con 0x557c6e0aaed0 2024-04-02T17:29:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.473+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660004020 con 0x557c6e0aaed0 2024-04-02T17:29:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.473+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f966002c440 con 0x557c6e0aaed0 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 10 client.27256 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 10 client.27256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 20 client.27256 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e0c1010 0x557c6e0c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 4 client.27256 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.474+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6dfc0cd0 con 0x557c6e0c1010 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.475+0000 7f9666a00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e0c1010 0x557c6e0c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.475+0000 7f9666a00640 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e0c1010 0x557c6e0c3420 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780f4ea409ac6acd server_cookie=7680b8289f73b701 in_seq=0 out_seq=0 2024-04-02T17:29:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.475+0000 7f9650c00640 10 client.27256 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9650c00640 1 -- 172.21.15.59:0/2975149242 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c6dfc0cd0 con 0x557c6e0c1010 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9650c00640 10 client.27256 handle_command_reply: tid=0 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 client.27256 shutdown 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 msgr2=0x557c6e0b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f966000cd50 tx=0x7f9660000f40 comp rx=0 tx=0).stop 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 shutdown_connections 2024-04-02T17:29:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e0c1010 0x557c6e0c3420 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2975149242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5480 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 >> 172.21.15.59:0/2975149242 conn(0x557c6e0a6440 msgr2=0x557c6e0a5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.480+0000 7f9650c00640 0 client.27256 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.481+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 shutdown_connections 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.481+0000 7f9670f6c740 1 -- 172.21.15.59:0/2975149242 wait complete. 2024-04-02T17:29:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.481+0000 7f9670f6c740 20 client.27256 trim_cache size 0 max 16384 2024-04-02T17:29:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.502+0000 7f9670f6c740 1 Processor -- start 2024-04-02T17:29:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 -- start start 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 0x557c6e0bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0b0a30 0x557c6e0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96681c0720 con 0x557c6e0ab8c0 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96681c2260 con 0x557c6e0b0a30 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9670f6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96681c0df0 con 0x557c6e0aaed0 2024-04-02T17:29:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56558/0 (socket says 172.21.15.59:56558) 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9667e00640 1 -- 172.21.15.59:0/2198305271 learned_addr learned my addr 172.21.15.59:0/2198305271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.503+0000 7f9667e00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0b0a30 0x557c6e0a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9666a00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 0x557c6e0bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9667e00640 1 -- 172.21.15.59:0/2198305271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0b0a30 msgr2=0x557c6e0a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9667e00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0b0a30 0x557c6e0a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9667e00640 1 -- 172.21.15.59:0/2198305271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 msgr2=0x557c6e0bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9667e00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 0x557c6e0bc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9667e00640 1 -- 172.21.15.59:0/2198305271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6ddf1620 con 0x557c6e0aaed0 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.504+0000 7f9666a00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 0x557c6e0bc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.505+0000 7f9667e00640 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9660005d70 tx=0x7f96600021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2890a351bedcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.505+0000 7f9653400640 1 -- 172.21.15.59:0/2198305271 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660027070 con 0x557c6e0aaed0 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.506+0000 7f9653400640 1 -- 172.21.15.59:0/2198305271 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f966002ca80 con 0x557c6e0aaed0 2024-04-02T17:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.506+0000 7f9653400640 1 -- 172.21.15.59:0/2198305271 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966000a7a0 con 0x557c6e0aaed0 2024-04-02T17:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 -- 172.21.15.59:0/2198305271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 msgr2=0x557c6e0b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9660005d70 tx=0x7f96600021d0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 -- 172.21.15.59:0/2198305271 shutdown_connections 2024-04-02T17:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0ab8c0 0x557c6e0bc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0b0a30 0x557c6e0a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 --2- 172.21.15.59:0/2198305271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b5920 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.507+0000 7f9670f6c740 1 -- 172.21.15.59:0/2198305271 >> 172.21.15.59:0/2198305271 conn(0x557c6e0a6440 msgr2=0x557c6e0a6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.508+0000 7f9670f6c740 1 -- 172.21.15.59:0/2198305271 shutdown_connections 2024-04-02T17:29:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.508+0000 7f9670f6c740 1 -- 172.21.15.59:0/2198305271 wait complete. 2024-04-02T17:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.508+0000 7f9670f6c740 1 -- start start 2024-04-02T17:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.509+0000 7f9670f6c740 1 Processor -- start 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96681c1b90 con 0x557c6e0b0a30 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96681c14c0 con 0x557c6e0ab8c0 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96681be700 con 0x557c6e0aaed0 2024-04-02T17:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9670f6c740 10 client.0 _ll_register_callbacks cb 0x7f9660003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9666a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9667e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9667e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9667e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32896/0 (socket says 172.21.15.59:32896) 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9666a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39092/0 (socket says 172.21.15.59:39092) 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.510+0000 7f9667e00640 1 -- 172.21.15.59:0/93121120 learned_addr learned my addr 172.21.15.59:0/93121120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9666a00640 1 -- 172.21.15.59:0/93121120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 msgr2=0x557c6e0b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9666a00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9666a00640 1 -- 172.21.15.59:0/93121120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 msgr2=0x557c6e0bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9666a00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9666a00640 1 -- 172.21.15.59:0/93121120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6e0bb850 con 0x557c6e0ab8c0 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.511+0000 7f9667e00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.512+0000 7f9666a00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x557c6e0c3cd0 tx=0x7f965400cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7868f3a68342672 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.512+0000 7f9650c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.512+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654004020 con 0x557c6e0ab8c0 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.512+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f96540041c0 con 0x557c6e0ab8c0 2024-04-02T17:29:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.512+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f966002ca80 con 0x557c6e0ab8c0 2024-04-02T17:29:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.513+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9654028050 con 0x557c6e0ab8c0 2024-04-02T17:29:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.513+0000 7f9670f6c740 10 client.27230 fetch_fsmap learned FSMap version 53 2024-04-02T17:29:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.513+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557c6e0bcbd0 con 0x557c6e0ab8c0 2024-04-02T17:29:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.514+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f966002ca80 con 0x557c6e0ab8c0 2024-04-02T17:29:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.514+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f9654003990 con 0x557c6e0ab8c0 2024-04-02T17:29:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.514+0000 7f9670f6c740 10 client.27230 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.514+0000 7f9670f6c740 10 client.27230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.514+0000 7f9670f6c740 20 client.27230 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.515+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e127cd0 0x557c6e12a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.515+0000 7f9670f6c740 4 client.27230 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.515+0000 7f9667e00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e127cd0 0x557c6e12a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.515+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c6e01e2b0 con 0x557c6e127cd0 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.516+0000 7f9667e00640 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e127cd0 0x557c6e12a0f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7ca3ec63ebc383 server_cookie=cf20969a8c7ffd1c in_seq=0 out_seq=0 2024-04-02T17:29:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.516+0000 7f9650c00640 10 client.27230 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:39.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 149675 41.69 MB/sec execute 66 sec latency 61.600 ms 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9650c00640 1 -- 172.21.15.59:0/93121120 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c6e01e2b0 con 0x557c6e127cd0 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9650c00640 10 client.27230 handle_command_reply: tid=0 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 client.27230 shutdown 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 msgr2=0x557c6e0ba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x557c6e0c3cd0 tx=0x7f965400cf50 comp rx=0 tx=0).stop 2024-04-02T17:29:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 shutdown_connections 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557c6e127cd0 0x557c6e12a0f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c6e0b0a30 0x557c6e0bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c6e0ab8c0 0x557c6e0ba2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9650c00640 0 client.27230 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 --2- 172.21.15.59:0/93121120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c6e0aaed0 0x557c6e0b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.736+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 >> 172.21.15.59:0/93121120 conn(0x557c6e0a6440 msgr2=0x557c6e0a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.737+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 shutdown_connections 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.737+0000 7f9670f6c740 1 -- 172.21.15.59:0/93121120 wait complete. 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.737+0000 7f9670f6c740 20 client.27230 trim_cache size 0 max 16384 2024-04-02T17:29:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:29:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:29:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:29:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:29:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 msgr2=0x7f964c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 0x7f964c0a61e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f96580efe50 tx=0x7f96580f5b20 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 msgr2=0x7f966819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f965802c250 tx=0x7f9658004e30 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 shutdown_connections 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f964c0a3d20 0x7f964c0a61e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9668110380 0x7f96681b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9668109460 0x7f966819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 --2- 172.21.15.59:0/2652531204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9668108b00 0x7f966819e300 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f965802c250 tx=0x7f9658004e30 comp rx=0 tx=0).stop 2024-04-02T17:29:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.739+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 >> 172.21.15.59:0/2652531204 conn(0x7f966806eb40 msgr2=0x7f966810be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.741+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 shutdown_connections 2024-04-02T17:29:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:39.741+0000 7f966e200640 1 -- 172.21.15.59:0/2652531204 wait complete. 2024-04-02T17:29:40.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:40 smithi154 ceph-mon[36479]: pgmap v887: 97 pgs: 97 active+clean; 711 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 996 op/s 2024-04-02T17:29:40.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:40 smithi059 ceph-mon[31664]: pgmap v887: 97 pgs: 97 active+clean; 711 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 996 op/s 2024-04-02T17:29:40.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:40 smithi164 ceph-mon[36499]: pgmap v887: 97 pgs: 97 active+clean; 711 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 996 op/s 2024-04-02T17:29:40.605 INFO:tasks.workunit.client.0.smithi059.stdout: 10 150252 41.44 MB/sec execute 67 sec latency 112.589 ms 2024-04-02T17:29:41.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 151120 41.45 MB/sec execute 68 sec latency 48.253 ms 2024-04-02T17:29:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:42 smithi154 ceph-mon[36479]: pgmap v888: 97 pgs: 97 active+clean; 695 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:29:42.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:42 smithi059 ceph-mon[31664]: pgmap v888: 97 pgs: 97 active+clean; 695 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:29:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:42 smithi164 ceph-mon[36499]: pgmap v888: 97 pgs: 97 active+clean; 695 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:29:42.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 152101 41.70 MB/sec execute 69 sec latency 42.959 ms 2024-04-02T17:29:43.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 152955 41.82 MB/sec execute 70 sec latency 34.514 ms 2024-04-02T17:29:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:44 smithi154 ceph-mon[36479]: pgmap v889: 97 pgs: 97 active+clean; 701 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-04-02T17:29:44.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:44 smithi059 ceph-mon[31664]: pgmap v889: 97 pgs: 97 active+clean; 701 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-04-02T17:29:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:44 smithi164 ceph-mon[36499]: pgmap v889: 97 pgs: 97 active+clean; 701 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 987 op/s 2024-04-02T17:29:44.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 153663 41.72 MB/sec execute 71 sec latency 33.640 ms 2024-04-02T17:29:45.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 154507 41.70 MB/sec execute 72 sec latency 45.556 ms 2024-04-02T17:29:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:46 smithi154 ceph-mon[36479]: pgmap v890: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 961 op/s 2024-04-02T17:29:46.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:46 smithi059 ceph-mon[31664]: pgmap v890: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 961 op/s 2024-04-02T17:29:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:46 smithi164 ceph-mon[36499]: pgmap v890: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 961 op/s 2024-04-02T17:29:46.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 155463 41.84 MB/sec execute 73 sec latency 46.020 ms 2024-04-02T17:29:47.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 156426 42.04 MB/sec execute 74 sec latency 36.859 ms 2024-04-02T17:29:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:48 smithi154 ceph-mon[36479]: pgmap v891: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-04-02T17:29:48.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:48 smithi059 ceph-mon[31664]: pgmap v891: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-04-02T17:29:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:48 smithi164 ceph-mon[36499]: pgmap v891: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-04-02T17:29:48.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 157167 42.02 MB/sec execute 75 sec latency 44.347 ms 2024-04-02T17:29:49.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 158043 42.06 MB/sec execute 76 sec latency 51.822 ms 2024-04-02T17:29:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:50 smithi154 ceph-mon[36479]: pgmap v892: 97 pgs: 97 active+clean; 726 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 764 op/s 2024-04-02T17:29:50.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:50 smithi059 ceph-mon[31664]: pgmap v892: 97 pgs: 97 active+clean; 726 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 764 op/s 2024-04-02T17:29:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:50 smithi164 ceph-mon[36499]: pgmap v892: 97 pgs: 97 active+clean; 726 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 764 op/s 2024-04-02T17:29:50.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 158907 42.06 MB/sec execute 77 sec latency 58.726 ms 2024-04-02T17:29:51.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 159828 42.25 MB/sec execute 78 sec latency 34.977 ms 2024-04-02T17:29:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:52 smithi154 ceph-mon[36479]: pgmap v893: 97 pgs: 97 active+clean; 732 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-04-02T17:29:52.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:52 smithi059 ceph-mon[31664]: pgmap v893: 97 pgs: 97 active+clean; 732 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-04-02T17:29:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:52 smithi164 ceph-mon[36499]: pgmap v893: 97 pgs: 97 active+clean; 732 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-04-02T17:29:52.606 INFO:tasks.workunit.client.0.smithi059.stdout: 10 160644 42.22 MB/sec execute 79 sec latency 35.000 ms 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 -- 172.21.15.59:0/1553664159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a40744e0 msgr2=0x7fd0a4074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 --2- 172.21.15.59:0/1553664159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a40744e0 0x7fd0a4074940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd090009860 tx=0x7fd09002bc10 comp rx=0 tx=0).stop 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 -- 172.21.15.59:0/1553664159 shutdown_connections 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 --2- 172.21.15.59:0/1553664159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a40744e0 0x7fd0a4074940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 --2- 172.21.15.59:0/1553664159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a41099e0 0x7fd0a410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.947+0000 7fd0ac400640 1 --2- 172.21.15.59:0/1553664159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a4075ac0 0x7fd0a4073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.948+0000 7fd0ac400640 1 -- 172.21.15.59:0/1553664159 >> 172.21.15.59:0/1553664159 conn(0x7fd0a40fd0c0 msgr2=0x7fd0a40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.948+0000 7fd0ac400640 1 -- 172.21.15.59:0/1553664159 shutdown_connections 2024-04-02T17:29:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.948+0000 7fd0ac400640 1 -- 172.21.15.59:0/1553664159 wait complete. 2024-04-02T17:29:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.949+0000 7fd0ac400640 1 Processor -- start 2024-04-02T17:29:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.950+0000 7fd0ac400640 1 -- start start 2024-04-02T17:29:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.950+0000 7fd0ac400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.950+0000 7fd0ac400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a4075ac0 0x7fd0a419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.950+0000 7fd0ac400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a41099e0 0x7fd0a41a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0ac400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410f990 con 0x7fd0a4075ac0 2024-04-02T17:29:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0ac400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410f810 con 0x7fd0a40744e0 2024-04-02T17:29:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0ac400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410fb10 con 0x7fd0a41099e0 2024-04-02T17:29:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33264/0 (socket says 172.21.15.59:33264) 2024-04-02T17:29:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0a9a00640 1 -- 172.21.15.59:0/2003300932 learned_addr learned my addr 172.21.15.59:0/2003300932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.951+0000 7fd0a9000640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a4075ac0 0x7fd0a419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0aa400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a41099e0 0x7fd0a41a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 -- 172.21.15.59:0/2003300932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a41099e0 msgr2=0x7fd0a41a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a41099e0 0x7fd0a41a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 -- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a4075ac0 msgr2=0x7fd0a419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a4075ac0 0x7fd0a419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 -- 172.21.15.59:0/2003300932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41a1b70 con 0x7fd0a40744e0 2024-04-02T17:29:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.952+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd09800efb0 tx=0x7fd09800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.953+0000 7fd0a2000640 1 -- 172.21.15.59:0/2003300932 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801b070 con 0x7fd0a40744e0 2024-04-02T17:29:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.953+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a1e60 con 0x7fd0a40744e0 2024-04-02T17:29:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.953+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a22d0 con 0x7fd0a40744e0 2024-04-02T17:29:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.953+0000 7fd0a2000640 1 -- 172.21.15.59:0/2003300932 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09800ba50 con 0x7fd0a40744e0 2024-04-02T17:29:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.953+0000 7fd0a2000640 1 -- 172.21.15.59:0/2003300932 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801b070 con 0x7fd0a40744e0 2024-04-02T17:29:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 Processor -- start 2024-04-02T17:29:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 -- start start 2024-04-02T17:29:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x56209415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x56209415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5620940bdd70 con 0x56209415da30 2024-04-02T17:29:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5620940662a0 con 0x5620941588c0 2024-04-02T17:29:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.958+0000 7fd0af19b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5620940b9d20 con 0x562094157ed0 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0a2000640 1 -- 172.21.15.59:0/2003300932 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd0980169b0 con 0x7fd0a40744e0 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47552/0 (socket says 172.21.15.59:47552) 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 -- 172.21.15.59:0/982156903 learned_addr learned my addr 172.21.15.59:0/982156903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0a9000640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x56209415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x56209415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 -- 172.21.15.59:0/982156903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 msgr2=0x56209415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x56209415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 -- 172.21.15.59:0/982156903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 msgr2=0x56209415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x56209415fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0aa400640 1 -- 172.21.15.59:0/982156903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209407a840 con 0x562094157ed0 2024-04-02T17:29:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.959+0000 7fd0a9000640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x56209415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.960+0000 7fd0aa400640 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009c10 tx=0x7fd09c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f675c05d7f7fd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.960+0000 7fd096a00640 1 -- 172.21.15.59:0/982156903 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c012450 con 0x562094157ed0 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.960+0000 7fd096a00640 1 -- 172.21.15.59:0/982156903 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c012ad0 con 0x562094157ed0 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.960+0000 7fd096a00640 1 -- 172.21.15.59:0/982156903 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00fbd0 con 0x562094157ed0 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 -- 172.21.15.59:0/982156903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 msgr2=0x5620941582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009c10 tx=0x7fd09c0117b0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 -- 172.21.15.59:0/982156903 shutdown_connections 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x56209415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x56209415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 --2- 172.21.15.59:0/982156903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562094157ed0 0x5620941582f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 -- 172.21.15.59:0/982156903 >> 172.21.15.59:0/982156903 conn(0x562094153440 msgr2=0x562094153840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0af19b740 1 -- 172.21.15.59:0/982156903 shutdown_connections 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.961+0000 7fd0a2000640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 0x7fd08c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.962+0000 7fd0a2000640 1 -- 172.21.15.59:0/2003300932 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fd098013070 con 0x7fd0a40744e0 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.962+0000 7fd0a9000640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 0x7fd08c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.962+0000 7fd0af19b740 1 -- 172.21.15.59:0/982156903 wait complete. 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.963+0000 7fd0af19b740 1 -- start start 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.963+0000 7fd0a9000640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 0x7fd08c0a64a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd09002c220 tx=0x7fd0900058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.963+0000 7fd0af19b740 1 Processor -- start 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x562094162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x562094169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5620941609e0 con 0x56209415da30 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562094079660 con 0x562094157ed0 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562094160b60 con 0x5620941588c0 2024-04-02T17:29:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0af19b740 10 client.0 _ll_register_callbacks cb 0x562094079050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33284/0 (socket says 172.21.15.59:33284) 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.964+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1344712073 learned_addr learned my addr 172.21.15.59:0/1344712073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x562094162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1344712073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 msgr2=0x562094162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x562094162920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1344712073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 msgr2=0x562094169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x562094169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1344712073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562094088e80 con 0x562094157ed0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.965+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd098098ac0 tx=0x7fd09809ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1cf95473f7b502 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.966+0000 7fd08b400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.966+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801b070 con 0x562094157ed0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.966+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fd09800bc00 con 0x562094157ed0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.966+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c012ad0 con 0x562094157ed0 2024-04-02T17:29:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd0980a04a0 con 0x562094157ed0 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 10 client.27242 fetch_fsmap learned FSMap version 53 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x562093dd2210 con 0x562094157ed0 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c012ad0 con 0x562094157ed0 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fd09800b040 con 0x562094157ed0 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 10 client.27242 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 10 client.27242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 20 client.27242 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.967+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941769f0 0x562094178e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.968+0000 7fd0af19b740 4 client.27242 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.968+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209406dcd0 con 0x5620941769f0 2024-04-02T17:29:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.968+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941769f0 0x562094178e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.969+0000 7fd08b400640 10 client.27242 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.969+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941769f0 0x562094178e40 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe5186df3bac2d server_cookie=90590071c4f6d690 in_seq=0 out_seq=0 2024-04-02T17:29:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd08b400640 1 -- 172.21.15.59:0/1344712073 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56209406dcd0 con 0x5620941769f0 2024-04-02T17:29:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd08b400640 10 client.27242 handle_command_reply: tid=0 2024-04-02T17:29:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 client.27242 shutdown 2024-04-02T17:29:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 msgr2=0x5620941623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd098098ac0 tx=0x7fd09809ba50 comp rx=0 tx=0).stop 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 shutdown_connections 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941769f0 0x562094178e40 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x562094169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x562094162920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1344712073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x5620941623e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 >> 172.21.15.59:0/1344712073 conn(0x562094153440 msgr2=0x562094152640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.972+0000 7fd08b400640 0 client.27242 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.973+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 shutdown_connections 2024-04-02T17:29:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.973+0000 7fd0af19b740 1 -- 172.21.15.59:0/1344712073 wait complete. 2024-04-02T17:29:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.973+0000 7fd0af19b740 20 client.27242 trim_cache size 0 max 16384 2024-04-02T17:29:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.998+0000 7fd0af19b740 1 Processor -- start 2024-04-02T17:29:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.998+0000 7fd0af19b740 1 -- start start 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x562094169920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56209415da30 0x562094162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd08400e730 con 0x562094157ed0 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd084010270 con 0x5620941588c0 2024-04-02T17:29:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0af19b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd08400ee00 con 0x56209415da30 2024-04-02T17:29:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56834/0 (socket says 172.21.15.59:56834) 2024-04-02T17:29:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1910141161 learned_addr learned my addr 172.21.15.59:0/1910141161 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:52.999+0000 7fd0aa400640 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56209415da30 0x562094162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1910141161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56209415da30 msgr2=0x562094162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56209415da30 0x562094162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1910141161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 msgr2=0x562094169920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x562094169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1910141161 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562093e9e620 con 0x562094157ed0 2024-04-02T17:29:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.000+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd09809c080 tx=0x7fd0980d3260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8528b9e7d4e9878c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.001+0000 7fd096000640 1 -- 172.21.15.59:0/1910141161 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098063070 con 0x562094157ed0 2024-04-02T17:29:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.001+0000 7fd096000640 1 -- 172.21.15.59:0/1910141161 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0980a4d60 con 0x562094157ed0 2024-04-02T17:29:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.001+0000 7fd096000640 1 -- 172.21.15.59:0/1910141161 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980a04a0 con 0x562094157ed0 2024-04-02T17:29:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 -- 172.21.15.59:0/1910141161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 msgr2=0x562094161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd09809c080 tx=0x7fd0980d3260 comp rx=0 tx=0).stop 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 -- 172.21.15.59:0/1910141161 shutdown_connections 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56209415da30 0x562094162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5620941588c0 0x562094169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1910141161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562094157ed0 0x562094161ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 -- 172.21.15.59:0/1910141161 >> 172.21.15.59:0/1910141161 conn(0x562094153440 msgr2=0x562094159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.002+0000 7fd0af19b740 1 -- 172.21.15.59:0/1910141161 shutdown_connections 2024-04-02T17:29:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.003+0000 7fd0af19b740 1 -- 172.21.15.59:0/1910141161 wait complete. 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.003+0000 7fd0af19b740 1 -- start start 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.003+0000 7fd0af19b740 1 Processor -- start 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x5620941671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x5620941681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd08400fba0 con 0x56209415da30 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd08400f4d0 con 0x562094157ed0 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd08400c6d0 con 0x5620941588c0 2024-04-02T17:29:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0af19b740 10 client.0 _ll_register_callbacks cb 0x7fd098098390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0a9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33298/0 (socket says 172.21.15.59:33298) 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.004+0000 7fd0a9a00640 1 -- 172.21.15.59:0/1754333794 learned_addr learned my addr 172.21.15.59:0/1754333794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x5620941671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0aa400640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x5620941681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 -- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 msgr2=0x562094166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 -- 172.21.15.59:0/1754333794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 msgr2=0x5620941681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x5620941681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9000640 1 -- 172.21.15.59:0/1754333794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620941687e0 con 0x5620941588c0 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.005+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.006+0000 7fd0a9000640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x5620941671e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd090034a80 tx=0x7fd090034940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e969b74891746db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.006+0000 7fd08b400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090039040 con 0x5620941588c0 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fd09002c5b0 con 0x5620941588c0 2024-04-02T17:29:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0980a4d60 con 0x5620941588c0 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd09005f050 con 0x5620941588c0 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd0af19b740 10 client.27280 fetch_fsmap learned FSMap version 53 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x562094169b60 con 0x5620941588c0 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.007+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0980a4d60 con 0x5620941588c0 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fd0900506c0 con 0x5620941588c0 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 10 client.27280 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 10 client.27280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 20 client.27280 populate_metadata read hostname 'smithi059' 2024-04-02T17:29:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941dd6a0 0x5620941dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:29:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 4 client.27280 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:29:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.008+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620940cb2b0 con 0x5620941dd6a0 2024-04-02T17:29:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.009+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941dd6a0 0x5620941dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:29:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.009+0000 7fd0a9a00640 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941dd6a0 0x5620941dfac0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cd67c4ab7a334c server_cookie=b8d0b817ba8c49b4 in_seq=0 out_seq=0 2024-04-02T17:29:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.009+0000 7fd08b400640 10 client.27280 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.197+0000 7fd08b400640 1 -- 172.21.15.59:0/1754333794 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620940cb2b0 con 0x5620941dd6a0 2024-04-02T17:29:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.197+0000 7fd08b400640 10 client.27280 handle_command_reply: tid=0 2024-04-02T17:29:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.197+0000 7fd0af19b740 1 client.27280 shutdown 2024-04-02T17:29:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 msgr2=0x5620941671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x5620941671e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd090034a80 tx=0x7fd090034940 comp rx=0 tx=0).stop 2024-04-02T17:29:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 shutdown_connections 2024-04-02T17:29:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5620941dd6a0 0x5620941dfac0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56209415da30 0x5620941681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620941588c0 0x5620941671e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 --2- 172.21.15.59:0/1754333794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562094157ed0 0x562094166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 >> 172.21.15.59:0/1754333794 conn(0x562094153440 msgr2=0x562094161f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.198+0000 7fd08b400640 0 client.27280 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:29:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.200+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 shutdown_connections 2024-04-02T17:29:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.200+0000 7fd0af19b740 1 -- 172.21.15.59:0/1754333794 wait complete. 2024-04-02T17:29:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.200+0000 7fd0af19b740 20 client.27280 trim_cache size 0 max 16384 2024-04-02T17:29:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:29:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:29:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:29:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.201+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 msgr2=0x7fd08c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.201+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 0x7fd08c0a64a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd09002c220 tx=0x7fd0900058b0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.201+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 msgr2=0x7fd0a4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.201+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd09800efb0 tx=0x7fd09800c800 comp rx=0 tx=0).stop 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 shutdown_connections 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fd08c0a3fe0 0x7fd08c0a64a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0a4075ac0 0x7fd0a419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0a41099e0 0x7fd0a41a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 --2- 172.21.15.59:0/2003300932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0a40744e0 0x7fd0a4199f50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:29:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 >> 172.21.15.59:0/2003300932 conn(0x7fd0a40fd0c0 msgr2=0x7fd0a4105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:29:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 shutdown_connections 2024-04-02T17:29:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:29:53.202+0000 7fd0ac400640 1 -- 172.21.15.59:0/2003300932 wait complete. 2024-04-02T17:29:53.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 161282 42.11 MB/sec execute 80 sec latency 196.800 ms 2024-04-02T17:29:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:54 smithi154 ceph-mon[36479]: pgmap v894: 97 pgs: 97 active+clean; 741 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 639 op/s 2024-04-02T17:29:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:54 smithi059 ceph-mon[31664]: pgmap v894: 97 pgs: 97 active+clean; 741 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 639 op/s 2024-04-02T17:29:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:54 smithi164 ceph-mon[36499]: pgmap v894: 97 pgs: 97 active+clean; 741 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 639 op/s 2024-04-02T17:29:54.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 162368 42.33 MB/sec execute 81 sec latency 28.128 ms 2024-04-02T17:29:55.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 163176 42.42 MB/sec execute 82 sec latency 58.843 ms 2024-04-02T17:29:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:56 smithi154 ceph-mon[36479]: pgmap v895: 97 pgs: 97 active+clean; 745 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 652 op/s 2024-04-02T17:29:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:56 smithi059 ceph-mon[31664]: pgmap v895: 97 pgs: 97 active+clean; 745 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 652 op/s 2024-04-02T17:29:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:56 smithi164 ceph-mon[36499]: pgmap v895: 97 pgs: 97 active+clean; 745 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.1 MiB/s wr, 652 op/s 2024-04-02T17:29:56.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 163827 42.33 MB/sec execute 83 sec latency 72.450 ms 2024-04-02T17:29:57.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 164742 42.31 MB/sec execute 84 sec latency 34.108 ms 2024-04-02T17:29:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:29:58 smithi154 ceph-mon[36479]: pgmap v896: 97 pgs: 97 active+clean; 752 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-04-02T17:29:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:29:58 smithi059 ceph-mon[31664]: pgmap v896: 97 pgs: 97 active+clean; 752 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-04-02T17:29:58.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:29:58 smithi164 ceph-mon[36499]: pgmap v896: 97 pgs: 97 active+clean; 752 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-04-02T17:29:58.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 165878 42.57 MB/sec execute 85 sec latency 37.307 ms 2024-04-02T17:29:59.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 166638 42.64 MB/sec execute 86 sec latency 162.938 ms 2024-04-02T17:30:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:29:59.998+0000 7fe5e1c00700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:30:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:00 smithi059 ceph-mon[31664]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:30:00.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:00 smithi164 ceph-mon[36499]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:30:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:00 smithi154 ceph-mon[36479]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-02T17:30:00.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 167216 42.43 MB/sec execute 87 sec latency 177.482 ms 2024-04-02T17:30:01.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:01 smithi164 ceph-mon[36499]: pgmap v897: 97 pgs: 97 active+clean; 756 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 679 op/s 2024-04-02T17:30:01.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:01 smithi154 ceph-mon[36479]: pgmap v897: 97 pgs: 97 active+clean; 756 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 679 op/s 2024-04-02T17:30:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:01 smithi059 ceph-mon[31664]: pgmap v897: 97 pgs: 97 active+clean; 756 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 679 op/s 2024-04-02T17:30:01.607 INFO:tasks.workunit.client.0.smithi059.stdout: 10 167859 42.36 MB/sec execute 88 sec latency 284.268 ms 2024-04-02T17:30:02.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 168863 42.35 MB/sec execute 89 sec latency 35.293 ms 2024-04-02T17:30:03.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:03 smithi164 ceph-mon[36499]: pgmap v898: 97 pgs: 97 active+clean; 763 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-04-02T17:30:03.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:03 smithi154 ceph-mon[36479]: pgmap v898: 97 pgs: 97 active+clean; 763 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-04-02T17:30:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:03 smithi059 ceph-mon[31664]: pgmap v898: 97 pgs: 97 active+clean; 763 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-04-02T17:30:03.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 169832 42.64 MB/sec execute 90 sec latency 32.315 ms 2024-04-02T17:30:04.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:04 smithi164 ceph-mon[36499]: pgmap v899: 97 pgs: 97 active+clean; 772 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 722 op/s 2024-04-02T17:30:04.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:04 smithi154 ceph-mon[36479]: pgmap v899: 97 pgs: 97 active+clean; 772 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 722 op/s 2024-04-02T17:30:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:04 smithi059 ceph-mon[31664]: pgmap v899: 97 pgs: 97 active+clean; 772 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 722 op/s 2024-04-02T17:30:04.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 170275 42.43 MB/sec execute 91 sec latency 372.990 ms 2024-04-02T17:30:05.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 170556 42.11 MB/sec execute 92 sec latency 586.991 ms 2024-04-02T17:30:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 -- 172.21.15.59:0/3501708408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22901003f0 msgr2=0x7f2290102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 --2- 172.21.15.59:0/3501708408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22901003f0 0x7f2290102820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f228402bc30 comp rx=0 tx=0).stop 2024-04-02T17:30:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 -- 172.21.15.59:0/3501708408 shutdown_connections 2024-04-02T17:30:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 --2- 172.21.15.59:0/3501708408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f22901051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 --2- 172.21.15.59:0/3501708408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2290105710 0x7f2290105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 --2- 172.21.15.59:0/3501708408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22901003f0 0x7f2290102820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.399+0000 7f2297800640 1 -- 172.21.15.59:0/3501708408 >> 172.21.15.59:0/3501708408 conn(0x7f22900fbf90 msgr2=0x7f22900fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:06 smithi154 ceph-mon[36479]: pgmap v900: 97 pgs: 97 active+clean; 808 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-04-02T17:30:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.400+0000 7f2297800640 1 -- 172.21.15.59:0/3501708408 shutdown_connections 2024-04-02T17:30:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.400+0000 7f2297800640 1 -- 172.21.15.59:0/3501708408 wait complete. 2024-04-02T17:30:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.402+0000 7f2297800640 1 Processor -- start 2024-04-02T17:30:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.402+0000 7f2297800640 1 -- start start 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.402+0000 7f2297800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22901003f0 0x7f229010b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.402+0000 7f2297800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f229010bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2297800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 0x7f22901ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2297800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2290113d80 con 0x7f2290102d60 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2297800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2290113c00 con 0x7f2290105710 2024-04-02T17:30:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2297800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2290113f00 con 0x7f22901003f0 2024-04-02T17:30:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f228fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f229010bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2295800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 0x7f22901ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2295800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 0x7f22901ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42212/0 (socket says 172.21.15.59:42212) 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2295800640 1 -- 172.21.15.59:0/1054387040 learned_addr learned my addr 172.21.15.59:0/1054387040 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.403+0000 7f2294e00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22901003f0 0x7f229010b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 -- 172.21.15.59:0/1054387040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22901003f0 msgr2=0x7f229010b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22901003f0 0x7f229010b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 -- 172.21.15.59:0/1054387040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 msgr2=0x7f22901ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 0x7f22901ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 -- 172.21.15.59:0/1054387040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229010c5d0 con 0x7f2290102d60 2024-04-02T17:30:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.404+0000 7f228fe00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f229010bf30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f227c005fd0 tx=0x7f227c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.405+0000 7f228d600640 1 -- 172.21.15.59:0/1054387040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c009c30 con 0x7f2290102d60 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.405+0000 7f228d600640 1 -- 172.21.15.59:0/1054387040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c009dd0 con 0x7f2290102d60 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.405+0000 7f228d600640 1 -- 172.21.15.59:0/1054387040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c010460 con 0x7f2290102d60 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.405+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229006b580 con 0x7f2290102d60 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.405+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229006ba80 con 0x7f2290102d60 2024-04-02T17:30:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.409+0000 7f229a595740 1 Processor -- start 2024-04-02T17:30:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 -- start start 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e368918c0 0x559e368963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e36898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e367f6d70 con 0x559e368918c0 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559e3679f2a0 con 0x559e36896a30 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.410+0000 7f229a595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559e367f2d20 con 0x559e36890ed0 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47020/0 (socket says 172.21.15.59:47020) 2024-04-02T17:30:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 -- 172.21.15.59:0/2427178073 learned_addr learned my addr 172.21.15.59:0/2427178073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 -- 172.21.15.59:0/2427178073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 msgr2=0x559e36898ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e36898ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 -- 172.21.15.59:0/2427178073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e368918c0 msgr2=0x559e368963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.411+0000 7f2295800640 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e368918c0 0x559e368963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.412+0000 7f2295800640 1 -- 172.21.15.59:0/2427178073 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e367b3840 con 0x559e36890ed0 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.412+0000 7f228d600640 1 -- 172.21.15.59:0/1054387040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f227c010600 con 0x7f2290102d60 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.412+0000 7f2295800640 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2288009f30 tx=0x7f2288010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4a227bdb15ca5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.413+0000 7f2282000640 1 -- 172.21.15.59:0/2427178073 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288016070 con 0x559e36890ed0 2024-04-02T17:30:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.413+0000 7f2282000640 1 -- 172.21.15.59:0/2427178073 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288012930 con 0x559e36890ed0 2024-04-02T17:30:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.413+0000 7f2282000640 1 -- 172.21.15.59:0/2427178073 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288011940 con 0x559e36890ed0 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f228d600640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 0x7f22780b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 -- 172.21.15.59:0/2427178073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 msgr2=0x559e368912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2288009f30 tx=0x7f2288010800 comp rx=0 tx=0).stop 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f228d600640 1 -- 172.21.15.59:0/1054387040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f227c0cff90 con 0x7f2290102d60 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 -- 172.21.15.59:0/2427178073 shutdown_connections 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e368918c0 0x559e368963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e36898ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 --2- 172.21.15.59:0/2427178073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e368912f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.414+0000 7f229a595740 1 -- 172.21.15.59:0/2427178073 >> 172.21.15.59:0/2427178073 conn(0x559e3688c440 msgr2=0x559e3688c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.415+0000 7f2294e00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 0x7f22780b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.415+0000 7f229a595740 1 -- 172.21.15.59:0/2427178073 shutdown_connections 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.415+0000 7f229a595740 1 -- 172.21.15.59:0/2427178073 wait complete. 2024-04-02T17:30:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.416+0000 7f229a595740 1 -- start start 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.416+0000 7f2294e00640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 0x7f22780b71a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f228402c7b0 tx=0x7f2284005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.416+0000 7f229a595740 1 Processor -- start 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e3689b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e368999e0 con 0x559e36896a30 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559e367b2660 con 0x559e368918c0 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559e36899b60 con 0x559e36890ed0 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.417+0000 7f229a595740 10 client.0 _ll_register_callbacks cb 0x559e367b2050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47030/0 (socket says 172.21.15.59:47030) 2024-04-02T17:30:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 -- 172.21.15.59:0/3354855508 learned_addr learned my addr 172.21.15.59:0/3354855508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f228fe00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e3689b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2295800640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 -- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 msgr2=0x559e3689b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e3689b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 -- 172.21.15.59:0/3354855508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 msgr2=0x559e368a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2294e00640 1 -- 172.21.15.59:0/3354855508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e367c1e80 con 0x559e36890ed0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.418+0000 7f2295800640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.419+0000 7f2294e00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2284035000 tx=0x7f228402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e487ffe5a76a8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.419+0000 7f2276a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.419+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228404d030 con 0x559e36890ed0 2024-04-02T17:30:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f22840046c0 con 0x559e36890ed0 2024-04-02T17:30:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2288012930 con 0x559e36890ed0 2024-04-02T17:30:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f22840048c0 con 0x559e36890ed0 2024-04-02T17:30:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f229a595740 10 client.27292 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x559e3650b210 con 0x559e36890ed0 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.420+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288012930 con 0x559e36890ed0 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f2284049450 con 0x559e36890ed0 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 10 client.27292 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 10 client.27292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 20 client.27292 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e368a72a0 0x559e368a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 4 client.27292 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.421+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e367a6cd0 con 0x559e368a72a0 2024-04-02T17:30:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.422+0000 7f228fe00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e368a72a0 0x559e368a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.423+0000 7f228fe00640 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e368a72a0 0x559e368a96d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cc46f1295de6a0 server_cookie=c489dcab5bc8a49 in_seq=0 out_seq=0 2024-04-02T17:30:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.423+0000 7f2276a00640 10 client.27292 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.428+0000 7f2276a00640 1 -- 172.21.15.59:0/3354855508 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e367a6cd0 con 0x559e368a72a0 2024-04-02T17:30:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.428+0000 7f2276a00640 10 client.27292 handle_command_reply: tid=0 2024-04-02T17:30:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.428+0000 7f229a595740 1 client.27292 shutdown 2024-04-02T17:30:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 msgr2=0x559e3689b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2284035000 tx=0x7f228402bc30 comp rx=0 tx=0).stop 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 shutdown_connections 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e368a72a0 0x559e368a96d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e3689b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 --2- 172.21.15.59:0/3354855508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689b3d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 >> 172.21.15.59:0/3354855508 conn(0x559e3688c440 msgr2=0x559e3688b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.429+0000 7f2276a00640 0 client.27292 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.430+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 shutdown_connections 2024-04-02T17:30:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.430+0000 7f229a595740 1 -- 172.21.15.59:0/3354855508 wait complete. 2024-04-02T17:30:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.431+0000 7f229a595740 20 client.27292 trim_cache size 0 max 16384 2024-04-02T17:30:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 Processor -- start 2024-04-02T17:30:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 -- start start 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36890ed0 0x559e3689b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e3688adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22901af810 con 0x559e36890ed0 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b1350 con 0x559e36896a30 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.462+0000 7f229a595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22901afee0 con 0x559e368918c0 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f2295800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e3688adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:47034/0 (socket says 172.21.15.59:47034) 2024-04-02T17:30:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 -- 172.21.15.59:0/1135828227 learned_addr learned my addr 172.21.15.59:0/1135828227 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f2294e00640 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36890ed0 0x559e3689b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 -- 172.21.15.59:0/1135828227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 msgr2=0x559e3688adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e3688adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 -- 172.21.15.59:0/1135828227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36890ed0 msgr2=0x559e3689b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36890ed0 0x559e3689b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f228fe00640 1 -- 172.21.15.59:0/1135828227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e365d7620 con 0x559e368918c0 2024-04-02T17:30:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.463+0000 7f2295800640 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e3688adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.464+0000 7f228fe00640 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f227c09c1a0 tx=0x7f227c0cd900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7011bb9978218212 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.465+0000 7f2281600640 1 -- 172.21.15.59:0/1135828227 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c020040 con 0x559e368918c0 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.465+0000 7f2281600640 1 -- 172.21.15.59:0/1135828227 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c0b5070 con 0x559e368918c0 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.465+0000 7f2281600640 1 -- 172.21.15.59:0/1135828227 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c098880 con 0x559e368918c0 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 -- 172.21.15.59:0/1135828227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 msgr2=0x559e368a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f227c09c1a0 tx=0x7f227c0cd900 comp rx=0 tx=0).stop 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 -- 172.21.15.59:0/1135828227 shutdown_connections 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e36896a30 0x559e3688adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e368918c0 0x559e368a28e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 --2- 172.21.15.59:0/1135828227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36890ed0 0x559e3689b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 -- 172.21.15.59:0/1135828227 >> 172.21.15.59:0/1135828227 conn(0x559e3688c440 msgr2=0x559e3688c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 -- 172.21.15.59:0/1135828227 shutdown_connections 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.466+0000 7f229a595740 1 -- 172.21.15.59:0/1135828227 wait complete. 2024-04-02T17:30:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.467+0000 7f229a595740 1 -- start start 2024-04-02T17:30:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.467+0000 7f229a595740 1 Processor -- start 2024-04-02T17:30:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b0c80 con 0x559e36896a30 2024-04-02T17:30:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b05b0 con 0x559e368918c0 2024-04-02T17:30:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2290072b40 con 0x559e36890ed0 2024-04-02T17:30:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f229a595740 10 client.0 _ll_register_callbacks cb 0x7f228402cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f228fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f228fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42240/0 (socket says 172.21.15.59:42240) 2024-04-02T17:30:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f228fe00640 1 -- 172.21.15.59:0/3955823437 learned_addr learned my addr 172.21.15.59:0/3955823437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.469+0000 7f2294e00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f2295800640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 -- 172.21.15.59:0/3955823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 msgr2=0x559e3689fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 -- 172.21.15.59:0/3955823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 msgr2=0x559e368a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 -- 172.21.15.59:0/3955823437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e368a1940 con 0x559e368918c0 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f2295800640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f228fe00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f227c0b2040 tx=0x7f227c0a0680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cf4b452173df38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.470+0000 7f2276a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c0ba060 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f227c0a0120 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227c0b5070 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f227c09a2b0 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f229a595740 10 client.27260 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x559e368a2c80 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.471+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c0b5070 con 0x559e368918c0 2024-04-02T17:30:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f227c003f60 con 0x559e368918c0 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 10 client.27260 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 10 client.27260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 20 client.27260 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e3690e000 0x559e36910420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 4 client.27260 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e368042b0 con 0x559e3690e000 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.472+0000 7f2294e00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e3690e000 0x559e36910420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.473+0000 7f2294e00640 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e3690e000 0x559e36910420 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7d5b144dda3b70 server_cookie=a8f73bce5b146e0d in_seq=0 out_seq=0 2024-04-02T17:30:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.473+0000 7f2276a00640 10 client.27260 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:06.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:06 smithi059 ceph-mon[31664]: pgmap v900: 97 pgs: 97 active+clean; 808 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-04-02T17:30:06.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:06 smithi164 ceph-mon[36499]: pgmap v900: 97 pgs: 97 active+clean; 808 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-04-02T17:30:06.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 170985 41.89 MB/sec execute 93 sec latency 100.375 ms 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f2276a00640 1 -- 172.21.15.59:0/3955823437 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e368042b0 con 0x559e3690e000 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f2276a00640 10 client.27260 handle_command_reply: tid=0 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 client.27260 shutdown 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 msgr2=0x559e368a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f227c0b2040 tx=0x7f227c0a0680 comp rx=0 tx=0).stop 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 shutdown_connections 2024-04-02T17:30:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559e3690e000 0x559e36910420 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e36896a30 0x559e368a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e368918c0 0x559e368a0340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 --2- 172.21.15.59:0/3955823437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e36890ed0 0x559e3689fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 >> 172.21.15.59:0/3955823437 conn(0x559e3688c440 msgr2=0x559e3688adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.681+0000 7f2276a00640 0 client.27260 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.682+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 shutdown_connections 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.682+0000 7f229a595740 1 -- 172.21.15.59:0/3955823437 wait complete. 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.682+0000 7f229a595740 20 client.27260 trim_cache size 0 max 16384 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:30:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 msgr2=0x7f22780b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 0x7f22780b71a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f228402c7b0 tx=0x7f2284005fd0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 msgr2=0x7f229010bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f229010bf30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f227c005fd0 tx=0x7f227c00dde0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 shutdown_connections 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f22780b4ce0 0x7f22780b71a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290102d60 0x7f229010bf30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2290105710 0x7f22901ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 --2- 172.21.15.59:0/1054387040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22901003f0 0x7f229010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 >> 172.21.15.59:0/1054387040 conn(0x7f22900fbf90 msgr2=0x7f2290103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 shutdown_connections 2024-04-02T17:30:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:06.683+0000 7f2297800640 1 -- 172.21.15.59:0/1054387040 wait complete. 2024-04-02T17:30:07.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 171761 41.80 MB/sec execute 94 sec latency 166.576 ms 2024-04-02T17:30:08.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:08 smithi164 ceph-mon[36499]: pgmap v901: 97 pgs: 97 active+clean; 804 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 764 op/s 2024-04-02T17:30:08.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:08 smithi154 ceph-mon[36479]: pgmap v901: 97 pgs: 97 active+clean; 804 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 764 op/s 2024-04-02T17:30:08.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:08 smithi059 ceph-mon[31664]: pgmap v901: 97 pgs: 97 active+clean; 804 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 764 op/s 2024-04-02T17:30:08.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 172915 42.10 MB/sec execute 95 sec latency 31.430 ms 2024-04-02T17:30:09.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 173777 42.22 MB/sec execute 96 sec latency 42.755 ms 2024-04-02T17:30:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:10 smithi154 ceph-mon[36479]: pgmap v902: 97 pgs: 97 active+clean; 791 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 854 op/s 2024-04-02T17:30:10.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:10 smithi059 ceph-mon[31664]: pgmap v902: 97 pgs: 97 active+clean; 791 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 854 op/s 2024-04-02T17:30:10.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:10 smithi164 ceph-mon[36499]: pgmap v902: 97 pgs: 97 active+clean; 791 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 854 op/s 2024-04-02T17:30:10.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 173989 41.89 MB/sec execute 97 sec latency 589.339 ms 2024-04-02T17:30:11.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 174188 41.55 MB/sec execute 98 sec latency 827.307 ms 2024-04-02T17:30:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:12 smithi154 ceph-mon[36479]: pgmap v903: 97 pgs: 97 active+clean; 805 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 949 op/s 2024-04-02T17:30:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:12 smithi059 ceph-mon[31664]: pgmap v903: 97 pgs: 97 active+clean; 805 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 949 op/s 2024-04-02T17:30:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:12 smithi164 ceph-mon[36499]: pgmap v903: 97 pgs: 97 active+clean; 805 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 949 op/s 2024-04-02T17:30:12.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 174499 41.29 MB/sec execute 99 sec latency 106.409 ms 2024-04-02T17:30:13.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 175273 41.23 MB/sec execute 100 sec latency 98.594 ms 2024-04-02T17:30:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:14 smithi154 ceph-mon[36479]: pgmap v904: 97 pgs: 97 active+clean; 802 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.08k op/s 2024-04-02T17:30:14.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:14 smithi059 ceph-mon[31664]: pgmap v904: 97 pgs: 97 active+clean; 802 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.08k op/s 2024-04-02T17:30:14.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:14 smithi164 ceph-mon[36499]: pgmap v904: 97 pgs: 97 active+clean; 802 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.08k op/s 2024-04-02T17:30:14.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 176332 41.46 MB/sec execute 101 sec latency 35.491 ms 2024-04-02T17:30:15.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 177192 41.50 MB/sec execute 102 sec latency 35.451 ms 2024-04-02T17:30:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:16 smithi154 ceph-mon[36479]: pgmap v905: 97 pgs: 97 active+clean; 781 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T17:30:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:16 smithi059 ceph-mon[31664]: pgmap v905: 97 pgs: 97 active+clean; 781 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T17:30:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:16 smithi164 ceph-mon[36499]: pgmap v905: 97 pgs: 97 active+clean; 781 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T17:30:16.608 INFO:tasks.workunit.client.0.smithi059.stdout: 10 178041 41.54 MB/sec execute 103 sec latency 68.837 ms 2024-04-02T17:30:17.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 178934 41.56 MB/sec execute 104 sec latency 61.516 ms 2024-04-02T17:30:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:18 smithi154 ceph-mon[36479]: pgmap v906: 97 pgs: 97 active+clean; 770 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-04-02T17:30:18.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:18 smithi059 ceph-mon[31664]: pgmap v906: 97 pgs: 97 active+clean; 770 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-04-02T17:30:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:18 smithi164 ceph-mon[36499]: pgmap v906: 97 pgs: 97 active+clean; 770 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-04-02T17:30:18.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 179889 41.67 MB/sec execute 105 sec latency 37.362 ms 2024-04-02T17:30:19.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 180681 41.66 MB/sec execute 106 sec latency 48.759 ms 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 -- 172.21.15.59:0/3777552940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 msgr2=0x7fae180749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 --2- 172.21.15.59:0/3777552940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae180749e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fae04009930 tx=0x7fae0402bc30 comp rx=0 tx=0).stop 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 -- 172.21.15.59:0/3777552940 shutdown_connections 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 --2- 172.21.15.59:0/3777552940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 0x7fae18110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 --2- 172.21.15.59:0/3777552940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae180749e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 --2- 172.21.15.59:0/3777552940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae18073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 -- 172.21.15.59:0/3777552940 >> 172.21.15.59:0/3777552940 conn(0x7fae180fd090 msgr2=0x7fae180ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 -- 172.21.15.59:0/3777552940 shutdown_connections 2024-04-02T17:30:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.847+0000 7fae20200640 1 -- 172.21.15.59:0/3777552940 wait complete. 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.848+0000 7fae20200640 1 Processor -- start 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.848+0000 7fae20200640 1 -- start start 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae1819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 0x7fae181a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae18113df0 con 0x7fae18075b40 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae18113c70 con 0x7fae18074560 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae20200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae18113f70 con 0x7fae1810e110 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae1d800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae1d800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38950/0 (socket says 172.21.15.59:38950) 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae1d800640 1 -- 172.21.15.59:0/4060781804 learned_addr learned my addr 172.21.15.59:0/4060781804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae1ce00640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae1819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.849+0000 7fae1e200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 0x7fae181a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1d800640 1 -- 172.21.15.59:0/4060781804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 msgr2=0x7fae181a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1d800640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 0x7fae181a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1d800640 1 -- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 msgr2=0x7fae1819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1d800640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae1819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1d800640 1 -- 172.21.15.59:0/4060781804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae181a6000 con 0x7fae18074560 2024-04-02T17:30:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.850+0000 7fae1ce00640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae1819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae1d800640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e9f0 tx=0x7fae0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae16000640 1 -- 172.21.15.59:0/4060781804 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00cd80 con 0x7fae18074560 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae16000640 1 -- 172.21.15.59:0/4060781804 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae0c010910 con 0x7fae18074560 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae16000640 1 -- 172.21.15.59:0/4060781804 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c018990 con 0x7fae18074560 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181a62f0 con 0x7fae18074560 2024-04-02T17:30:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.851+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a6860 con 0x7fae18074560 2024-04-02T17:30:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 Processor -- start 2024-04-02T17:30:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 -- start start 2024-04-02T17:30:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56507952cd70 con 0x5650795cca30 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650794d52a0 con 0x5650795c78c0 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.855+0000 7fae22f7c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565079528d20 con 0x5650795c6ed0 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae1e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae16000640 1 -- 172.21.15.59:0/4060781804 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae0c010ac0 con 0x7fae18074560 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae1e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:37582/0 (socket says 172.21.15.59:37582) 2024-04-02T17:30:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae1e200640 1 -- 172.21.15.59:0/1040423890 learned_addr learned my addr 172.21.15.59:0/1040423890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae1d800640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.856+0000 7fae1ce00640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1e200640 1 -- 172.21.15.59:0/1040423890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 msgr2=0x5650795cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1e200640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1e200640 1 -- 172.21.15.59:0/1040423890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 msgr2=0x5650795ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1e200640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1e200640 1 -- 172.21.15.59:0/1040423890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650794e9840 con 0x5650795c6ed0 2024-04-02T17:30:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.857+0000 7fae1ce00640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:30:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.858+0000 7fae1e200640 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fae10009fc0 tx=0x7fae10010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b1a0a8809266ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.858+0000 7fae16000640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 0x7fae000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.858+0000 7fae0aa00640 1 -- 172.21.15.59:0/1040423890 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10003b70 con 0x5650795c6ed0 2024-04-02T17:30:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.858+0000 7fae0aa00640 1 -- 172.21.15.59:0/1040423890 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10003d10 con 0x5650795c6ed0 2024-04-02T17:30:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.858+0000 7fae0aa00640 1 -- 172.21.15.59:0/1040423890 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10011b40 con 0x5650795c6ed0 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae16000640 1 -- 172.21.15.59:0/4060781804 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fae0c0d08d0 con 0x7fae18074560 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae1d800640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 0x7fae000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 -- 172.21.15.59:0/1040423890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 msgr2=0x5650795c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fae10009fc0 tx=0x7fae10010860 comp rx=0 tx=0).stop 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 -- 172.21.15.59:0/1040423890 shutdown_connections 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 --2- 172.21.15.59:0/1040423890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c6ed0 0x5650795c72f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.859+0000 7fae22f7c740 1 -- 172.21.15.59:0/1040423890 >> 172.21.15.59:0/1040423890 conn(0x5650795c2440 msgr2=0x5650795c2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.860+0000 7fae22f7c740 1 -- 172.21.15.59:0/1040423890 shutdown_connections 2024-04-02T17:30:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.860+0000 7fae22f7c740 1 -- 172.21.15.59:0/1040423890 wait complete. 2024-04-02T17:30:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.860+0000 7fae1d800640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 0x7fae000a6190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c6a0 tx=0x7fae0c018420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.860+0000 7fae22f7c740 1 -- start start 2024-04-02T17:30:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.861+0000 7fae22f7c740 1 Processor -- start 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c6ed0 0x5650795d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650795cf9e0 con 0x5650795c6ed0 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650794e8660 con 0x5650795c78c0 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650795cfb60 con 0x5650795cca30 2024-04-02T17:30:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae22f7c740 10 client.0 _ll_register_callbacks cb 0x5650794e8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae1ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae1ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38964/0 (socket says 172.21.15.59:38964) 2024-04-02T17:30:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae1ce00640 1 -- 172.21.15.59:0/4009898490 learned_addr learned my addr 172.21.15.59:0/4009898490 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.862+0000 7fae1e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c6ed0 0x5650795d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.863+0000 7fae1ce00640 1 -- 172.21.15.59:0/4009898490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 msgr2=0x5650795d8a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:30:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.863+0000 7fae1ce00640 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d8a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.863+0000 7fae1ce00640 1 -- 172.21.15.59:0/4009898490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c6ed0 msgr2=0x5650795d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.863+0000 7fae1ce00640 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c6ed0 0x5650795d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.863+0000 7fae1ce00640 1 -- 172.21.15.59:0/4009898490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650794f7e80 con 0x5650795c78c0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.864+0000 7fae1ce00640 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fae04009900 tx=0x7fae04004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4056ac82f4b689 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.864+0000 7fadff400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.864+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04005d50 con 0x5650795c78c0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.864+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fae04005250 con 0x5650795c78c0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.865+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10003d10 con 0x5650795c78c0 2024-04-02T17:30:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.865+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fae04005d50 con 0x5650795c78c0 2024-04-02T17:30:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.865+0000 7fae22f7c740 10 client.27272 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.865+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x565079241210 con 0x5650795c78c0 2024-04-02T17:30:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.865+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10003d10 con 0x5650795c78c0 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fae040035f0 con 0x5650795c78c0 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 10 client.27272 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 10 client.27272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 20 client.27272 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x5650795df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 4 client.27272 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650794dccd0 con 0x5650795dd280 2024-04-02T17:30:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.866+0000 7fae1e200640 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x5650795df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.867+0000 7fae1e200640 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x5650795df6b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb57ec42256f971 server_cookie=188caf82167a9c69 in_seq=0 out_seq=0 2024-04-02T17:30:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.867+0000 7fadff400640 10 client.27272 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.870+0000 7fadff400640 1 -- 172.21.15.59:0/4009898490 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650794dccd0 con 0x5650795dd280 2024-04-02T17:30:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.870+0000 7fadff400640 10 client.27272 handle_command_reply: tid=0 2024-04-02T17:30:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.870+0000 7fae22f7c740 1 client.27272 shutdown 2024-04-02T17:30:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 msgr2=0x5650795d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fae04009900 tx=0x7fae04004370 comp rx=0 tx=0).stop 2024-04-02T17:30:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 shutdown_connections 2024-04-02T17:30:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x5650795df6b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fadff400640 0 client.27272 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c78c0 0x5650795d19c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 --2- 172.21.15.59:0/4009898490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c6ed0 0x5650795d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.871+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 >> 172.21.15.59:0/4009898490 conn(0x5650795c2440 msgr2=0x5650795c1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.872+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 shutdown_connections 2024-04-02T17:30:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.872+0000 7fae22f7c740 1 -- 172.21.15.59:0/4009898490 wait complete. 2024-04-02T17:30:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.872+0000 7fae22f7c740 20 client.27272 trim_cache size 0 max 16384 2024-04-02T17:30:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.896+0000 7fae22f7c740 1 Processor -- start 2024-04-02T17:30:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 -- start start 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c78c0 0x5650795d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae181af8d0 con 0x5650795cca30 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae181b1410 con 0x5650795c6ed0 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae22f7c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae181affa0 con 0x5650795c78c0 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae1e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae1e200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38968/0 (socket says 172.21.15.59:38968) 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae1e200640 1 -- 172.21.15.59:0/368824713 learned_addr learned my addr 172.21.15.59:0/368824713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae1ce00640 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c78c0 0x5650795d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.897+0000 7fae1e200640 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 -- 172.21.15.59:0/368824713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c78c0 msgr2=0x5650795d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c78c0 0x5650795d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 -- 172.21.15.59:0/368824713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 msgr2=0x5650795e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 -- 172.21.15.59:0/368824713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507930d620 con 0x5650795c6ed0 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae1e200640 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f90 tx=0x7fae100048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4542939a16c7c628 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.898+0000 7fae0a000640 1 -- 172.21.15.59:0/368824713 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10004cb0 con 0x5650795c6ed0 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.899+0000 7fae0a000640 1 -- 172.21.15.59:0/368824713 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10004e50 con 0x5650795c6ed0 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.899+0000 7fae0a000640 1 -- 172.21.15.59:0/368824713 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1001b780 con 0x5650795c6ed0 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.899+0000 7fae22f7c740 1 -- 172.21.15.59:0/368824713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 msgr2=0x5650795c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.899+0000 7fae22f7c740 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae10009f90 tx=0x7fae100048a0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.901+0000 7fae22f7c740 1 -- 172.21.15.59:0/368824713 shutdown_connections 2024-04-02T17:30:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.901+0000 7fae22f7c740 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795cca30 0x5650795e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.901+0000 7fae22f7c740 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795c78c0 0x5650795d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.901+0000 7fae22f7c740 1 --2- 172.21.15.59:0/368824713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795c5db0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.901+0000 7fae22f7c740 1 -- 172.21.15.59:0/368824713 >> 172.21.15.59:0/368824713 conn(0x5650795c2440 msgr2=0x5650795c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.902+0000 7fae22f7c740 1 -- 172.21.15.59:0/368824713 shutdown_connections 2024-04-02T17:30:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.902+0000 7fae22f7c740 1 -- 172.21.15.59:0/368824713 wait complete. 2024-04-02T17:30:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.902+0000 7fae22f7c740 1 -- start start 2024-04-02T17:30:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 Processor -- start 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c78c0 0x5650795d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae181b0d40 con 0x5650795c78c0 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae181b0670 con 0x5650795c6ed0 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae181ad8b0 con 0x5650795cca30 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.903+0000 7fae22f7c740 10 client.0 _ll_register_callbacks cb 0x7fae04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.904+0000 7fae1e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.904+0000 7fae1e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:37606/0 (socket says 172.21.15.59:37606) 2024-04-02T17:30:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.904+0000 7fae1e200640 1 -- 172.21.15.59:0/739271972 learned_addr learned my addr 172.21.15.59:0/739271972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.904+0000 7fae1e200640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1ce00640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c78c0 0x5650795d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1e200640 1 -- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 msgr2=0x5650795d6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1e200640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795d6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1e200640 1 -- 172.21.15.59:0/739271972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c78c0 msgr2=0x5650795d6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1e200640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c78c0 0x5650795d6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.905+0000 7fae1e200640 1 -- 172.21.15.59:0/739271972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650795d7f80 con 0x5650795cca30 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.906+0000 7fae1e200640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fae10006010 tx=0x7fae10011500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73824de711897880 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.906+0000 7fadff400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.906+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10011b50 con 0x5650795cca30 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.906+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fae10011cf0 con 0x5650795cca30 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.906+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10004e50 con 0x5650795cca30 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.907+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fae10005000 con 0x5650795cca30 2024-04-02T17:30:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.907+0000 7fae22f7c740 10 client.27313 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.907+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x565079643b50 con 0x5650795cca30 2024-04-02T17:30:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.908+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10004e50 con 0x5650795cca30 2024-04-02T17:30:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.908+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fae1002a1c0 con 0x5650795cca30 2024-04-02T17:30:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 10 client.27313 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 20 client.27313 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x565079646300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 4 client.27313 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:30:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56507953a2b0 con 0x5650795dd280 2024-04-02T17:30:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.909+0000 7fae1ce00640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x565079646300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.910+0000 7fadff400640 10 client.27313 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:19.912+0000 7fae1ce00640 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x565079646300 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c6a36d510e5945 server_cookie=8e02b1856da0fe4d in_seq=0 out_seq=0 2024-04-02T17:30:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.140+0000 7fadff400640 1 -- 172.21.15.59:0/739271972 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56507953a2b0 con 0x5650795dd280 2024-04-02T17:30:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.140+0000 7fadff400640 10 client.27313 handle_command_reply: tid=0 2024-04-02T17:30:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.140+0000 7fae22f7c740 1 client.27313 shutdown 2024-04-02T17:30:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 msgr2=0x5650795d7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fae10006010 tx=0x7fae10011500 comp rx=0 tx=0).stop 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 shutdown_connections 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5650795dd280 0x565079646300 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650795c78c0 0x5650795d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650795cca30 0x5650795d7990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 --2- 172.21.15.59:0/739271972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650795c6ed0 0x5650795d6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 >> 172.21.15.59:0/739271972 conn(0x5650795c2440 msgr2=0x5650795c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fadff400640 0 client.27313 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 shutdown_connections 2024-04-02T17:30:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.141+0000 7fae22f7c740 1 -- 172.21.15.59:0/739271972 wait complete. 2024-04-02T17:30:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.142+0000 7fae22f7c740 20 client.27313 trim_cache size 0 max 16384 2024-04-02T17:30:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:30:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:30:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:30:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.142+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 msgr2=0x7fae000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.142+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 0x7fae000a6190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c6a0 tx=0x7fae0c018420 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.142+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 msgr2=0x7fae1819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.142+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e9f0 tx=0x7fae0c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 shutdown_connections 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fae000a3cd0 0x7fae000a6190 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae18075b40 0x7fae1819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae1810e110 0x7fae181a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 --2- 172.21.15.59:0/4060781804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae18074560 0x7fae1819e3e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 >> 172.21.15.59:0/4060781804 conn(0x7fae180fd090 msgr2=0x7fae181014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 shutdown_connections 2024-04-02T17:30:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:20.143+0000 7fae20200640 1 -- 172.21.15.59:0/4060781804 wait complete. 2024-04-02T17:30:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:20 smithi154 ceph-mon[36479]: pgmap v907: 97 pgs: 97 active+clean; 775 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-04-02T17:30:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:20 smithi059 ceph-mon[31664]: pgmap v907: 97 pgs: 97 active+clean; 775 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-04-02T17:30:20.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:20 smithi164 ceph-mon[36499]: pgmap v907: 97 pgs: 97 active+clean; 775 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-04-02T17:30:20.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 181393 41.63 MB/sec execute 107 sec latency 135.962 ms 2024-04-02T17:30:21.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 182088 41.61 MB/sec execute 108 sec latency 318.407 ms 2024-04-02T17:30:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:22 smithi154 ceph-mon[36479]: pgmap v908: 97 pgs: 97 active+clean; 780 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 990 op/s 2024-04-02T17:30:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:22 smithi059 ceph-mon[31664]: pgmap v908: 97 pgs: 97 active+clean; 780 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 990 op/s 2024-04-02T17:30:22.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:22 smithi164 ceph-mon[36499]: pgmap v908: 97 pgs: 97 active+clean; 780 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 990 op/s 2024-04-02T17:30:22.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 182951 41.68 MB/sec execute 109 sec latency 35.023 ms 2024-04-02T17:30:23.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 183735 41.66 MB/sec execute 110 sec latency 61.056 ms 2024-04-02T17:30:24.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:24 smithi154 ceph-mon[36479]: pgmap v909: 97 pgs: 97 active+clean; 786 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 951 op/s 2024-04-02T17:30:24.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:24 smithi059 ceph-mon[31664]: pgmap v909: 97 pgs: 97 active+clean; 786 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 951 op/s 2024-04-02T17:30:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:24 smithi164 ceph-mon[36499]: pgmap v909: 97 pgs: 97 active+clean; 786 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 951 op/s 2024-04-02T17:30:24.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 184639 41.73 MB/sec execute 111 sec latency 37.840 ms 2024-04-02T17:30:25.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 185582 41.78 MB/sec execute 112 sec latency 31.857 ms 2024-04-02T17:30:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:26 smithi154 ceph-mon[36479]: pgmap v910: 97 pgs: 97 active+clean; 794 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 824 op/s 2024-04-02T17:30:26.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:26 smithi059 ceph-mon[31664]: pgmap v910: 97 pgs: 97 active+clean; 794 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 824 op/s 2024-04-02T17:30:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:26 smithi164 ceph-mon[36499]: pgmap v910: 97 pgs: 97 active+clean; 794 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 824 op/s 2024-04-02T17:30:26.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 186473 41.89 MB/sec execute 113 sec latency 50.822 ms 2024-04-02T17:30:27.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 187238 41.87 MB/sec execute 114 sec latency 55.623 ms 2024-04-02T17:30:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:28 smithi154 ceph-mon[36479]: pgmap v911: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 642 op/s 2024-04-02T17:30:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:28 smithi059 ceph-mon[31664]: pgmap v911: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 642 op/s 2024-04-02T17:30:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:28 smithi164 ceph-mon[36499]: pgmap v911: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 642 op/s 2024-04-02T17:30:28.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 188072 41.85 MB/sec execute 115 sec latency 86.119 ms 2024-04-02T17:30:29.609 INFO:tasks.workunit.client.0.smithi059.stdout: 10 188991 41.98 MB/sec execute 116 sec latency 51.601 ms 2024-04-02T17:30:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:30 smithi154 ceph-mon[36479]: pgmap v912: 97 pgs: 97 active+clean; 805 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-04-02T17:30:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:30 smithi059 ceph-mon[31664]: pgmap v912: 97 pgs: 97 active+clean; 805 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-04-02T17:30:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:30 smithi164 ceph-mon[36499]: pgmap v912: 97 pgs: 97 active+clean; 805 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-04-02T17:30:30.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 189873 42.00 MB/sec execute 117 sec latency 41.384 ms 2024-04-02T17:30:31.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 190414 41.91 MB/sec execute 118 sec latency 191.843 ms 2024-04-02T17:30:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:32 smithi154 ceph-mon[36479]: pgmap v913: 97 pgs: 97 active+clean; 817 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 700 op/s 2024-04-02T17:30:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:32 smithi059 ceph-mon[31664]: pgmap v913: 97 pgs: 97 active+clean; 817 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 700 op/s 2024-04-02T17:30:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:32 smithi164 ceph-mon[36499]: pgmap v913: 97 pgs: 97 active+clean; 817 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 700 op/s 2024-04-02T17:30:32.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 191191 41.88 MB/sec execute 119 sec latency 92.502 ms 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 -- 172.21.15.59:0/3722982971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04108160 msgr2=0x7f3c04108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 --2- 172.21.15.59:0/3722982971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04108160 0x7f3c04108580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009900 tx=0x7f3bf402bc30 comp rx=0 tx=0).stop 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 -- 172.21.15.59:0/3722982971 shutdown_connections 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 --2- 172.21.15.59:0/3722982971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04109d10 0x7f3c041105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 --2- 172.21.15.59:0/3722982971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 0x7f3c041097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 --2- 172.21.15.59:0/3722982971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04108160 0x7f3c04108580 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 -- 172.21.15.59:0/3722982971 >> 172.21.15.59:0/3722982971 conn(0x7f3c04077110 msgr2=0x7f3c04079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.328+0000 7f3c09a00640 1 -- 172.21.15.59:0/3722982971 shutdown_connections 2024-04-02T17:30:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.329+0000 7f3c09a00640 1 -- 172.21.15.59:0/3722982971 wait complete. 2024-04-02T17:30:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.330+0000 7f3c09a00640 1 Processor -- start 2024-04-02T17:30:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.330+0000 7f3c09a00640 1 -- start start 2024-04-02T17:30:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.330+0000 7f3c09a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04108160 0x7f3c0419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c09a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 0x7f3c0419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c09a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c09a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c04113e10 con 0x7f3c04108160 2024-04-02T17:30:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c09a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c04113c90 con 0x7f3c04109350 2024-04-02T17:30:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c09a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3c04113f90 con 0x7f3c04109d10 2024-04-02T17:30:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c03400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c03400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58790/0 (socket says 172.21.15.59:58790) 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c03400640 1 -- 172.21.15.59:0/1704926793 learned_addr learned my addr 172.21.15.59:0/1704926793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c02000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 0x7f3c0419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.331+0000 7f3c02a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04108160 0x7f3c0419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 -- 172.21.15.59:0/1704926793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 msgr2=0x7f3c0419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 0x7f3c0419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 -- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04108160 msgr2=0x7f3c0419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04108160 0x7f3c0419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 -- 172.21.15.59:0/1704926793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c041a6070 con 0x7f3c04109d10 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3c03400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00eb10 tx=0x7f3bfc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3bfb400640 1 -- 172.21.15.59:0/1704926793 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc004b20 con 0x7f3c04109d10 2024-04-02T17:30:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3bfb400640 1 -- 172.21.15.59:0/1704926793 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bfc004cc0 con 0x7f3c04109d10 2024-04-02T17:30:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.332+0000 7f3bfb400640 1 -- 172.21.15.59:0/1704926793 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc010790 con 0x7f3c04109d10 2024-04-02T17:30:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.333+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c041a6360 con 0x7f3c04109d10 2024-04-02T17:30:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.333+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c041a6860 con 0x7f3c04109d10 2024-04-02T17:30:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.339+0000 7f3c0c5e0740 1 Processor -- start 2024-04-02T17:30:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.339+0000 7f3bfb400640 1 -- 172.21.15.59:0/1704926793 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3bfc005020 con 0x7f3c04109d10 2024-04-02T17:30:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.340+0000 7f3c0c5e0740 1 -- start start 2024-04-02T17:30:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557632638d70 con 0x5576326d2ed0 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5576325e12a0 con 0x5576326d38c0 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.341+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557632634d20 con 0x5576326d8a30 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.342+0000 7f3bfb400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 0x7f3be80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.342+0000 7f3bfb400640 1 -- 172.21.15.59:0/1704926793 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f3bfc0d0930 con 0x7f3c04109d10 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.343+0000 7f3c03400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 0x7f3be80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.343+0000 7f3c02000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.343+0000 7f3c02000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34048/0 (socket says 172.21.15.59:34048) 2024-04-02T17:30:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.343+0000 7f3c02000640 1 -- 172.21.15.59:0/2941359523 learned_addr learned my addr 172.21.15.59:0/2941359523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.344+0000 7f3c02a00640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.344+0000 7f3c02000640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02a00640 1 -- 172.21.15.59:0/2941359523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 msgr2=0x5576326daec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02a00640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326daec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02a00640 1 -- 172.21.15.59:0/2941359523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 msgr2=0x5576326d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02a00640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02a00640 1 -- 172.21.15.59:0/2941359523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576325f5840 con 0x5576326d38c0 2024-04-02T17:30:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c02000640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:30:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.345+0000 7f3c03400640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 0x7f3be80a6190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0a52f0 tx=0x7f3bfc005970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.346+0000 7f3c02a00640 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326d83c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009230 tx=0x7f3bf4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7cf7cf04f14bee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.346+0000 7f3befe00640 1 -- 172.21.15.59:0/2941359523 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4044020 con 0x5576326d38c0 2024-04-02T17:30:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.347+0000 7f3befe00640 1 -- 172.21.15.59:0/2941359523 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf402cb80 con 0x5576326d38c0 2024-04-02T17:30:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.347+0000 7f3befe00640 1 -- 172.21.15.59:0/2941359523 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4044020 con 0x5576326d38c0 2024-04-02T17:30:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/2941359523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 msgr2=0x5576326d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326d83c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009230 tx=0x7f3bf4009560 comp rx=0 tx=0).stop 2024-04-02T17:30:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/2941359523 shutdown_connections 2024-04-02T17:30:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326d83c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/2941359523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.349+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/2941359523 >> 172.21.15.59:0/2941359523 conn(0x5576326ce440 msgr2=0x5576326ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.350+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/2941359523 shutdown_connections 2024-04-02T17:30:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.354+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/2941359523 wait complete. 2024-04-02T17:30:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.355+0000 7f3c0c5e0740 1 -- start start 2024-04-02T17:30:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.355+0000 7f3c0c5e0740 1 Processor -- start 2024-04-02T17:30:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.358+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 0x5576326dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576326db980 con 0x5576326d2ed0 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5576325f4660 con 0x5576326d8a30 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576326dbb00 con 0x5576326d38c0 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.359+0000 7f3c0c5e0740 10 client.0 _ll_register_callbacks cb 0x5576325f4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.360+0000 7f3c02000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.361+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 0x5576326dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.362+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 0x5576326dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58818/0 (socket says 172.21.15.59:58818) 2024-04-02T17:30:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.362+0000 7f3c02a00640 1 -- 172.21.15.59:0/1623896431 learned_addr learned my addr 172.21.15.59:0/1623896431 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.362+0000 7f3c02000640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.364+0000 7f3c02000640 1 -- 172.21.15.59:0/1623896431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 msgr2=0x5576326dd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.364+0000 7f3c02000640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 0x5576326dd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.364+0000 7f3c02000640 1 -- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 msgr2=0x5576326e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.364+0000 7f3c02000640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.364+0000 7f3c02000640 1 -- 172.21.15.59:0/1623896431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557632603e80 con 0x5576326d2ed0 2024-04-02T17:30:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.365+0000 7f3c02000640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dd3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009530 tx=0x7f3bf0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6117cd0f4194a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.365+0000 7f3becc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:30:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.366+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000fe30 con 0x5576326d2ed0 2024-04-02T17:30:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.366+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f3bf0013120 con 0x5576326d2ed0 2024-04-02T17:30:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.367+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf402cb80 con 0x5576326d2ed0 2024-04-02T17:30:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.367+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3bf0004030 con 0x5576326d2ed0 2024-04-02T17:30:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.368+0000 7f3c0c5e0740 10 client.17166 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.368+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55763234d210 con 0x5576326d2ed0 2024-04-02T17:30:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.368+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf402cb80 con 0x5576326d2ed0 2024-04-02T17:30:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.368+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf0003c70 con 0x5576326d2ed0 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 10 client.17166 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 20 client.17166 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5576326e91d0 0x5576326eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 4 client.17166 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:30:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.369+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576325e8cd0 con 0x5576326e91d0 2024-04-02T17:30:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.370+0000 7f3c02a00640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5576326e91d0 0x5576326eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.372+0000 7f3c02a00640 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5576326e91d0 0x5576326eb5e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d29d0057389a3b7 server_cookie=63cd55e3ec4fece0 in_seq=0 out_seq=0 2024-04-02T17:30:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.372+0000 7f3becc00640 10 client.17166 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3becc00640 1 -- 172.21.15.59:0/1623896431 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576325e8cd0 con 0x5576326e91d0 2024-04-02T17:30:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3becc00640 10 client.17166 handle_command_reply: tid=0 2024-04-02T17:30:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 client.17166 shutdown 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 msgr2=0x5576326dd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dd3e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009530 tx=0x7f3bf0005f90 comp rx=0 tx=0).stop 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 shutdown_connections 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5576326e91d0 0x5576326eb5e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d38c0 0x5576326dd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1623896431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dd3e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 >> 172.21.15.59:0/1623896431 conn(0x5576326ce440 msgr2=0x5576326cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.374+0000 7f3becc00640 0 client.17166 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.375+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 shutdown_connections 2024-04-02T17:30:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.375+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1623896431 wait complete. 2024-04-02T17:30:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.375+0000 7f3c0c5e0740 20 client.17166 trim_cache size 0 max 16384 2024-04-02T17:30:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 Processor -- start 2024-04-02T17:30:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 -- start start 2024-04-02T17:30:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041af960 con 0x5576326d2ed0 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041b14a0 con 0x5576326d38c0 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.404+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041b0030 con 0x5576326d8a30 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.405+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.405+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34732/0 (socket says 172.21.15.59:34732) 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.405+0000 7f3c02a00640 1 -- 172.21.15.59:0/957582079 learned_addr learned my addr 172.21.15.59:0/957582079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.405+0000 7f3c02000640 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.405+0000 7f3c02000640 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 -- 172.21.15.59:0/957582079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 msgr2=0x5576326dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 -- 172.21.15.59:0/957582079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 msgr2=0x5576326dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 -- 172.21.15.59:0/957582079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557632419620 con 0x5576326d38c0 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.406+0000 7f3c02a00640 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bf402c7b0 tx=0x7f3bf4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b46090ad8dfedde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.407+0000 7f3bef400640 1 -- 172.21.15.59:0/957582079 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4044040 con 0x5576326d38c0 2024-04-02T17:30:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.407+0000 7f3bef400640 1 -- 172.21.15.59:0/957582079 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf402cb40 con 0x5576326d38c0 2024-04-02T17:30:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.407+0000 7f3bef400640 1 -- 172.21.15.59:0/957582079 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4004e50 con 0x5576326d38c0 2024-04-02T17:30:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/957582079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 msgr2=0x5576326dcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bf402c7b0 tx=0x7f3bf4000bc0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/957582079 shutdown_connections 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d8a30 0x5576326dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d38c0 0x5576326dcff0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/957582079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d2ed0 0x5576326dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.408+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/957582079 >> 172.21.15.59:0/957582079 conn(0x5576326ce440 msgr2=0x5576326ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.409+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/957582079 shutdown_connections 2024-04-02T17:30:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.409+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/957582079 wait complete. 2024-04-02T17:30:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.409+0000 7f3c0c5e0740 1 -- start start 2024-04-02T17:30:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 Processor -- start 2024-04-02T17:30:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d2ed0 0x5576326e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 0x5576326e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041b0dd0 con 0x5576326d38c0 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041b0700 con 0x5576326d8a30 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3c041ad940 con 0x5576326d2ed0 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.413+0000 7f3c0c5e0740 10 client.0 _ll_register_callbacks cb 0x7f3bf0013c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.414+0000 7f3c02000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.414+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 0x5576326e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.414+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 0x5576326e1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34080/0 (socket says 172.21.15.59:34080) 2024-04-02T17:30:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.414+0000 7f3c02000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34742/0 (socket says 172.21.15.59:34742) 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.414+0000 7f3c02000640 1 -- 172.21.15.59:0/1664152823 learned_addr learned my addr 172.21.15.59:0/1664152823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.415+0000 7f3c02000640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d2ed0 0x5576326e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.415+0000 7f3c02000640 1 -- 172.21.15.59:0/1664152823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d2ed0 msgr2=0x5576326e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.415+0000 7f3c02000640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d2ed0 0x5576326e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.415+0000 7f3c02000640 1 -- 172.21.15.59:0/1664152823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 msgr2=0x5576326e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.415+0000 7f3c02000640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 0x5576326e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.416+0000 7f3c02000640 1 -- 172.21.15.59:0/1664152823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576326e2d80 con 0x5576326d8a30 2024-04-02T17:30:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.416+0000 7f3c02000640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0013c40 tx=0x7f3bf0015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac6efd5aa3d49fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.416+0000 7f3becc00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:30:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.417+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0010aa0 con 0x5576326d8a30 2024-04-02T17:30:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.417+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f3bf001c040 con 0x5576326d8a30 2024-04-02T17:30:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.417+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf402cb40 con 0x5576326d8a30 2024-04-02T17:30:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.419+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3bf0010e50 con 0x5576326d8a30 2024-04-02T17:30:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.419+0000 7f3c0c5e0740 10 client.27296 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.419+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5576326e4090 con 0x5576326d8a30 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.419+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf402cb40 con 0x5576326d8a30 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf0003bb0 con 0x5576326d8a30 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 10 client.27296 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 10 client.27296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 20 client.27296 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55763274fcc0 0x5576327520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 4 client.27296 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:30:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.420+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576326462b0 con 0x55763274fcc0 2024-04-02T17:30:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.421+0000 7f3c02a00640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55763274fcc0 0x5576327520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.421+0000 7f3c02a00640 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55763274fcc0 0x5576327520e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b85a7ed39c9066 server_cookie=34dee21cd622ecea in_seq=0 out_seq=0 2024-04-02T17:30:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.421+0000 7f3becc00640 10 client.27296 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:33.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 191937 41.89 MB/sec execute 120 sec latency 101.302 ms 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3becc00640 1 -- 172.21.15.59:0/1664152823 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576326462b0 con 0x55763274fcc0 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3becc00640 10 client.27296 handle_command_reply: tid=0 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 client.27296 shutdown 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 msgr2=0x5576326e2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0013c40 tx=0x7f3bf0015040 comp rx=0 tx=0).stop 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 shutdown_connections 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55763274fcc0 0x5576327520e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576326d38c0 0x5576326e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576326d8a30 0x5576326e2790 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 --2- 172.21.15.59:0/1664152823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576326d2ed0 0x5576326e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.699+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 >> 172.21.15.59:0/1664152823 conn(0x5576326ce440 msgr2=0x5576326dbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.700+0000 7f3becc00640 0 client.27296 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.700+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 shutdown_connections 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.700+0000 7f3c0c5e0740 1 -- 172.21.15.59:0/1664152823 wait complete. 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.700+0000 7f3c0c5e0740 20 client.27296 trim_cache size 0 max 16384 2024-04-02T17:30:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:30:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:30:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:30:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:30:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.701+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 msgr2=0x7f3be80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.701+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 0x7f3be80a6190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0a52f0 tx=0x7f3bfc005970 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 msgr2=0x7f3c041a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00eb10 tx=0x7f3bfc00c2d0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 shutdown_connections 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c04109d10 0x7f3c041a5a00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c04109350 0x7f3c0419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3be80a3cd0 0x7f3be80a6190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 --2- 172.21.15.59:0/1704926793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c04108160 0x7f3c0419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.702+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 >> 172.21.15.59:0/1704926793 conn(0x7f3c04077110 msgr2=0x7f3c0410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.704+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 shutdown_connections 2024-04-02T17:30:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:33.705+0000 7f3c09a00640 1 -- 172.21.15.59:0/1704926793 wait complete. 2024-04-02T17:30:34.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:34 smithi154 ceph-mon[36479]: pgmap v914: 97 pgs: 97 active+clean; 821 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-04-02T17:30:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:34 smithi059 ceph-mon[31664]: pgmap v914: 97 pgs: 97 active+clean; 821 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-04-02T17:30:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:34 smithi164 ceph-mon[36499]: pgmap v914: 97 pgs: 97 active+clean; 821 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 649 op/s 2024-04-02T17:30:34.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 192767 41.88 MB/sec execute 121 sec latency 150.478 ms 2024-04-02T17:30:35.610 INFO:tasks.workunit.client.0.smithi059.stdout: 10 193700 41.98 MB/sec execute 122 sec latency 49.832 ms 2024-04-02T17:30:36.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:35 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:30:36.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:35 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:30:36.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:35 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:30:36.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 193973 41.79 MB/sec execute 123 sec latency 637.386 ms 2024-04-02T17:30:37.050 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:36 smithi059 ceph-mon[31664]: pgmap v915: 97 pgs: 97 active+clean; 829 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-02T17:30:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:36 smithi154 ceph-mon[36479]: pgmap v915: 97 pgs: 97 active+clean; 829 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-02T17:30:37.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:36 smithi164 ceph-mon[36499]: pgmap v915: 97 pgs: 97 active+clean; 829 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-02T17:30:37.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 194150 41.49 MB/sec execute 124 sec latency 994.700 ms 2024-04-02T17:30:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: pgmap v916: 97 pgs: 97 active+clean; 863 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:38 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: pgmap v916: 97 pgs: 97 active+clean; 863 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-04-02T17:30:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:38 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: pgmap v916: 97 pgs: 97 active+clean; 863 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.597 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.598 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.598 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:38 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:38.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 194551 41.31 MB/sec execute 125 sec latency 277.378 ms 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:30:39.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:39 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:30:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:30:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:30:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:30:39.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:39 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:30:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi154 to 174.1M 2024-04-02T17:30:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi154 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-04-02T17:30:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: Adjusting osd_memory_target on smithi164 to 174.1M 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: Unable to set osd_memory_target on smithi164 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:30:39.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:39 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:30:39.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 195168 41.25 MB/sec execute 126 sec latency 213.477 ms 2024-04-02T17:30:40.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:40 smithi154 ceph-mon[36479]: pgmap v917: 97 pgs: 97 active+clean; 868 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-02T17:30:40.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:40 smithi059 ceph-mon[31664]: pgmap v917: 97 pgs: 97 active+clean; 868 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-02T17:30:40.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:40 smithi164 ceph-mon[36499]: pgmap v917: 97 pgs: 97 active+clean; 868 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-02T17:30:40.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 195853 41.26 MB/sec execute 127 sec latency 314.786 ms 2024-04-02T17:30:41.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 196656 41.24 MB/sec execute 128 sec latency 47.241 ms 2024-04-02T17:30:42.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:42 smithi154 ceph-mon[36479]: pgmap v918: 97 pgs: 97 active+clean; 858 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:30:42.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:42 smithi059 ceph-mon[31664]: pgmap v918: 97 pgs: 97 active+clean; 858 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:30:42.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:42 smithi164 ceph-mon[36499]: pgmap v918: 97 pgs: 97 active+clean; 858 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2024-04-02T17:30:42.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 197401 41.25 MB/sec execute 129 sec latency 52.073 ms 2024-04-02T17:30:43.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 198225 41.21 MB/sec execute 130 sec latency 55.824 ms 2024-04-02T17:30:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:44 smithi154 ceph-mon[36479]: pgmap v919: 97 pgs: 97 active+clean; 822 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-04-02T17:30:44.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:44 smithi059 ceph-mon[31664]: pgmap v919: 97 pgs: 97 active+clean; 822 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-04-02T17:30:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:44 smithi164 ceph-mon[36499]: pgmap v919: 97 pgs: 97 active+clean; 822 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-04-02T17:30:44.612 INFO:tasks.workunit.client.0.smithi059.stdout: 10 199112 41.32 MB/sec execute 131 sec latency 76.248 ms 2024-04-02T17:30:45.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 199871 41.30 MB/sec execute 132 sec latency 53.828 ms 2024-04-02T17:30:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:46 smithi154 ceph-mon[36479]: pgmap v920: 97 pgs: 97 active+clean; 830 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-02T17:30:46.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:46 smithi059 ceph-mon[31664]: pgmap v920: 97 pgs: 97 active+clean; 830 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-02T17:30:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:46 smithi164 ceph-mon[36499]: pgmap v920: 97 pgs: 97 active+clean; 830 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-02T17:30:46.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 200662 41.34 MB/sec execute 133 sec latency 43.792 ms 2024-04-02T17:30:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 -- 172.21.15.59:0/2964946708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c073040 msgr2=0x7fc98c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 --2- 172.21.15.59:0/2964946708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c073040 0x7fc98c0734c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc978009900 tx=0x7fc97802bc80 comp rx=0 tx=0).stop 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 -- 172.21.15.59:0/2964946708 shutdown_connections 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 --2- 172.21.15.59:0/2964946708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 0x7fc98c112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 --2- 172.21.15.59:0/2964946708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c073040 0x7fc98c0734c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 --2- 172.21.15.59:0/2964946708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c1096e0 0x7fc98c072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.878+0000 7fc992000640 1 -- 172.21.15.59:0/2964946708 >> 172.21.15.59:0/2964946708 conn(0x7fc98c06e970 msgr2=0x7fc98c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.879+0000 7fc992000640 1 -- 172.21.15.59:0/2964946708 shutdown_connections 2024-04-02T17:30:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.879+0000 7fc992000640 1 -- 172.21.15.59:0/2964946708 wait complete. 2024-04-02T17:30:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.881+0000 7fc992000640 1 Processor -- start 2024-04-02T17:30:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.881+0000 7fc992000640 1 -- start start 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.881+0000 7fc992000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.881+0000 7fc992000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 0x7fc98c1a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc992000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 0x7fc98c1a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc992000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c115de0 con 0x7fc98c073040 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc992000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c115c60 con 0x7fc98c1096e0 2024-04-02T17:30:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc992000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c115f60 con 0x7fc98c10fe40 2024-04-02T17:30:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36836/0 (socket says 172.21.15.59:36836) 2024-04-02T17:30:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc990c00640 1 -- 172.21.15.59:0/2792170793 learned_addr learned my addr 172.21.15.59:0/2792170793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc98be00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 0x7fc98c1a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc991600640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 0x7fc98c1a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc98be00640 1 -- 172.21.15.59:0/2792170793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 msgr2=0x7fc98c1a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc98be00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 0x7fc98c1a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc98be00640 1 -- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 msgr2=0x7fc98c1a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.882+0000 7fc98be00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.883+0000 7fc98be00640 1 -- 172.21.15.59:0/2792170793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c1aa230 con 0x7fc98c1096e0 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.883+0000 7fc990c00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:30:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.883+0000 7fc98be00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 0x7fc98c1a2b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc978034aa0 tx=0x7fc97802c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.884+0000 7fc989600640 1 -- 172.21.15.59:0/2792170793 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978043020 con 0x7fc98c1096e0 2024-04-02T17:30:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.884+0000 7fc989600640 1 -- 172.21.15.59:0/2792170793 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97802cb30 con 0x7fc98c1096e0 2024-04-02T17:30:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.884+0000 7fc989600640 1 -- 172.21.15.59:0/2792170793 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978039040 con 0x7fc98c1096e0 2024-04-02T17:30:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.884+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98c1aa4c0 con 0x7fc98c1096e0 2024-04-02T17:30:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.884+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98c1aa980 con 0x7fc98c1096e0 2024-04-02T17:30:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.888+0000 7fc996264740 1 Processor -- start 2024-04-02T17:30:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.888+0000 7fc996264740 1 -- start start 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.888+0000 7fc996264740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc996264740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc996264740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbf50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc996264740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fcacb52d70 con 0x55fcacbedaf0 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc996264740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fcacafb2a0 con 0x55fcacbf2c60 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc996264740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fcacb4ed20 con 0x55fcacbed100 2024-04-02T17:30:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36846/0 (socket says 172.21.15.59:36846) 2024-04-02T17:30:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 -- 172.21.15.59:0/2644775250 learned_addr learned my addr 172.21.15.59:0/2644775250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 -- 172.21.15.59:0/2644775250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 msgr2=0x55fcacbed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 -- 172.21.15.59:0/2644775250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 msgr2=0x55fcacbf50f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbf50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.889+0000 7fc990c00640 1 -- 172.21.15.59:0/2644775250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcacb0f840 con 0x55fcacbedaf0 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.890+0000 7fc989600640 1 -- 172.21.15.59:0/2792170793 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc978046070 con 0x7fc98c1096e0 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.890+0000 7fc990c00640 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc980009e20 tx=0x7fc9800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb471d7d06a95104 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.890+0000 7fc97dc00640 1 -- 172.21.15.59:0/2644775250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980012650 con 0x55fcacbedaf0 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.890+0000 7fc97dc00640 1 -- 172.21.15.59:0/2644775250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc980012cd0 con 0x55fcacbedaf0 2024-04-02T17:30:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.890+0000 7fc97dc00640 1 -- 172.21.15.59:0/2644775250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000fd70 con 0x55fcacbedaf0 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.891+0000 7fc996264740 1 -- 172.21.15.59:0/2644775250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 msgr2=0x55fcacbf25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.891+0000 7fc996264740 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc980009e20 tx=0x7fc9800117b0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc996264740 1 -- 172.21.15.59:0/2644775250 shutdown_connections 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc996264740 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf25f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc996264740 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbf50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc996264740 1 --2- 172.21.15.59:0/2644775250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc996264740 1 -- 172.21.15.59:0/2644775250 >> 172.21.15.59:0/2644775250 conn(0x55fcacbe8670 msgr2=0x55fcacbe8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc989600640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 0x7fc9740a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.892+0000 7fc989600640 1 -- 172.21.15.59:0/2792170793 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc9780f19c0 con 0x7fc98c1096e0 2024-04-02T17:30:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.893+0000 7fc990c00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 0x7fc9740a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.893+0000 7fc996264740 1 -- 172.21.15.59:0/2644775250 shutdown_connections 2024-04-02T17:30:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.894+0000 7fc996264740 1 -- 172.21.15.59:0/2644775250 wait complete. 2024-04-02T17:30:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.894+0000 7fc990c00640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 0x7fc9740a6060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc980011cb0 tx=0x7fc980009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.894+0000 7fc996264740 1 -- start start 2024-04-02T17:30:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.894+0000 7fc996264740 1 Processor -- start 2024-04-02T17:30:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbf2c60 0x55fcacbfebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fcacbf5c10 con 0x55fcacbedaf0 2024-04-02T17:30:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fcacb0e660 con 0x55fcacbed100 2024-04-02T17:30:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc996264740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fcacbf5d90 con 0x55fcacbf2c60 2024-04-02T17:30:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.895+0000 7fc990c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.896+0000 7fc990c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34600/0 (socket says 172.21.15.59:34600) 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.896+0000 7fc990c00640 1 -- 172.21.15.59:0/1236133944 learned_addr learned my addr 172.21.15.59:0/1236133944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.896+0000 7fc98be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.896+0000 7fc996264740 10 client.0 _ll_register_callbacks cb 0x55fcacb0e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.896+0000 7fc991600640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbf2c60 0x55fcacbfebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 -- 172.21.15.59:0/1236133944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbf2c60 msgr2=0x55fcacbfebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbf2c60 0x55fcacbfebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 -- 172.21.15.59:0/1236133944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 msgr2=0x55fcacbf7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 -- 172.21.15.59:0/1236133944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcacb1de80 con 0x55fcacbed100 2024-04-02T17:30:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc990c00640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55fcacbee560 tx=0x7fc9800117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4323b966b4672 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc972000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:30:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.897+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980032020 con 0x55fcacbed100 2024-04-02T17:30:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.898+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc980012cd0 con 0x55fcacbed100 2024-04-02T17:30:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.898+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc980012450 con 0x55fcacbed100 2024-04-02T17:30:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.898+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc98003d050 con 0x55fcacbed100 2024-04-02T17:30:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.898+0000 7fc996264740 10 client.27305 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.898+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fcac867210 con 0x55fcacbed100 2024-04-02T17:30:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc980012cd0 con 0x55fcacbed100 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fc98002d070 con 0x55fcacbed100 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 10 client.27305 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 10 client.27305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 20 client.27305 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc031e0 0x55fcacc055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 4 client.27305 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.899+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcacb02cd0 con 0x55fcacc031e0 2024-04-02T17:30:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.900+0000 7fc98be00640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc031e0 0x55fcacc055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.901+0000 7fc98be00640 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc031e0 0x55fcacc055d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7f29a345127aaf server_cookie=dc3edb0a544522f1 in_seq=0 out_seq=0 2024-04-02T17:30:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.901+0000 7fc972000640 10 client.27305 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.902+0000 7fc972000640 1 -- 172.21.15.59:0/1236133944 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcacb02cd0 con 0x55fcacc031e0 2024-04-02T17:30:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.902+0000 7fc972000640 10 client.27305 handle_command_reply: tid=0 2024-04-02T17:30:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.902+0000 7fc996264740 1 client.27305 shutdown 2024-04-02T17:30:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.903+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 msgr2=0x55fcacbf7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.903+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55fcacbee560 tx=0x7fc9800117b0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 shutdown_connections 2024-04-02T17:30:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc031e0 0x55fcacc055d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbedaf0 0x55fcacbf7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbf2c60 0x55fcacbfebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 --2- 172.21.15.59:0/1236133944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbed100 0x55fcacbf7650 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 >> 172.21.15.59:0/1236133944 conn(0x55fcacbe8670 msgr2=0x55fcacbe7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc972000640 0 client.27305 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 shutdown_connections 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 1 -- 172.21.15.59:0/1236133944 wait complete. 2024-04-02T17:30:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.904+0000 7fc996264740 20 client.27305 trim_cache size 0 max 16384 2024-04-02T17:30:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 Processor -- start 2024-04-02T17:30:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 -- start start 2024-04-02T17:30:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbf7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 0x55fcacbf77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b3920 con 0x55fcacbf2c60 2024-04-02T17:30:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b5460 con 0x55fcacbedaf0 2024-04-02T17:30:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.930+0000 7fc996264740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b3ff0 con 0x55fcacbed100 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc991600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 0x55fcacbf77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:34606/0 (socket says 172.21.15.59:34606) 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 -- 172.21.15.59:0/3170295622 learned_addr learned my addr 172.21.15.59:0/3170295622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc990c00640 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbf7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 -- 172.21.15.59:0/3170295622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 msgr2=0x55fcacbf7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbf7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 -- 172.21.15.59:0/3170295622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 msgr2=0x55fcacbf77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 0x55fcacbf77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc98be00640 1 -- 172.21.15.59:0/3170295622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcac933620 con 0x55fcacbedaf0 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.931+0000 7fc991600640 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 0x55fcacbf77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.932+0000 7fc98be00640 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc9780bdb70 tx=0x7fc978009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1be53d1e5ebc647 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.932+0000 7fc97d200640 1 -- 172.21.15.59:0/3170295622 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780a8040 con 0x55fcacbedaf0 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.932+0000 7fc97d200640 1 -- 172.21.15.59:0/3170295622 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9780c6850 con 0x55fcacbedaf0 2024-04-02T17:30:46.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.932+0000 7fc97d200640 1 -- 172.21.15.59:0/3170295622 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978004220 con 0x55fcacbedaf0 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 -- 172.21.15.59:0/3170295622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 msgr2=0x55fcacbfeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc9780bdb70 tx=0x7fc978009260 comp rx=0 tx=0).stop 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 -- 172.21.15.59:0/3170295622 shutdown_connections 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbf2c60 0x55fcacbf77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbedaf0 0x55fcacbfeb90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 --2- 172.21.15.59:0/3170295622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbed100 0x55fcacbf7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 -- 172.21.15.59:0/3170295622 >> 172.21.15.59:0/3170295622 conn(0x55fcacbe8670 msgr2=0x55fcacbe8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 -- 172.21.15.59:0/3170295622 shutdown_connections 2024-04-02T17:30:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.933+0000 7fc996264740 1 -- 172.21.15.59:0/3170295622 wait complete. 2024-04-02T17:30:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 -- start start 2024-04-02T17:30:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 Processor -- start 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbedaf0 0x55fcacbfc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbfd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b4d90 con 0x55fcacbed100 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b46c0 con 0x55fcacbf2c60 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c1b1900 con 0x55fcacbedaf0 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.934+0000 7fc996264740 10 client.0 _ll_register_callbacks cb 0x7fc980005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36874/0 (socket says 172.21.15.59:36874) 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 -- 172.21.15.59:0/2966064345 learned_addr learned my addr 172.21.15.59:0/2966064345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc98be00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbedaf0 0x55fcacbfc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 -- 172.21.15.59:0/2966064345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbedaf0 msgr2=0x55fcacbfc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:30:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbedaf0 0x55fcacbfc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 -- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 msgr2=0x55fcacbfd130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:30:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbfd130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 -- 172.21.15.59:0/2966064345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcacbfd720 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc990c00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc15207fade2d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.935+0000 7fc972000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980046020 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc980004870 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9780c6850 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc980051050 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc996264740 10 client.17190 fetch_fsmap learned FSMap version 53 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.936+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fcacbfea30 con 0x55fcacbed100 2024-04-02T17:30:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9780c6850 con 0x55fcacbed100 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7fc98002d040 con 0x55fcacbed100 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 10 client.17190 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 20 client.17190 populate_metadata read hostname 'smithi059' 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc69ac0 0x55fcacc6bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 4 client.17190 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.937+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcacb602b0 con 0x55fcacc69ac0 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.938+0000 7fc98be00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc69ac0 0x55fcacc6bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:30:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.938+0000 7fc98be00640 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc69ac0 0x55fcacc6bee0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b63716c2dcb67bd server_cookie=3af5c9b5d1453063 in_seq=0 out_seq=0 2024-04-02T17:30:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:46.939+0000 7fc972000640 10 client.17190 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.162+0000 7fc972000640 1 -- 172.21.15.59:0/2966064345 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcacb602b0 con 0x55fcacc69ac0 2024-04-02T17:30:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.162+0000 7fc972000640 10 client.17190 handle_command_reply: tid=0 2024-04-02T17:30:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.162+0000 7fc996264740 1 client.17190 shutdown 2024-04-02T17:30:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 msgr2=0x55fcacbfbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980005f90 comp rx=0 tx=0).stop 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 shutdown_connections 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55fcacc69ac0 0x55fcacc6bee0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fcacbf2c60 0x55fcacbfd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcacbedaf0 0x55fcacbfc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 --2- 172.21.15.59:0/2966064345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fcacbed100 0x55fcacbfbbe0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc972000640 0 client.17190 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 >> 172.21.15.59:0/2966064345 conn(0x55fcacbe8670 msgr2=0x55fcacbf71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.163+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 shutdown_connections 2024-04-02T17:30:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.164+0000 7fc996264740 1 -- 172.21.15.59:0/2966064345 wait complete. 2024-04-02T17:30:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.164+0000 7fc996264740 20 client.17190 trim_cache size 0 max 16384 2024-04-02T17:30:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:30:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:30:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:30:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.166+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 msgr2=0x7fc9740a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.166+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 0x7fc9740a6060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc980011cb0 tx=0x7fc980009f70 comp rx=0 tx=0).stop 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.166+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 msgr2=0x7fc98c1a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.166+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 0x7fc98c1a2b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc978034aa0 tx=0x7fc97802c940 comp rx=0 tx=0).stop 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 shutdown_connections 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc98c10fe40 0x7fc98c1a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc98c1096e0 0x7fc98c1a2b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc9740a3ba0 0x7fc9740a6060 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 --2- 172.21.15.59:0/2792170793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc98c073040 0x7fc98c1a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:30:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 >> 172.21.15.59:0/2792170793 conn(0x7fc98c06e970 msgr2=0x7fc98c109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:30:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 shutdown_connections 2024-04-02T17:30:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:30:47.167+0000 7fc992000640 1 -- 172.21.15.59:0/2792170793 wait complete. 2024-04-02T17:30:47.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 201313 41.25 MB/sec execute 134 sec latency 145.628 ms 2024-04-02T17:30:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:48 smithi154 ceph-mon[36479]: pgmap v921: 97 pgs: 97 active+clean; 838 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 948 op/s 2024-04-02T17:30:48.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:48 smithi059 ceph-mon[31664]: pgmap v921: 97 pgs: 97 active+clean; 838 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 948 op/s 2024-04-02T17:30:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:48 smithi164 ceph-mon[36499]: pgmap v921: 97 pgs: 97 active+clean; 838 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 948 op/s 2024-04-02T17:30:48.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 202159 41.32 MB/sec execute 135 sec latency 35.180 ms 2024-04-02T17:30:49.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 203083 41.36 MB/sec execute 136 sec latency 34.942 ms 2024-04-02T17:30:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:50 smithi154 ceph-mon[36479]: pgmap v922: 97 pgs: 97 active+clean; 840 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 818 op/s 2024-04-02T17:30:50.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:50 smithi059 ceph-mon[31664]: pgmap v922: 97 pgs: 97 active+clean; 840 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 818 op/s 2024-04-02T17:30:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:50 smithi164 ceph-mon[36499]: pgmap v922: 97 pgs: 97 active+clean; 840 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 818 op/s 2024-04-02T17:30:50.613 INFO:tasks.workunit.client.0.smithi059.stdout: 10 203849 41.34 MB/sec execute 137 sec latency 76.971 ms 2024-04-02T17:30:51.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 204754 41.44 MB/sec execute 138 sec latency 44.323 ms 2024-04-02T17:30:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:52 smithi154 ceph-mon[36479]: pgmap v923: 97 pgs: 97 active+clean; 844 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 893 op/s 2024-04-02T17:30:52.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:52 smithi059 ceph-mon[31664]: pgmap v923: 97 pgs: 97 active+clean; 844 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 893 op/s 2024-04-02T17:30:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:52 smithi164 ceph-mon[36499]: pgmap v923: 97 pgs: 97 active+clean; 844 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 893 op/s 2024-04-02T17:30:52.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 205638 41.45 MB/sec execute 139 sec latency 44.195 ms 2024-04-02T17:30:53.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 206582 41.51 MB/sec execute 140 sec latency 37.459 ms 2024-04-02T17:30:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:54 smithi154 ceph-mon[36479]: pgmap v924: 97 pgs: 97 active+clean; 851 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 693 op/s 2024-04-02T17:30:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:54 smithi059 ceph-mon[31664]: pgmap v924: 97 pgs: 97 active+clean; 851 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 693 op/s 2024-04-02T17:30:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:54 smithi164 ceph-mon[36499]: pgmap v924: 97 pgs: 97 active+clean; 851 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 693 op/s 2024-04-02T17:30:54.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 207438 41.58 MB/sec execute 141 sec latency 41.170 ms 2024-04-02T17:30:55.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 208280 41.56 MB/sec execute 142 sec latency 70.418 ms 2024-04-02T17:30:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:56 smithi154 ceph-mon[36479]: pgmap v925: 97 pgs: 97 active+clean; 862 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-04-02T17:30:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:56 smithi059 ceph-mon[31664]: pgmap v925: 97 pgs: 97 active+clean; 862 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-04-02T17:30:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:56 smithi164 ceph-mon[36499]: pgmap v925: 97 pgs: 97 active+clean; 862 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.1 MiB/s wr, 699 op/s 2024-04-02T17:30:56.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 209098 41.61 MB/sec execute 143 sec latency 65.228 ms 2024-04-02T17:30:57.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 210054 41.67 MB/sec execute 144 sec latency 42.748 ms 2024-04-02T17:30:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:30:58 smithi154 ceph-mon[36479]: pgmap v926: 97 pgs: 97 active+clean; 866 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 661 op/s 2024-04-02T17:30:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:30:58 smithi059 ceph-mon[31664]: pgmap v926: 97 pgs: 97 active+clean; 866 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 661 op/s 2024-04-02T17:30:58.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:30:58 smithi164 ceph-mon[36499]: pgmap v926: 97 pgs: 97 active+clean; 866 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.7 MiB/s wr, 661 op/s 2024-04-02T17:30:58.614 INFO:tasks.workunit.client.0.smithi059.stdout: 10 210929 41.69 MB/sec execute 145 sec latency 40.415 ms 2024-04-02T17:30:59.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 211759 41.73 MB/sec execute 146 sec latency 34.416 ms 2024-04-02T17:31:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.357+0000 7f4d2e800640 1 -- 172.21.15.59:0/88348687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d2810de20 msgr2=0x7f4d2810e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.357+0000 7f4d2e800640 1 --2- 172.21.15.59:0/88348687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d2810de20 0x7f4d2810e2a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ca0 tx=0x7f4d2002c120 comp rx=0 tx=0).stop 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 -- 172.21.15.59:0/88348687 shutdown_connections 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 --2- 172.21.15.59:0/88348687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810b470 0x7f4d2810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 --2- 172.21.15.59:0/88348687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d2810de20 0x7f4d2810e2a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 --2- 172.21.15.59:0/88348687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d28108b00 0x7f4d2810af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 -- 172.21.15.59:0/88348687 >> 172.21.15.59:0/88348687 conn(0x7f4d2806eb40 msgr2=0x7f4d28070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 -- 172.21.15.59:0/88348687 shutdown_connections 2024-04-02T17:31:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.358+0000 7f4d2e800640 1 -- 172.21.15.59:0/88348687 wait complete. 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 Processor -- start 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 -- start start 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2810b470 0x7f4d281a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 0x7f4d281ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2811c5a0 con 0x7f4d2810de20 2024-04-02T17:31:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2811c420 con 0x7f4d28108b00 2024-04-02T17:31:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d2e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2811c720 con 0x7f4d2810b470 2024-04-02T17:31:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.359+0000 7f4d27400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38888/0 (socket says 172.21.15.59:38888) 2024-04-02T17:31:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 -- 172.21.15.59:0/1133159762 learned_addr learned my addr 172.21.15.59:0/1133159762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d26a00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2810b470 0x7f4d281a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 0x7f4d281ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 -- 172.21.15.59:0/1133159762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2810b470 msgr2=0x7f4d281a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2810b470 0x7f4d281a7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 -- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 msgr2=0x7f4d281ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 0x7f4d281ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 -- 172.21.15.59:0/1133159762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d281ae6e0 con 0x7f4d28108b00 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 0x7f4d281ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:31:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.360+0000 7f4d27400640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e9f0 tx=0x7f4d1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.361+0000 7f4d1fe00640 1 -- 172.21.15.59:0/1133159762 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800cd80 con 0x7f4d28108b00 2024-04-02T17:31:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.361+0000 7f4d1fe00640 1 -- 172.21.15.59:0/1133159762 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18010910 con 0x7f4d28108b00 2024-04-02T17:31:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.361+0000 7f4d1fe00640 1 -- 172.21.15.59:0/1133159762 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18018990 con 0x7f4d28108b00 2024-04-02T17:31:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.361+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d281ae9d0 con 0x7f4d28108b00 2024-04-02T17:31:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.361+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281aeed0 con 0x7f4d28108b00 2024-04-02T17:31:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.365+0000 7f4d31437740 1 Processor -- start 2024-04-02T17:31:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.365+0000 7f4d31437740 1 -- start start 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817ee33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817ee5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b817e43d70 con 0x55b817ede8c0 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b817dec2a0 con 0x55b817edded0 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d31437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b817e3fd20 con 0x55b817ee3a30 2024-04-02T17:31:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38902/0 (socket says 172.21.15.59:38902) 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 -- 172.21.15.59:0/1497545289 learned_addr learned my addr 172.21.15.59:0/1497545289 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d26a00640 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817ee5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 -- 172.21.15.59:0/1497545289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 msgr2=0x55b817ee5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817ee5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 -- 172.21.15.59:0/1497545289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 msgr2=0x55b817ee33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d1fe00640 1 -- 172.21.15.59:0/1133159762 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4d18010ac0 con 0x7f4d28108b00 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817ee33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d27e00640 1 -- 172.21.15.59:0/1497545289 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b817e00840 con 0x55b817edded0 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.366+0000 7f4d26a00640 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817ee5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.367+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4d20032610 tx=0x7f4d200323c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ff6c6aa65123a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.367+0000 7f4d1cc00640 1 -- 172.21.15.59:0/1497545289 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20039040 con 0x55b817edded0 2024-04-02T17:31:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.367+0000 7f4d1cc00640 1 -- 172.21.15.59:0/1497545289 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20032cb0 con 0x55b817edded0 2024-04-02T17:31:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.367+0000 7f4d1cc00640 1 -- 172.21.15.59:0/1497545289 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20034440 con 0x55b817edded0 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 -- 172.21.15.59:0/1497545289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 msgr2=0x55b817ede2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4d20032610 tx=0x7f4d200323c0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d1fe00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 0x7f4d0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 -- 172.21.15.59:0/1497545289 shutdown_connections 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817ee33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817ee5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d1fe00640 1 -- 172.21.15.59:0/1133159762 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f4d180d0800 con 0x7f4d28108b00 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 --2- 172.21.15.59:0/1497545289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ede2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 -- 172.21.15.59:0/1497545289 >> 172.21.15.59:0/1497545289 conn(0x55b817ed9440 msgr2=0x55b817ed9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 0x7f4d0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 -- 172.21.15.59:0/1497545289 shutdown_connections 2024-04-02T17:31:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.368+0000 7f4d31437740 1 -- 172.21.15.59:0/1497545289 wait complete. 2024-04-02T17:31:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.369+0000 7f4d31437740 1 -- start start 2024-04-02T17:31:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.369+0000 7f4d27e00640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 0x7f4d0c0a6210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d20032610 tx=0x7f4d2002c120 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.369+0000 7f4d31437740 1 Processor -- start 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.370+0000 7f4d31437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 0x55b817ee8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d31437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 0x55b817ee8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d31437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817eef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d27400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 0x55b817ee8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d27400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 0x55b817ee8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53036/0 (socket says 172.21.15.59:53036) 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 0x55b817ee8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.371+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 0x55b817ee8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38912/0 (socket says 172.21.15.59:38912) 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27400640 1 -- 172.21.15.59:0/3522250779 learned_addr learned my addr 172.21.15.59:0/3522250779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b817ee6980 con 0x55b817edded0 2024-04-02T17:31:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b817dff660 con 0x55b817ede8c0 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817eef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b817ee6b00 con 0x55b817ee3a30 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d31437740 10 client.0 _ll_register_callbacks cb 0x55b817dff050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 -- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 msgr2=0x55b817ee8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 0x55b817ee8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 -- 172.21.15.59:0/3522250779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 msgr2=0x55b817ee8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 0x55b817ee8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.372+0000 7f4d27e00640 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b817e0ee80 con 0x55b817ee3a30 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.373+0000 7f4d27e00640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817eef9a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4d20037040 tx=0x7f4d200323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6187ab609deb97ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.373+0000 7f4d11600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.373+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20059070 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.373+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f4d20034440 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.374+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d20032cb0 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.374+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f4d20063050 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.374+0000 7f4d31437740 10 client.27352 fetch_fsmap learned FSMap version 53 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.374+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b817b58210 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.374+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d20032cb0 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f4d20054070 con 0x55b817ee3a30 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 10 client.27352 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 10 client.27352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 20 client.27352 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817ef3e50 0x55b817ef6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 4 client.27352 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b817df3cd0 con 0x55b817ef3e50 2024-04-02T17:31:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.375+0000 7f4d27400640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817ef3e50 0x55b817ef6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.376+0000 7f4d27400640 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817ef3e50 0x55b817ef6240 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f66e96ebd33a8f3 server_cookie=1c3ce26c5177577b in_seq=0 out_seq=0 2024-04-02T17:31:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.376+0000 7f4d11600640 10 client.27352 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d11600640 1 -- 172.21.15.59:0/3522250779 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b817df3cd0 con 0x55b817ef3e50 2024-04-02T17:31:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d11600640 10 client.27352 handle_command_reply: tid=0 2024-04-02T17:31:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 client.27352 shutdown 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 msgr2=0x55b817eef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817eef9a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4d20037040 tx=0x7f4d200323c0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 shutdown_connections 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817ef3e50 0x55b817ef6240 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ee3a30 0x55b817eef9a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ede8c0 0x55b817ee8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 --2- 172.21.15.59:0/3522250779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817edded0 0x55b817ee8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 >> 172.21.15.59:0/3522250779 conn(0x55b817ed9440 msgr2=0x55b817ed8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d11600640 0 client.27352 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 shutdown_connections 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 1 -- 172.21.15.59:0/3522250779 wait complete. 2024-04-02T17:31:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.379+0000 7f4d31437740 20 client.27352 trim_cache size 0 max 16384 2024-04-02T17:31:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:00 smithi154 ceph-mon[36479]: pgmap v927: 97 pgs: 97 active+clean; 872 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 616 op/s 2024-04-02T17:31:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.410+0000 7f4d31437740 1 Processor -- start 2024-04-02T17:31:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.410+0000 7f4d31437740 1 -- start start 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ed7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 0x55b817ed82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b7f70 con 0x55b817ee3a30 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b9ab0 con 0x55b817edded0 2024-04-02T17:31:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d31437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b8640 con 0x55b817ede8c0 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41930/0 (socket says 172.21.15.59:41930) 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d26a00640 1 -- 172.21.15.59:0/824466096 learned_addr learned my addr 172.21.15.59:0/824466096 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.411+0000 7f4d27e00640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 0x55b817ed82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 -- 172.21.15.59:0/824466096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 msgr2=0x55b817ed7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d27400640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ed7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ed7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 -- 172.21.15.59:0/824466096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 msgr2=0x55b817ed82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 0x55b817ed82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 -- 172.21.15.59:0/824466096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b817c24620 con 0x55b817ede8c0 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d27400640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ed7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d27e00640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 0x55b817ed82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:31:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.412+0000 7f4d26a00640 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d30 tx=0x7f4d14005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec22a2c57b51e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.413+0000 7f4d13e00640 1 -- 172.21.15.59:0/824466096 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14009e20 con 0x55b817ede8c0 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.413+0000 7f4d13e00640 1 -- 172.21.15.59:0/824466096 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14021070 con 0x55b817ede8c0 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.413+0000 7f4d13e00640 1 -- 172.21.15.59:0/824466096 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1401c770 con 0x55b817ede8c0 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 -- 172.21.15.59:0/824466096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 msgr2=0x55b817eef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d30 tx=0x7f4d14005f90 comp rx=0 tx=0).stop 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 -- 172.21.15.59:0/824466096 shutdown_connections 2024-04-02T17:31:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ee3a30 0x55b817ed82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817ede8c0 0x55b817eef960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 --2- 172.21.15.59:0/824466096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817edded0 0x55b817ed7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.414+0000 7f4d31437740 1 -- 172.21.15.59:0/824466096 >> 172.21.15.59:0/824466096 conn(0x55b817ed9440 msgr2=0x55b817ef5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.415+0000 7f4d31437740 1 -- 172.21.15.59:0/824466096 shutdown_connections 2024-04-02T17:31:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.415+0000 7f4d31437740 1 -- 172.21.15.59:0/824466096 wait complete. 2024-04-02T17:31:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.416+0000 7f4d31437740 1 -- start start 2024-04-02T17:31:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.416+0000 7f4d31437740 1 Processor -- start 2024-04-02T17:31:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817edded0 0x55b817eec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 0x55b817f582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b93e0 con 0x55b817ede8c0 2024-04-02T17:31:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b8d10 con 0x55b817ee3a30 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281b5f50 con 0x55b817edded0 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.417+0000 7f4d31437740 10 client.0 _ll_register_callbacks cb 0x7f4d18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 0x55b817f582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53052/0 (socket says 172.21.15.59:53052) 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d26a00640 1 -- 172.21.15.59:0/2853865298 learned_addr learned my addr 172.21.15.59:0/2853865298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27400640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817edded0 0x55b817eec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 -- 172.21.15.59:0/2853865298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817edded0 msgr2=0x55b817eec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817edded0 0x55b817eec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 -- 172.21.15.59:0/2853865298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 msgr2=0x55b817eecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d27e00640 1 -- 172.21.15.59:0/2853865298 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b817eed3e0 con 0x55b817ee3a30 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.418+0000 7f4d26a00640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.419+0000 7f4d27e00640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 0x55b817f582d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4d20040720 tx=0x7f4d20032a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679d19e257a9d3da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.419+0000 7f4d11600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.419+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20059040 con 0x55b817ee3a30 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.420+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f4d2003f680 con 0x55b817ee3a30 2024-04-02T17:31:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.420+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14021070 con 0x55b817ee3a30 2024-04-02T17:31:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.420+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f4d20059040 con 0x55b817ee3a30 2024-04-02T17:31:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.420+0000 7f4d31437740 10 client.27323 fetch_fsmap learned FSMap version 53 2024-04-02T17:31:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.420+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b817f6a8b0 con 0x55b817ee3a30 2024-04-02T17:31:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.421+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2004f070 con 0x55b817ee3a30 2024-04-02T17:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 53) ==== 8354+0+0 (secure 0 0 0) 0x7f4d20054040 con 0x55b817ee3a30 2024-04-02T17:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 10 client.27323 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T17:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 10 client.27323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 20 client.27323 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817f6b1a0 0x55b817f6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 4 client.27323 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.422+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4d14021070 con 0x55b817f6b1a0 2024-04-02T17:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.423+0000 7f4d27400640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817f6b1a0 0x55b817f6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.426+0000 7f4d27400640 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817f6b1a0 0x55b817f6d5c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cac83a6913eef87 server_cookie=76af366af7803dea in_seq=0 out_seq=0 2024-04-02T17:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.426+0000 7f4d11600640 10 client.27323 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:00.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:00 smithi059 ceph-mon[31664]: pgmap v927: 97 pgs: 97 active+clean; 872 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 616 op/s 2024-04-02T17:31:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:00 smithi164 ceph-mon[36499]: pgmap v927: 97 pgs: 97 active+clean; 872 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.5 MiB/s wr, 616 op/s 2024-04-02T17:31:00.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 212418 41.67 MB/sec execute 147 sec latency 123.639 ms 2024-04-02T17:31:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.672+0000 7f4d11600640 1 -- 172.21.15.59:0/2853865298 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4d14021070 con 0x55b817f6b1a0 2024-04-02T17:31:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.672+0000 7f4d11600640 10 client.27323 handle_command_reply: tid=0 2024-04-02T17:31:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.672+0000 7f4d31437740 1 client.27323 shutdown 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 msgr2=0x55b817f582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 0x55b817f582d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4d20040720 tx=0x7f4d20032a80 comp rx=0 tx=0).stop 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 shutdown_connections 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55b817f6b1a0 0x55b817f6d5c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b817ede8c0 0x55b817eecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b817ee3a30 0x55b817f582d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 --2- 172.21.15.59:0/2853865298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b817edded0 0x55b817eec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 >> 172.21.15.59:0/2853865298 conn(0x55b817ed9440 msgr2=0x55b817edc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d11600640 0 client.27323 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.673+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 shutdown_connections 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.674+0000 7f4d31437740 1 -- 172.21.15.59:0/2853865298 wait complete. 2024-04-02T17:31:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.674+0000 7f4d31437740 20 client.27323 trim_cache size 0 max 16384 2024-04-02T17:31:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:31:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:31:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:31:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 msgr2=0x7f4d0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 0x7f4d0c0a6210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d20032610 tx=0x7f4d2002c120 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 msgr2=0x7f4d281a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e9f0 tx=0x7f4d1800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 shutdown_connections 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4d0c0a3d50 0x7f4d0c0a6210 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d2810de20 0x7f4d281ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d2810b470 0x7f4d281a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 --2- 172.21.15.59:0/1133159762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d28108b00 0x7f4d281a6b10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.675+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 >> 172.21.15.59:0/1133159762 conn(0x7f4d2806eb40 msgr2=0x7f4d2810abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.676+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 shutdown_connections 2024-04-02T17:31:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:00.676+0000 7f4d2e800640 1 -- 172.21.15.59:0/1133159762 wait complete. 2024-04-02T17:31:01.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 213238 41.72 MB/sec execute 148 sec latency 151.192 ms 2024-04-02T17:31:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:02 smithi154 ceph-mon[36479]: pgmap v928: 97 pgs: 97 active+clean; 882 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 710 op/s 2024-04-02T17:31:02.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:02 smithi059 ceph-mon[31664]: pgmap v928: 97 pgs: 97 active+clean; 882 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 710 op/s 2024-04-02T17:31:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:02 smithi164 ceph-mon[36499]: pgmap v928: 97 pgs: 97 active+clean; 882 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 710 op/s 2024-04-02T17:31:02.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 214065 41.70 MB/sec execute 149 sec latency 72.043 ms 2024-04-02T17:31:03.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 214727 41.69 MB/sec execute 150 sec latency 105.967 ms 2024-04-02T17:31:04.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:04 smithi154 ceph-mon[36479]: pgmap v929: 97 pgs: 97 active+clean; 893 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-04-02T17:31:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:04 smithi059 ceph-mon[31664]: pgmap v929: 97 pgs: 97 active+clean; 893 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-04-02T17:31:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:04 smithi164 ceph-mon[36499]: pgmap v929: 97 pgs: 97 active+clean; 893 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-04-02T17:31:04.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 215590 41.69 MB/sec execute 151 sec latency 36.672 ms 2024-04-02T17:31:05.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 216272 41.68 MB/sec execute 152 sec latency 178.677 ms 2024-04-02T17:31:06.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:06 smithi154 ceph-mon[36479]: pgmap v930: 97 pgs: 97 active+clean; 891 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 708 op/s 2024-04-02T17:31:06.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:06 smithi059 ceph-mon[31664]: pgmap v930: 97 pgs: 97 active+clean; 891 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 708 op/s 2024-04-02T17:31:06.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:06 smithi164 ceph-mon[36499]: pgmap v930: 97 pgs: 97 active+clean; 891 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 708 op/s 2024-04-02T17:31:06.615 INFO:tasks.workunit.client.0.smithi059.stdout: 10 217118 41.74 MB/sec execute 153 sec latency 49.464 ms 2024-04-02T17:31:07.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 217568 41.58 MB/sec execute 154 sec latency 279.164 ms 2024-04-02T17:31:08.374 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:08 smithi164 ceph-mon[36499]: pgmap v931: 97 pgs: 97 active+clean; 896 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-04-02T17:31:08.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:08 smithi154 ceph-mon[36479]: pgmap v931: 97 pgs: 97 active+clean; 896 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-04-02T17:31:08.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:08 smithi059 ceph-mon[31664]: pgmap v931: 97 pgs: 97 active+clean; 896 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 643 op/s 2024-04-02T17:31:08.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 217939 41.47 MB/sec execute 155 sec latency 341.277 ms 2024-04-02T17:31:09.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 218622 41.44 MB/sec execute 156 sec latency 65.628 ms 2024-04-02T17:31:10.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:10 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:10.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:10 smithi059 ceph-mon[31664]: pgmap v932: 97 pgs: 97 active+clean; 906 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-02T17:31:10.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:10 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:10.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:10 smithi164 ceph-mon[36499]: pgmap v932: 97 pgs: 97 active+clean; 906 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-02T17:31:10.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 219585 41.48 MB/sec execute 157 sec latency 39.836 ms 2024-04-02T17:31:10.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:10 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:10.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:10 smithi154 ceph-mon[36479]: pgmap v932: 97 pgs: 97 active+clean; 906 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-02T17:31:11.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 220519 41.59 MB/sec execute 158 sec latency 48.424 ms 2024-04-02T17:31:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:12 smithi154 ceph-mon[36479]: pgmap v933: 97 pgs: 97 active+clean; 906 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-04-02T17:31:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:12 smithi059 ceph-mon[31664]: pgmap v933: 97 pgs: 97 active+clean; 906 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-04-02T17:31:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:12 smithi164 ceph-mon[36499]: pgmap v933: 97 pgs: 97 active+clean; 906 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-04-02T17:31:12.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 221198 41.50 MB/sec execute 159 sec latency 84.219 ms 2024-04-02T17:31:13.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 221692 41.42 MB/sec execute 160 sec latency 261.776 ms 2024-04-02T17:31:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.900+0000 7fe96fe00640 1 -- 172.21.15.59:0/3718393832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970101970 msgr2=0x7fe970101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.900+0000 7fe96fe00640 1 --2- 172.21.15.59:0/3718393832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970101970 0x7fe970101d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe960009900 tx=0x7fe96002bc60 comp rx=0 tx=0).stop 2024-04-02T17:31:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.902+0000 7fe96fe00640 1 -- 172.21.15.59:0/3718393832 shutdown_connections 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.902+0000 7fe96fe00640 1 --2- 172.21.15.59:0/3718393832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe970102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.902+0000 7fe96fe00640 1 --2- 172.21.15.59:0/3718393832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970103520 0x7fe970109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.902+0000 7fe96fe00640 1 --2- 172.21.15.59:0/3718393832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970101970 0x7fe970101d90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.902+0000 7fe96fe00640 1 -- 172.21.15.59:0/3718393832 >> 172.21.15.59:0/3718393832 conn(0x7fe9700fcee0 msgr2=0x7fe9700ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.903+0000 7fe96fe00640 1 -- 172.21.15.59:0/3718393832 shutdown_connections 2024-04-02T17:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.903+0000 7fe96fe00640 1 -- 172.21.15.59:0/3718393832 wait complete. 2024-04-02T17:31:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.904+0000 7fe96fe00640 1 Processor -- start 2024-04-02T17:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.907+0000 7fe96fe00640 1 -- start start 2024-04-02T17:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe97019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970103520 0x7fe9701a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d610 con 0x7fe970102b60 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d490 con 0x7fe970103520 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.912+0000 7fe96fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010d790 con 0x7fe970101970 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe97019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52328/0 (socket says 172.21.15.59:52328) 2024-04-02T17:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96ea00640 1 -- 172.21.15.59:0/926728705 learned_addr learned my addr 172.21.15.59:0/926728705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96ea00640 1 -- 172.21.15.59:0/926728705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970103520 msgr2=0x7fe9701a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.913+0000 7fe96ea00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970103520 0x7fe9701a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.914+0000 7fe96ea00640 1 -- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 msgr2=0x7fe97019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.914+0000 7fe96ea00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe97019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.914+0000 7fe96ea00640 1 -- 172.21.15.59:0/926728705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701a5ea0 con 0x7fe970101970 2024-04-02T17:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.914+0000 7fe96e000640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe97019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.914+0000 7fe96ea00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe96002c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.915+0000 7fe974a00640 1 -- 172.21.15.59:0/926728705 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960041070 con 0x7fe970101970 2024-04-02T17:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.915+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701a6130 con 0x7fe970101970 2024-04-02T17:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.915+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701a6670 con 0x7fe970101970 2024-04-02T17:31:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.915+0000 7fe974a00640 1 -- 172.21.15.59:0/926728705 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960005310 con 0x7fe970101970 2024-04-02T17:31:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.915+0000 7fe974a00640 1 -- 172.21.15.59:0/926728705 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960003d60 con 0x7fe970101970 2024-04-02T17:31:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.926+0000 7fe974a00640 1 -- 172.21.15.59:0/926728705 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe960044070 con 0x7fe970101970 2024-04-02T17:31:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.931+0000 7fe978bbd740 1 Processor -- start 2024-04-02T17:31:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.931+0000 7fe978bbd740 1 -- start start 2024-04-02T17:31:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.931+0000 7fe974a00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 0x7fe9540a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.932+0000 7fe974a00640 1 -- 172.21.15.59:0/926728705 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fe9600f1fc0 con 0x7fe970101970 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.933+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fbaf0 0x55a1a63005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a63030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe978bbd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6260d70 con 0x55a1a62fbaf0 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe978bbd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a1a62092a0 con 0x55a1a6300c60 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe978bbd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a1a625cd20 con 0x55a1a62fb100 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52340/0 (socket says 172.21.15.59:52340) 2024-04-02T17:31:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe96ea00640 1 -- 172.21.15.59:0/1980060938 learned_addr learned my addr 172.21.15.59:0/1980060938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.934+0000 7fe96f400640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 0x7fe9540a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 -- 172.21.15.59:0/1980060938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 msgr2=0x55a1a63030f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a63030f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 -- 172.21.15.59:0/1980060938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fbaf0 msgr2=0x55a1a63005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fbaf0 0x55a1a63005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 -- 172.21.15.59:0/1980060938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a621d840 con 0x55a1a62fb100 2024-04-02T17:31:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9600ba5b0 tx=0x7fe96002bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a71c1123ad79b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe95be00640 1 -- 172.21.15.59:0/1980060938 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96004b040 con 0x55a1a62fb100 2024-04-02T17:31:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.935+0000 7fe95be00640 1 -- 172.21.15.59:0/1980060938 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9600be8e0 con 0x55a1a62fb100 2024-04-02T17:31:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.936+0000 7fe95be00640 1 -- 172.21.15.59:0/1980060938 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600054c0 con 0x55a1a62fb100 2024-04-02T17:31:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.938+0000 7fe978bbd740 1 -- 172.21.15.59:0/1980060938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 msgr2=0x55a1a62fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.938+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9600ba5b0 tx=0x7fe96002bc60 comp rx=0 tx=0).stop 2024-04-02T17:31:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.939+0000 7fe978bbd740 1 -- 172.21.15.59:0/1980060938 shutdown_connections 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.939+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fbaf0 0x55a1a63005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.939+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a63030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.939+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1980060938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a62fb520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.939+0000 7fe978bbd740 1 -- 172.21.15.59:0/1980060938 >> 172.21.15.59:0/1980060938 conn(0x55a1a62f6670 msgr2=0x55a1a62f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.940+0000 7fe96f400640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 0x7fe9540a61a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe968005c40 tx=0x7fe968005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.941+0000 7fe978bbd740 1 -- 172.21.15.59:0/1980060938 shutdown_connections 2024-04-02T17:31:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.941+0000 7fe978bbd740 1 -- 172.21.15.59:0/1980060938 wait complete. 2024-04-02T17:31:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.941+0000 7fe978bbd740 1 -- start start 2024-04-02T17:31:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.942+0000 7fe978bbd740 1 Processor -- start 2024-04-02T17:31:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.942+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.942+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.942+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.943+0000 7fe96e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.943+0000 7fe96e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52354/0 (socket says 172.21.15.59:52354) 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.943+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe96ea00640 1 -- 172.21.15.59:0/1003382733 learned_addr learned my addr 172.21.15.59:0/1003382733 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a1a63039d0 con 0x55a1a62fb100 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a1a621c660 con 0x55a1a6300c60 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6303b50 con 0x55a1a62fbaf0 2024-04-02T17:31:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe96f400640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.944+0000 7fe978bbd740 10 client.0 _ll_register_callbacks cb 0x55a1a621c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96e000640 1 -- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 msgr2=0x55a1a631d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96e000640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96e000640 1 -- 172.21.15.59:0/1003382733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 msgr2=0x55a1a6316260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96e000640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96e000640 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a622be80 con 0x55a1a62fbaf0 2024-04-02T17:31:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.945+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.946+0000 7fe96e000640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55a1a62f75e0 tx=0x7fe95c010a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07c469ffc12ba69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.946+0000 7fe958c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.946+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c004230 con 0x55a1a62fbaf0 2024-04-02T17:31:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.947+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fe95c0043d0 con 0x55a1a62fbaf0 2024-04-02T17:31:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.948+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9600be8e0 con 0x55a1a62fbaf0 2024-04-02T17:31:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.948+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe95c012720 con 0x55a1a62fbaf0 2024-04-02T17:31:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.948+0000 7fe978bbd740 10 client.27370 fetch_fsmap learned FSMap version 54 2024-04-02T17:31:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.948+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a1a5f75210 con 0x55a1a62fbaf0 2024-04-02T17:31:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.949+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9600be8e0 con 0x55a1a62fbaf0 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.949+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 54) ==== 8358+0+0 (secure 0 0 0) 0x7fe95c00f040 con 0x55a1a62fbaf0 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.949+0000 7fe978bbd740 10 client.27370 fetch_fsmap finished waiting for FSMap version 54 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.949+0000 7fe978bbd740 10 client.27370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.949+0000 7fe978bbd740 20 client.27370 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.950+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6322110 0x55a1a6324590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.950+0000 7fe978bbd740 4 client.27370 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:31:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.950+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a6210cd0 con 0x55a1a6322110 2024-04-02T17:31:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.951+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6322110 0x55a1a6324590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.951+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6322110 0x55a1a6324590 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a350794d636e39c server_cookie=650aec65cc21f5ba in_seq=0 out_seq=0 2024-04-02T17:31:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.951+0000 7fe958c00640 10 client.27370 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.956+0000 7fe958c00640 1 -- 172.21.15.59:0/1003382733 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1a6210cd0 con 0x55a1a6322110 2024-04-02T17:31:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.956+0000 7fe958c00640 10 client.27370 handle_command_reply: tid=0 2024-04-02T17:31:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.956+0000 7fe978bbd740 1 client.27370 shutdown 2024-04-02T17:31:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 msgr2=0x55a1a63167a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55a1a62f75e0 tx=0x7fe95c010a80 comp rx=0 tx=0).stop 2024-04-02T17:31:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 shutdown_connections 2024-04-02T17:31:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6322110 0x55a1a6324590 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fbaf0 0x55a1a63167a0 secure :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55a1a62f75e0 tx=0x7fe95c010a80 comp rx=0 tx=0).stop 2024-04-02T17:31:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1003382733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a6316260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 >> 172.21.15.59:0/1003382733 conn(0x55a1a62f6670 msgr2=0x55a1a62f5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.960+0000 7fe958c00640 0 client.27370 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.962+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 shutdown_connections 2024-04-02T17:31:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.962+0000 7fe978bbd740 1 -- 172.21.15.59:0/1003382733 wait complete. 2024-04-02T17:31:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:13.962+0000 7fe978bbd740 20 client.27370 trim_cache size 0 max 16384 2024-04-02T17:31:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.005+0000 7fe978bbd740 1 Processor -- start 2024-04-02T17:31:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.005+0000 7fe978bbd740 1 -- start start 2024-04-02T17:31:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a62fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a631d7a0 0x55a1a62f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000e630 con 0x55a1a62fb100 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe950010170 con 0x55a1a6300c60 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.007+0000 7fe978bbd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000ed00 con 0x55a1a631d7a0 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.008+0000 7fe96e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.008+0000 7fe96e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38546/0 (socket says 172.21.15.59:38546) 2024-04-02T17:31:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.008+0000 7fe96e000640 1 -- 172.21.15.59:0/3596870807 learned_addr learned my addr 172.21.15.59:0/3596870807 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.008+0000 7fe96ea00640 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a62fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 -- 172.21.15.59:0/3596870807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a631d7a0 msgr2=0x55a1a62f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a631d7a0 0x55a1a62f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 -- 172.21.15.59:0/3596870807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 msgr2=0x55a1a62fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 -- 172.21.15.59:0/3596870807 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a6041620 con 0x55a1a62fb100 2024-04-02T17:31:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.009+0000 7fe96ea00640 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a62fcf10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe96002c810 tx=0x7fe96002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaafcda0cd0d4e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.010+0000 7fe96e000640 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.010+0000 7fe95b400640 1 -- 172.21.15.59:0/3596870807 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96004b040 con 0x55a1a62fb100 2024-04-02T17:31:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.010+0000 7fe95b400640 1 -- 172.21.15.59:0/3596870807 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9600c6e50 con 0x55a1a62fb100 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.012+0000 7fe95b400640 1 -- 172.21.15.59:0/3596870807 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600c2120 con 0x55a1a62fb100 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 -- 172.21.15.59:0/3596870807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 msgr2=0x55a1a62fcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a62fcf10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe96002c810 tx=0x7fe96002c8e0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 -- 172.21.15.59:0/3596870807 shutdown_connections 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a631d7a0 0x55a1a62f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a62fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 --2- 172.21.15.59:0/3596870807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a62fb100 0x55a1a62fcf10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.013+0000 7fe978bbd740 1 -- 172.21.15.59:0/3596870807 >> 172.21.15.59:0/3596870807 conn(0x55a1a62f6670 msgr2=0x55a1a62f6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.014+0000 7fe978bbd740 1 -- 172.21.15.59:0/3596870807 shutdown_connections 2024-04-02T17:31:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.014+0000 7fe978bbd740 1 -- 172.21.15.59:0/3596870807 wait complete. 2024-04-02T17:31:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.014+0000 7fe978bbd740 1 -- start start 2024-04-02T17:31:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.014+0000 7fe978bbd740 1 Processor -- start 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a631a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a631d7a0 0x55a1a631bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000faa0 con 0x55a1a631d7a0 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000f3d0 con 0x55a1a6300c60 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe95000c610 con 0x55a1a62fb100 2024-04-02T17:31:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.017+0000 7fe978bbd740 10 client.0 _ll_register_callbacks cb 0x7fe95c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.018+0000 7fe96e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.018+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a631a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.018+0000 7fe96e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38552/0 (socket says 172.21.15.59:38552) 2024-04-02T17:31:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.018+0000 7fe96e000640 1 -- 172.21.15.59:0/1563168217 learned_addr learned my addr 172.21.15.59:0/1563168217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.018+0000 7fe96f400640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a631d7a0 0x55a1a631bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 -- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 msgr2=0x55a1a631af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 -- 172.21.15.59:0/1563168217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a631d7a0 msgr2=0x55a1a631bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a631d7a0 0x55a1a631bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 -- 172.21.15.59:0/1563168217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a631c530 con 0x55a1a62fb100 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe96ea00640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a631a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55a1a6304ad0 tx=0x7fe960005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1567bd08477a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.019+0000 7fe958c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.020+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600dc020 con 0x55a1a62fb100 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.020+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fe9600c2320 con 0x55a1a62fb100 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.020+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1a626e2b0 con 0x55a1a62fb100 2024-04-02T17:31:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.020+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe96004b040 con 0x55a1a62fb100 2024-04-02T17:31:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.021+0000 7fe978bbd740 10 client.27373 fetch_fsmap learned FSMap version 54 2024-04-02T17:31:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.021+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a1a6388790 con 0x55a1a62fb100 2024-04-02T17:31:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a1a626e2b0 con 0x55a1a62fb100 2024-04-02T17:31:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 54) ==== 8358+0+0 (secure 0 0 0) 0x7fe9600bd630 con 0x55a1a62fb100 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 10 client.27373 fetch_fsmap finished waiting for FSMap version 54 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 20 client.27373 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6388db0 0x55a1a638b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 4 client.27373 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:31:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.022+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1a6268600 con 0x55a1a6388db0 2024-04-02T17:31:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.031+0000 7fe96e000640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6388db0 0x55a1a638b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.034+0000 7fe96e000640 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6388db0 0x55a1a638b1a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b567eb38a0573b server_cookie=7c973865659405f in_seq=0 out_seq=0 2024-04-02T17:31:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:14.035+0000 7fe958c00640 10 client.27373 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:14 smithi154 ceph-mon[36479]: pgmap v934: 97 pgs: 97 active+clean; 909 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 762 op/s 2024-04-02T17:31:14.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:14 smithi059 ceph-mon[31664]: pgmap v934: 97 pgs: 97 active+clean; 909 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 762 op/s 2024-04-02T17:31:14.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:14 smithi164 ceph-mon[36499]: pgmap v934: 97 pgs: 97 active+clean; 909 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 762 op/s 2024-04-02T17:31:14.616 INFO:tasks.workunit.client.0.smithi059.stdout: 10 221708 41.17 MB/sec execute 161 sec latency 1200.687 ms 2024-04-02T17:31:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.136+0000 7fe958c00640 1 -- 172.21.15.59:0/1563168217 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1a6268600 con 0x55a1a6388db0 2024-04-02T17:31:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.137+0000 7fe958c00640 10 client.27373 handle_command_reply: tid=0 2024-04-02T17:31:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.137+0000 7fe978bbd740 1 client.27373 shutdown 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.137+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 msgr2=0x55a1a631a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.137+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a631a9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55a1a6304ad0 tx=0x7fe960005c90 comp rx=0 tx=0).stop 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 shutdown_connections 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a1a6388db0 0x55a1a638b1a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a1a631d7a0 0x55a1a631bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1a6300c60 0x55a1a631af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 --2- 172.21.15.59:0/1563168217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a1a62fb100 0x55a1a631a9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 >> 172.21.15.59:0/1563168217 conn(0x55a1a62f6670 msgr2=0x55a1a6316260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe958c00640 0 client.27373 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 shutdown_connections 2024-04-02T17:31:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.138+0000 7fe978bbd740 1 -- 172.21.15.59:0/1563168217 wait complete. 2024-04-02T17:31:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.139+0000 7fe978bbd740 20 client.27373 trim_cache size 0 max 16384 2024-04-02T17:31:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.139+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 msgr2=0x7fe9540a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.139+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 0x7fe9540a61a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe968005c40 tx=0x7fe968005bd0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.139+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 msgr2=0x7fe97019e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.139+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe96002c6d0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 shutdown_connections 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe9540a3ce0 0x7fe9540a61a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe970102b60 0x7fe97019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970103520 0x7fe9701a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 --2- 172.21.15.59:0/926728705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe970101970 0x7fe97019e280 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 >> 172.21.15.59:0/926728705 conn(0x7fe9700fcee0 msgr2=0x7fe970103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 shutdown_connections 2024-04-02T17:31:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:15.142+0000 7fe96fe00640 1 -- 172.21.15.59:0/926728705 wait complete. 2024-04-02T17:31:15.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 221989 41.03 MB/sec execute 162 sec latency 573.441 ms 2024-04-02T17:31:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:16 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:16 smithi154 ceph-mon[36479]: pgmap v935: 97 pgs: 97 active+clean; 978 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 821 op/s 2024-04-02T17:31:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:16 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:16 smithi059 ceph-mon[31664]: pgmap v935: 97 pgs: 97 active+clean; 978 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 821 op/s 2024-04-02T17:31:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:16 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:31:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:16 smithi164 ceph-mon[36499]: pgmap v935: 97 pgs: 97 active+clean; 978 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 821 op/s 2024-04-02T17:31:16.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 222448 40.86 MB/sec execute 163 sec latency 103.949 ms 2024-04-02T17:31:17.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 223164 40.86 MB/sec execute 164 sec latency 57.686 ms 2024-04-02T17:31:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:18 smithi154 ceph-mon[36479]: pgmap v936: 97 pgs: 97 active+clean; 982 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 948 op/s 2024-04-02T17:31:18.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:18 smithi059 ceph-mon[31664]: pgmap v936: 97 pgs: 97 active+clean; 982 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 948 op/s 2024-04-02T17:31:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:18 smithi164 ceph-mon[36499]: pgmap v936: 97 pgs: 97 active+clean; 982 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 948 op/s 2024-04-02T17:31:18.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 223996 40.94 MB/sec execute 165 sec latency 51.215 ms 2024-04-02T17:31:19.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 224710 40.91 MB/sec execute 166 sec latency 47.014 ms 2024-04-02T17:31:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:20 smithi154 ceph-mon[36479]: pgmap v937: 97 pgs: 97 active+clean; 962 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-04-02T17:31:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:20 smithi059 ceph-mon[31664]: pgmap v937: 97 pgs: 97 active+clean; 962 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-04-02T17:31:20.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:20 smithi164 ceph-mon[36499]: pgmap v937: 97 pgs: 97 active+clean; 962 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-04-02T17:31:20.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 225394 40.86 MB/sec execute 167 sec latency 40.065 ms 2024-04-02T17:31:21.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 226277 40.92 MB/sec execute 168 sec latency 56.305 ms 2024-04-02T17:31:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:22 smithi154 ceph-mon[36479]: pgmap v938: 97 pgs: 97 active+clean; 895 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-04-02T17:31:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:22 smithi059 ceph-mon[31664]: pgmap v938: 97 pgs: 97 active+clean; 895 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-04-02T17:31:22.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:22 smithi164 ceph-mon[36499]: pgmap v938: 97 pgs: 97 active+clean; 895 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-04-02T17:31:22.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 227160 40.96 MB/sec execute 169 sec latency 48.973 ms 2024-04-02T17:31:23.617 INFO:tasks.workunit.client.0.smithi059.stdout: 10 227891 40.99 MB/sec execute 170 sec latency 52.242 ms 2024-04-02T17:31:24.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:24 smithi154 ceph-mon[36479]: pgmap v939: 97 pgs: 97 active+clean; 906 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-04-02T17:31:24.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:24 smithi059 ceph-mon[31664]: pgmap v939: 97 pgs: 97 active+clean; 906 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-04-02T17:31:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:24 smithi164 ceph-mon[36499]: pgmap v939: 97 pgs: 97 active+clean; 906 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-04-02T17:31:24.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 228654 40.94 MB/sec execute 171 sec latency 62.475 ms 2024-04-02T17:31:25.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 229630 41.02 MB/sec execute 172 sec latency 36.845 ms 2024-04-02T17:31:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:26 smithi154 ceph-mon[36479]: pgmap v940: 97 pgs: 97 active+clean; 914 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-04-02T17:31:26.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:26 smithi059 ceph-mon[31664]: pgmap v940: 97 pgs: 97 active+clean; 914 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-04-02T17:31:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:26 smithi164 ceph-mon[36499]: pgmap v940: 97 pgs: 97 active+clean; 914 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-04-02T17:31:26.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 230483 41.05 MB/sec execute 173 sec latency 48.654 ms 2024-04-02T17:31:27.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 231143 41.03 MB/sec execute 174 sec latency 53.098 ms 2024-04-02T17:31:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 -- 172.21.15.59:0/3115313285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109d10 msgr2=0x7ff4901105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 --2- 172.21.15.59:0/3115313285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109d10 0x7ff4901105a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ca0 tx=0x7ff48802c100 comp rx=0 tx=0).stop 2024-04-02T17:31:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 -- 172.21.15.59:0/3115313285 shutdown_connections 2024-04-02T17:31:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 --2- 172.21.15.59:0/3115313285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109350 0x7ff4901097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 --2- 172.21.15.59:0/3115313285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109d10 0x7ff4901105a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 --2- 172.21.15.59:0/3115313285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff490108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.355+0000 7ff496e00640 1 -- 172.21.15.59:0/3115313285 >> 172.21.15.59:0/3115313285 conn(0x7ff490077110 msgr2=0x7ff490079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.356+0000 7ff496e00640 1 -- 172.21.15.59:0/3115313285 shutdown_connections 2024-04-02T17:31:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.356+0000 7ff496e00640 1 -- 172.21.15.59:0/3115313285 wait complete. 2024-04-02T17:31:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.357+0000 7ff496e00640 1 Processor -- start 2024-04-02T17:31:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.358+0000 7ff496e00640 1 -- start start 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109350 0x7ff49019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109d10 0x7ff4901a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff490113dc0 con 0x7ff490109d10 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff490113c40 con 0x7ff490108160 2024-04-02T17:31:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff496e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff490113f40 con 0x7ff490109350 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55074/0 (socket says 172.21.15.59:55074) 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff48fe00640 1 -- 172.21.15.59:0/2446486029 learned_addr learned my addr 172.21.15.59:0/2446486029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff494e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109d10 0x7ff4901a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.359+0000 7ff48f400640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109350 0x7ff49019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 -- 172.21.15.59:0/2446486029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109350 msgr2=0x7ff49019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109350 0x7ff49019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 -- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109d10 msgr2=0x7ff4901a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109d10 0x7ff4901a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 -- 172.21.15.59:0/2446486029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a5f90 con 0x7ff490108160 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.360+0000 7ff48fe00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e9c0 tx=0x7ff48000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.361+0000 7ff48cc00640 1 -- 172.21.15.59:0/2446486029 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000cd80 con 0x7ff490108160 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.361+0000 7ff48cc00640 1 -- 172.21.15.59:0/2446486029 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4800035f0 con 0x7ff490108160 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.361+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a6280 con 0x7ff490108160 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.361+0000 7ff48cc00640 1 -- 172.21.15.59:0/2446486029 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480010690 con 0x7ff490108160 2024-04-02T17:31:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.361+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a6820 con 0x7ff490108160 2024-04-02T17:31:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.367+0000 7ff499b8f740 1 Processor -- start 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 -- start start 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2e8c0 0x55dd9af333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af35ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ae93d70 con 0x55dd9af33a30 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ae3c2a0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.368+0000 7ff499b8f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ae8fd20 con 0x55dd9af2ded0 2024-04-02T17:31:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.369+0000 7ff494e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.369+0000 7ff48cc00640 1 -- 172.21.15.59:0/2446486029 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff4800037a0 con 0x7ff490108160 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.369+0000 7ff494e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50314/0 (socket says 172.21.15.59:50314) 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.369+0000 7ff494e00640 1 -- 172.21.15.59:0/3629697705 learned_addr learned my addr 172.21.15.59:0/3629697705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff48f400640 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af35ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 -- 172.21.15.59:0/3629697705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2e8c0 msgr2=0x55dd9af333c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2e8c0 0x55dd9af333c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 -- 172.21.15.59:0/3629697705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 msgr2=0x55dd9af35ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af35ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 -- 172.21.15.59:0/3629697705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9ae50840 con 0x55dd9af2ded0 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff48f400640 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af35ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.370+0000 7ff494e00640 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ca0 tx=0x7ff4880323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca8e5fc7684789a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.371+0000 7ff485600640 1 -- 172.21.15.59:0/3629697705 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488043070 con 0x55dd9af2ded0 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.371+0000 7ff485600640 1 -- 172.21.15.59:0/3629697705 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff488003970 con 0x55dd9af2ded0 2024-04-02T17:31:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.371+0000 7ff485600640 1 -- 172.21.15.59:0/3629697705 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488039040 con 0x55dd9af2ded0 2024-04-02T17:31:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.372+0000 7ff499b8f740 1 -- 172.21.15.59:0/3629697705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 msgr2=0x55dd9af2e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.372+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ca0 tx=0x7ff4880323c0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff48cc00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 0x7ff4740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 -- 172.21.15.59:0/3629697705 shutdown_connections 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff48cc00640 1 -- 172.21.15.59:0/2446486029 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7ff4800d0820 con 0x7ff490108160 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af35ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2e8c0 0x55dd9af333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3629697705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2ded0 0x55dd9af2e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 -- 172.21.15.59:0/3629697705 >> 172.21.15.59:0/3629697705 conn(0x55dd9af29440 msgr2=0x55dd9af29840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff499b8f740 1 -- 172.21.15.59:0/3629697705 shutdown_connections 2024-04-02T17:31:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.373+0000 7ff494e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 0x7ff4740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.374+0000 7ff499b8f740 1 -- 172.21.15.59:0/3629697705 wait complete. 2024-04-02T17:31:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.374+0000 7ff499b8f740 1 -- start start 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff494e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 0x7ff4740a6210 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff488009a60 tx=0x7ff48802c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 Processor -- start 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af38400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd9af36980 con 0x55dd9af33a30 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ae4f660 con 0x55dd9af2ded0 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dd9af36b00 con 0x55dd9af2e8c0 2024-04-02T17:31:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.375+0000 7ff499b8f740 10 client.0 _ll_register_callbacks cb 0x55dd9ae4f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.376+0000 7ff48f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.376+0000 7ff48f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:50316/0 (socket says 172.21.15.59:50316) 2024-04-02T17:31:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.376+0000 7ff48f400640 1 -- 172.21.15.59:0/1792453875 learned_addr learned my addr 172.21.15.59:0/1792453875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff494e00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48fe00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af38400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48f400640 1 -- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 msgr2=0x55dd9af38400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48f400640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af38400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48f400640 1 -- 172.21.15.59:0/1792453875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 msgr2=0x55dd9af3f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48f400640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af3f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.377+0000 7ff48f400640 1 -- 172.21.15.59:0/1792453875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9ae5ee80 con 0x55dd9af2e8c0 2024-04-02T17:31:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff48fe00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af38400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff48f400640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff47800bf40 tx=0x7ff478011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85164cdbe1c8de01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff47e000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478004220 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7ff4780043c0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.378+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff488003970 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff494e00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff478012660 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff499b8f740 10 client.27385 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dd9aba8210 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff488003970 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7ff478003c70 con 0x55dd9af2e8c0 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff499b8f740 10 client.27385 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.379+0000 7ff499b8f740 10 client.27385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff499b8f740 20 client.27385 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9af44670 0x55dd9af46b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff499b8f740 4 client.27385 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:31:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd9ae43cd0 con 0x55dd9af44670 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff48fe00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9af44670 0x55dd9af46b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff47e000640 10 client.27385 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.380+0000 7ff48fe00640 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9af44670 0x55dd9af46b10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1985d6cce0865c server_cookie=90c65b6fda231e6c in_seq=0 out_seq=0 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.381+0000 7ff47e000640 1 -- 172.21.15.59:0/1792453875 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd9ae43cd0 con 0x55dd9af44670 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.381+0000 7ff47e000640 10 client.27385 handle_command_reply: tid=0 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.381+0000 7ff499b8f740 1 client.27385 shutdown 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 msgr2=0x55dd9af38940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff47800bf40 tx=0x7ff478011870 comp rx=0 tx=0).stop 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 shutdown_connections 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9af44670 0x55dd9af46b10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af38940 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 --2- 172.21.15.59:0/1792453875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af38400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff47e000640 0 client.27385 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 >> 172.21.15.59:0/1792453875 conn(0x55dd9af29440 msgr2=0x55dd9af28640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 shutdown_connections 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 1 -- 172.21.15.59:0/1792453875 wait complete. 2024-04-02T17:31:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.382+0000 7ff499b8f740 20 client.27385 trim_cache size 0 max 16384 2024-04-02T17:31:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:28 smithi154 ceph-mon[36479]: pgmap v941: 97 pgs: 97 active+clean; 920 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 895 op/s 2024-04-02T17:31:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 Processor -- start 2024-04-02T17:31:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 -- start start 2024-04-02T17:31:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901af810 con 0x55dd9af33a30 2024-04-02T17:31:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b1350 con 0x55dd9af2ded0 2024-04-02T17:31:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.409+0000 7ff499b8f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901afee0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.410+0000 7ff494e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.410+0000 7ff494e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60350/0 (socket says 172.21.15.59:60350) 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.410+0000 7ff494e00640 1 -- 172.21.15.59:0/489411888 learned_addr learned my addr 172.21.15.59:0/489411888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.410+0000 7ff48fe00640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48f400640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 -- 172.21.15.59:0/489411888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 msgr2=0x55dd9af3f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 -- 172.21.15.59:0/489411888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 msgr2=0x55dd9af27db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 -- 172.21.15.59:0/489411888 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9ac74620 con 0x55dd9af2ded0 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff494e00640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.411+0000 7ff48fe00640 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af29820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55dd9af2f330 tx=0x7ff4800bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a99ac7e53e7d5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.412+0000 7ff484c00640 1 -- 172.21.15.59:0/489411888 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800b5070 con 0x55dd9af2ded0 2024-04-02T17:31:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.412+0000 7ff484c00640 1 -- 172.21.15.59:0/489411888 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dd9ae43cd0 con 0x55dd9af2ded0 2024-04-02T17:31:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.412+0000 7ff484c00640 1 -- 172.21.15.59:0/489411888 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480003b30 con 0x55dd9af2ded0 2024-04-02T17:31:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 -- 172.21.15.59:0/489411888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 msgr2=0x55dd9af29820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af29820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55dd9af2f330 tx=0x7ff4800bd040 comp rx=0 tx=0).stop 2024-04-02T17:31:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 -- 172.21.15.59:0/489411888 shutdown_connections 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af33a30 0x55dd9af27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 --2- 172.21.15.59:0/489411888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af2ded0 0x55dd9af29820 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.413+0000 7ff499b8f740 1 -- 172.21.15.59:0/489411888 >> 172.21.15.59:0/489411888 conn(0x55dd9af29440 msgr2=0x55dd9af36f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.414+0000 7ff499b8f740 1 -- 172.21.15.59:0/489411888 shutdown_connections 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.414+0000 7ff499b8f740 1 -- 172.21.15.59:0/489411888 wait complete. 2024-04-02T17:31:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.415+0000 7ff499b8f740 1 -- start start 2024-04-02T17:31:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.415+0000 7ff499b8f740 1 Processor -- start 2024-04-02T17:31:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.417+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af2ded0 0x55dd9af3abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.417+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.417+0000 7ff499b8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 0x55dd9af3e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.417+0000 7ff499b8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0c80 con 0x55dd9af2ded0 2024-04-02T17:31:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff499b8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b05b0 con 0x55dd9af33a30 2024-04-02T17:31:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff499b8f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901ad7b0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff499b8f740 10 client.0 _ll_register_callbacks cb 0x7ff47800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff494e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 0x55dd9af3e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff494e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 0x55dd9af3e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:55100/0 (socket says 172.21.15.59:55100) 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff494e00640 1 -- 172.21.15.59:0/3355903187 learned_addr learned my addr 172.21.15.59:0/3355903187 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff48f400640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.418+0000 7ff48fe00640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af2ded0 0x55dd9af3abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.419+0000 7ff48f400640 1 -- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 msgr2=0x55dd9af3e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.419+0000 7ff48f400640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 0x55dd9af3e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.419+0000 7ff48f400640 1 -- 172.21.15.59:0/3355903187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af2ded0 msgr2=0x55dd9af3abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.419+0000 7ff48f400640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af2ded0 0x55dd9af3abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.419+0000 7ff48f400640 1 -- 172.21.15.59:0/3355903187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9af3e700 con 0x55dd9af2e8c0 2024-04-02T17:31:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.420+0000 7ff48f400640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3b0f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff47800dd10 tx=0x7ff47800bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16145352c89abb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.420+0000 7ff47e000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.420+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478033020 con 0x55dd9af2e8c0 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.421+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7ff47801be00 con 0x55dd9af2e8c0 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.421+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd9ae43cd0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.421+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff478012660 con 0x55dd9af2e8c0 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.421+0000 7ff499b8f740 10 client.27391 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.421+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dd9af3fa10 con 0x55dd9af2e8c0 2024-04-02T17:31:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.423+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dd9ae43cd0 con 0x55dd9af2e8c0 2024-04-02T17:31:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.423+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7ff478003c70 con 0x55dd9af2e8c0 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.423+0000 7ff499b8f740 10 client.27391 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff499b8f740 10 client.27391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff499b8f740 20 client.27391 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9afbc360 0x55dd9afbe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff499b8f740 4 client.27391 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:31:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd9aea12b0 con 0x55dd9afbc360 2024-04-02T17:31:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.424+0000 7ff48fe00640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9afbc360 0x55dd9afbe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.425+0000 7ff48fe00640 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9afbc360 0x55dd9afbe780 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bd0375ed6ac532 server_cookie=8c571d3625236f1b in_seq=0 out_seq=0 2024-04-02T17:31:28.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.425+0000 7ff47e000640 10 client.27391 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:28.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:28 smithi059 ceph-mon[31664]: pgmap v941: 97 pgs: 97 active+clean; 920 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 895 op/s 2024-04-02T17:31:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:28 smithi164 ceph-mon[36499]: pgmap v941: 97 pgs: 97 active+clean; 920 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 895 op/s 2024-04-02T17:31:28.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 231784 40.97 MB/sec execute 175 sec latency 104.659 ms 2024-04-02T17:31:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.670+0000 7ff47e000640 1 -- 172.21.15.59:0/3355903187 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd9aea12b0 con 0x55dd9afbc360 2024-04-02T17:31:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.670+0000 7ff47e000640 10 client.27391 handle_command_reply: tid=0 2024-04-02T17:31:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.671+0000 7ff499b8f740 1 client.27391 shutdown 2024-04-02T17:31:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.672+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 msgr2=0x55dd9af3b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.672+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3b0f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff47800dd10 tx=0x7ff47800bac0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.672+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 shutdown_connections 2024-04-02T17:31:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.672+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55dd9afbc360 0x55dd9afbe780 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.673+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dd9af33a30 0x55dd9af3e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.673+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd9af2e8c0 0x55dd9af3b0f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.673+0000 7ff499b8f740 1 --2- 172.21.15.59:0/3355903187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd9af2ded0 0x55dd9af3abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.673+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 >> 172.21.15.59:0/3355903187 conn(0x55dd9af29440 msgr2=0x55dd9af29820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.673+0000 7ff47e000640 0 client.27391 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.674+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 shutdown_connections 2024-04-02T17:31:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.674+0000 7ff499b8f740 1 -- 172.21.15.59:0/3355903187 wait complete. 2024-04-02T17:31:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.674+0000 7ff499b8f740 20 client.27391 trim_cache size 0 max 16384 2024-04-02T17:31:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:31:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:31:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:31:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:31:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.676+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 msgr2=0x7ff4740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.676+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 0x7ff4740a6210 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff488009a60 tx=0x7ff48802c100 comp rx=0 tx=0).stop 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.676+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 msgr2=0x7ff49019e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.676+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff48000e9c0 tx=0x7ff48000ee80 comp rx=0 tx=0).stop 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 shutdown_connections 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff4740a3d50 0x7ff4740a6210 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff490109d10 0x7ff4901a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff490109350 0x7ff49019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 --2- 172.21.15.59:0/2446486029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff490108160 0x7ff49019e370 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 >> 172.21.15.59:0/2446486029 conn(0x7ff490077110 msgr2=0x7ff49010c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 shutdown_connections 2024-04-02T17:31:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:28.677+0000 7ff496e00640 1 -- 172.21.15.59:0/2446486029 wait complete. 2024-04-02T17:31:29.619 INFO:tasks.workunit.client.0.smithi059.stdout: 10 232612 40.96 MB/sec execute 176 sec latency 139.122 ms 2024-04-02T17:31:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:30 smithi154 ceph-mon[36479]: pgmap v942: 97 pgs: 97 active+clean; 921 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-04-02T17:31:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:30 smithi059 ceph-mon[31664]: pgmap v942: 97 pgs: 97 active+clean; 921 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-04-02T17:31:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:30 smithi164 ceph-mon[36499]: pgmap v942: 97 pgs: 97 active+clean; 921 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-04-02T17:31:30.618 INFO:tasks.workunit.client.0.smithi059.stdout: 10 233462 41.05 MB/sec execute 177 sec latency 48.024 ms 2024-04-02T17:31:31.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 234195 41.02 MB/sec execute 178 sec latency 49.603 ms 2024-04-02T17:31:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:32 smithi154 ceph-mon[36479]: pgmap v943: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-04-02T17:31:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:32 smithi059 ceph-mon[31664]: pgmap v943: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-04-02T17:31:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:32 smithi164 ceph-mon[36499]: pgmap v943: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 680 op/s 2024-04-02T17:31:32.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 234840 40.97 MB/sec execute 179 sec latency 123.209 ms 2024-04-02T17:31:33.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 235640 40.98 MB/sec execute 180 sec latency 67.281 ms 2024-04-02T17:31:34.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:34 smithi154 ceph-mon[36479]: pgmap v944: 97 pgs: 97 active+clean; 931 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 632 op/s 2024-04-02T17:31:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:34 smithi059 ceph-mon[31664]: pgmap v944: 97 pgs: 97 active+clean; 931 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 632 op/s 2024-04-02T17:31:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:34 smithi164 ceph-mon[36499]: pgmap v944: 97 pgs: 97 active+clean; 931 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 632 op/s 2024-04-02T17:31:34.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 236569 41.03 MB/sec execute 181 sec latency 55.799 ms 2024-04-02T17:31:35.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 237393 41.08 MB/sec execute 182 sec latency 45.104 ms 2024-04-02T17:31:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:36 smithi154 ceph-mon[36479]: pgmap v945: 97 pgs: 97 active+clean; 943 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 634 op/s 2024-04-02T17:31:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:36 smithi059 ceph-mon[31664]: pgmap v945: 97 pgs: 97 active+clean; 943 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 634 op/s 2024-04-02T17:31:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:36 smithi164 ceph-mon[36499]: pgmap v945: 97 pgs: 97 active+clean; 943 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 634 op/s 2024-04-02T17:31:36.621 INFO:tasks.workunit.client.0.smithi059.stdout: 10 238075 41.03 MB/sec execute 183 sec latency 53.203 ms 2024-04-02T17:31:37.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 238797 41.02 MB/sec execute 184 sec latency 74.592 ms 2024-04-02T17:31:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:38 smithi154 ceph-mon[36479]: pgmap v946: 97 pgs: 97 active+clean; 947 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.4 MiB/s wr, 603 op/s 2024-04-02T17:31:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:38 smithi059 ceph-mon[31664]: pgmap v946: 97 pgs: 97 active+clean; 947 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.4 MiB/s wr, 603 op/s 2024-04-02T17:31:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:38 smithi164 ceph-mon[36499]: pgmap v946: 97 pgs: 97 active+clean; 947 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.4 MiB/s wr, 603 op/s 2024-04-02T17:31:38.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 239532 41.02 MB/sec execute 185 sec latency 200.828 ms 2024-04-02T17:31:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:39 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:31:39.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:39 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:31:39.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 240253 41.04 MB/sec execute 186 sec latency 91.960 ms 2024-04-02T17:31:39.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:39 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:31:40.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:40 smithi059 ceph-mon[31664]: pgmap v947: 97 pgs: 97 active+clean; 950 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-04-02T17:31:40.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:40 smithi164 ceph-mon[36499]: pgmap v947: 97 pgs: 97 active+clean; 950 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-04-02T17:31:40.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 240689 40.87 MB/sec execute 187 sec latency 381.628 ms 2024-04-02T17:31:40.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:40 smithi154 ceph-mon[36479]: pgmap v947: 97 pgs: 97 active+clean; 950 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-04-02T17:31:41.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 241385 40.90 MB/sec execute 188 sec latency 87.997 ms 2024-04-02T17:31:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:41 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:31:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:41 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:31:41.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:41 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:31:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 -- 172.21.15.59:0/2396208912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274075b40 msgr2=0x7fa274073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 --2- 172.21.15.59:0/2396208912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274075b40 0x7fa274073f90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa268009900 tx=0x7fa26802bc30 comp rx=0 tx=0).stop 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 -- 172.21.15.59:0/2396208912 shutdown_connections 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 --2- 172.21.15.59:0/2396208912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2741099f0 0x7fa27410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 --2- 172.21.15.59:0/2396208912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa274074560 0x7fa2740749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 --2- 172.21.15.59:0/2396208912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274075b40 0x7fa274073f90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.866+0000 7fa27b600640 1 -- 172.21.15.59:0/2396208912 >> 172.21.15.59:0/2396208912 conn(0x7fa2740fd0c0 msgr2=0x7fa2740ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.867+0000 7fa27b600640 1 -- 172.21.15.59:0/2396208912 shutdown_connections 2024-04-02T17:31:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.867+0000 7fa27b600640 1 -- 172.21.15.59:0/2396208912 wait complete. 2024-04-02T17:31:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.868+0000 7fa27b600640 1 Processor -- start 2024-04-02T17:31:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.868+0000 7fa27b600640 1 -- start start 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274074560 0x7fa274199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa274075b40 0x7fa27419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa27410f9c0 con 0x7fa274074560 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa27410f840 con 0x7fa2741099f0 2024-04-02T17:31:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.869+0000 7fa27b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa27410fb40 con 0x7fa274075b40 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.870+0000 7fa279600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58142/0 (socket says 172.21.15.59:58142) 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 -- 172.21.15.59:0/1046166801 learned_addr learned my addr 172.21.15.59:0/1046166801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa273e00640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa274075b40 0x7fa27419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 -- 172.21.15.59:0/1046166801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa274075b40 msgr2=0x7fa27419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa274075b40 0x7fa27419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 -- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274074560 msgr2=0x7fa274199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274074560 0x7fa274199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.871+0000 7fa279600640 1 -- 172.21.15.59:0/1046166801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2741a1b70 con 0x7fa2741099f0 2024-04-02T17:31:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.872+0000 7fa279600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00eb10 tx=0x7fa26c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.872+0000 7fa271600640 1 -- 172.21.15.59:0/1046166801 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004b20 con 0x7fa2741099f0 2024-04-02T17:31:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.872+0000 7fa271600640 1 -- 172.21.15.59:0/1046166801 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c004cc0 con 0x7fa2741099f0 2024-04-02T17:31:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.872+0000 7fa271600640 1 -- 172.21.15.59:0/1046166801 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c010760 con 0x7fa2741099f0 2024-04-02T17:31:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.872+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2741a1e60 con 0x7fa2741099f0 2024-04-02T17:31:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.873+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2741a22c0 con 0x7fa2741099f0 2024-04-02T17:31:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.878+0000 7fa271600640 1 -- 172.21.15.59:0/1046166801 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa26c005020 con 0x7fa2741099f0 2024-04-02T17:31:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.879+0000 7fa27e223740 1 Processor -- start 2024-04-02T17:31:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 -- start start 2024-04-02T17:31:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e4074240 0x55e2e40766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e2e3fcbd70 con 0x55e2e406e6e0 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2e3f742a0 con 0x55e2e406f0d0 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.881+0000 7fa27e223740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e2e3fc7d20 con 0x55e2e4074240 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.882+0000 7fa271600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 0x7fa25c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.882+0000 7fa271600640 1 -- 172.21.15.59:0/1046166801 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fa26c0d09a0 con 0x7fa2741099f0 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.882+0000 7fa273e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.882+0000 7fa278c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.883+0000 7fa278c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58148/0 (socket says 172.21.15.59:58148) 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.883+0000 7fa278c00640 1 -- 172.21.15.59:0/1171078670 learned_addr learned my addr 172.21.15.59:0/1171078670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.883+0000 7fa279600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 0x7fa25c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.883+0000 7fa273e00640 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e4074240 0x55e2e40766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa278c00640 1 -- 172.21.15.59:0/1171078670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e4074240 msgr2=0x55e2e40766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa278c00640 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e4074240 0x55e2e40766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa278c00640 1 -- 172.21.15.59:0/1171078670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 msgr2=0x55e2e406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa278c00640 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa278c00640 1 -- 172.21.15.59:0/1171078670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e3f88840 con 0x55e2e406f0d0 2024-04-02T17:31:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa273e00640 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:31:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.884+0000 7fa279600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 0x7fa25c0a6190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa26c09cbb0 tx=0x7fa26c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.885+0000 7fa278c00640 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa268036040 tx=0x7fa26802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfcb74108d3ec35 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.885+0000 7fa265c00640 1 -- 172.21.15.59:0/1171078670 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268041020 con 0x55e2e406f0d0 2024-04-02T17:31:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.885+0000 7fa265c00640 1 -- 172.21.15.59:0/1171078670 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26802cb80 con 0x55e2e406f0d0 2024-04-02T17:31:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.885+0000 7fa265c00640 1 -- 172.21.15.59:0/1171078670 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268004920 con 0x55e2e406f0d0 2024-04-02T17:31:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.886+0000 7fa27e223740 1 -- 172.21.15.59:0/1171078670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 msgr2=0x55e2e4073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.886+0000 7fa27e223740 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa268036040 tx=0x7fa26802c5d0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.886+0000 7fa27e223740 1 -- 172.21.15.59:0/1171078670 shutdown_connections 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.886+0000 7fa27e223740 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e4074240 0x55e2e40766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e406f0d0 0x55e2e4073bd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 --2- 172.21.15.59:0/1171078670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 -- 172.21.15.59:0/1171078670 >> 172.21.15.59:0/1171078670 conn(0x55e2e4061440 msgr2=0x55e2e4061840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 -- 172.21.15.59:0/1171078670 shutdown_connections 2024-04-02T17:31:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 -- 172.21.15.59:0/1171078670 wait complete. 2024-04-02T17:31:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.887+0000 7fa27e223740 1 -- start start 2024-04-02T17:31:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 Processor -- start 2024-04-02T17:31:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4067f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e406cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e2e4065f70 con 0x55e2e406f0d0 2024-04-02T17:31:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2e3f87660 con 0x55e2e4074240 2024-04-02T17:31:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e2e40660f0 con 0x55e2e406e6e0 2024-04-02T17:31:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.888+0000 7fa27e223740 10 client.0 _ll_register_callbacks cb 0x55e2e3f87050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35426/0 (socket says 172.21.15.59:35426) 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 -- 172.21.15.59:0/536592496 learned_addr learned my addr 172.21.15.59:0/536592496 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e406cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa278c00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4067f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 -- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 msgr2=0x55e2e406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e406cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 -- 172.21.15.59:0/536592496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 msgr2=0x55e2e4067f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.889+0000 7fa273e00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4067f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.890+0000 7fa273e00640 1 -- 172.21.15.59:0/536592496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e3f96e80 con 0x55e2e406e6e0 2024-04-02T17:31:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.890+0000 7fa273e00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c23a761a0f9e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.890+0000 7fa25a000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.890+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260010d20 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fa260012050 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26802cb80 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa260012250 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa27e223740 10 client.27403 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e2e3ce0210 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.891+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26802cb80 con 0x55e2e406e6e0 2024-04-02T17:31:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fa260003b10 con 0x55e2e406e6e0 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 10 client.27403 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 10 client.27403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 20 client.27403 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e408d1d0 0x55e2e408f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 4 client.27403 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2e3f7bcd0 con 0x55e2e408d1d0 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.892+0000 7fa278c00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e408d1d0 0x55e2e408f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.893+0000 7fa278c00640 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e408d1d0 0x55e2e408f600 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c257167cbe6a72 server_cookie=db9f9fb12adf6f94 in_seq=0 out_seq=0 2024-04-02T17:31:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.893+0000 7fa25a000640 10 client.27403 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.894+0000 7fa25a000640 1 -- 172.21.15.59:0/536592496 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2e3f7bcd0 con 0x55e2e408d1d0 2024-04-02T17:31:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.894+0000 7fa25a000640 10 client.27403 handle_command_reply: tid=0 2024-04-02T17:31:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.894+0000 7fa27e223740 1 client.27403 shutdown 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.894+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 msgr2=0x55e2e40679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.894+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260014040 comp rx=0 tx=0).stop 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 shutdown_connections 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e408d1d0 0x55e2e408f600 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4067f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e406cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa25a000640 0 client.27403 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 --2- 172.21.15.59:0/536592496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e40679d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 >> 172.21.15.59:0/536592496 conn(0x55e2e4061440 msgr2=0x55e2e4060640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 shutdown_connections 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 1 -- 172.21.15.59:0/536592496 wait complete. 2024-04-02T17:31:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.895+0000 7fa27e223740 20 client.27403 trim_cache size 0 max 16384 2024-04-02T17:31:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:41 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:31:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:41 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:31:41.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:41 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:31:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 Processor -- start 2024-04-02T17:31:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 -- start start 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e4067e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a6f50 con 0x55e2e406f0d0 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a8a90 con 0x55e2e4074240 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.924+0000 7fa27e223740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a7620 con 0x55e2e406e6e0 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa273e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa278c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa278c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47488/0 (socket says 172.21.15.59:47488) 2024-04-02T17:31:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa273e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58172/0 (socket says 172.21.15.59:58172) 2024-04-02T17:31:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa273e00640 1 -- 172.21.15.59:0/2730447853 learned_addr learned my addr 172.21.15.59:0/2730447853 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.925+0000 7fa273e00640 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e4067e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 -- 172.21.15.59:0/2730447853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 msgr2=0x55e2e4067e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e4067e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 -- 172.21.15.59:0/2730447853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 msgr2=0x55e2e4066490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 -- 172.21.15.59:0/2730447853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e3dac620 con 0x55e2e4074240 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa278c00640 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.926+0000 7fa273e00640 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa260012860 tx=0x7fa26000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7897ce6e8c47c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.927+0000 7fa265200640 1 -- 172.21.15.59:0/2730447853 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260010d20 con 0x55e2e4074240 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.927+0000 7fa265200640 1 -- 172.21.15.59:0/2730447853 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa260031070 con 0x55e2e4074240 2024-04-02T17:31:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.927+0000 7fa265200640 1 -- 172.21.15.59:0/2730447853 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600038e0 con 0x55e2e4074240 2024-04-02T17:31:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 -- 172.21.15.59:0/2730447853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 msgr2=0x55e2e405fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa260012860 tx=0x7fa26000b480 comp rx=0 tx=0).stop 2024-04-02T17:31:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 -- 172.21.15.59:0/2730447853 shutdown_connections 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406f0d0 0x55e2e4066490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e405fdb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 --2- 172.21.15.59:0/2730447853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406e6e0 0x55e2e4067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 -- 172.21.15.59:0/2730447853 >> 172.21.15.59:0/2730447853 conn(0x55e2e4061440 msgr2=0x55e2e4061820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 -- 172.21.15.59:0/2730447853 shutdown_connections 2024-04-02T17:31:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.928+0000 7fa27e223740 1 -- 172.21.15.59:0/2730447853 wait complete. 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.929+0000 7fa27e223740 1 -- start start 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.929+0000 7fa27e223740 1 Processor -- start 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.929+0000 7fa27e223740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e409e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e409f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a83c0 con 0x55e2e406e6e0 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a7cf0 con 0x55e2e4074240 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741a4f50 con 0x55e2e406f0d0 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa27e223740 10 client.0 _ll_register_callbacks cb 0x7fa2600129f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35438/0 (socket says 172.21.15.59:35438) 2024-04-02T17:31:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 -- 172.21.15.59:0/266990660 learned_addr learned my addr 172.21.15.59:0/266990660 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa273e00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e409e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 -- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 msgr2=0x55e2e409f8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e409f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 -- 172.21.15.59:0/266990660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 msgr2=0x55e2e409e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e409e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa278c00640 1 -- 172.21.15.59:0/266990660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e409fe90 con 0x55e2e406f0d0 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.930+0000 7fa273e00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e409e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.931+0000 7fa278c00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa268036040 tx=0x7fa26802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3c8cc3838b6b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.931+0000 7fa25a000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.931+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268041040 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.931+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fa26804db50 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.931+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa260031070 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.932+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa268004920 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.932+0000 7fa27e223740 10 client.27409 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.932+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e2e40f3460 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.932+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa260031070 con 0x55e2e406f0d0 2024-04-02T17:31:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.932+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fa26804d590 con 0x55e2e406f0d0 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 10 client.27409 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 10 client.27409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 20 client.27409 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e40f3a30 0x55e2e40f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 4 client.27409 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2e3fd92b0 con 0x55e2e40f3a30 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.933+0000 7fa273e00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e40f3a30 0x55e2e40f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.934+0000 7fa25a000640 10 client.27409 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:41.934+0000 7fa273e00640 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e40f3a30 0x55e2e40f5e20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dda82f4b22dfc91 server_cookie=eb055e3eaecafcc6 in_seq=0 out_seq=0 2024-04-02T17:31:42.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:41 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:31:42.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:41 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:31:42.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:41 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:31:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.173+0000 7fa25a000640 1 -- 172.21.15.59:0/266990660 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2e3fd92b0 con 0x55e2e40f3a30 2024-04-02T17:31:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.173+0000 7fa25a000640 10 client.27409 handle_command_reply: tid=0 2024-04-02T17:31:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 client.27409 shutdown 2024-04-02T17:31:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 msgr2=0x55e2e409e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa268036040 tx=0x7fa26802df90 comp rx=0 tx=0).stop 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 shutdown_connections 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e2e40f3a30 0x55e2e40f5e20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2e4074240 0x55e2e409f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e406f0d0 0x55e2e409e900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 --2- 172.21.15.59:0/266990660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e2e406e6e0 0x55e2e409e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 >> 172.21.15.59:0/266990660 conn(0x55e2e4061440 msgr2=0x55e2e4066490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.174+0000 7fa25a000640 0 client.27409 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.175+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 shutdown_connections 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.175+0000 7fa27e223740 1 -- 172.21.15.59:0/266990660 wait complete. 2024-04-02T17:31:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.175+0000 7fa27e223740 20 client.27409 trim_cache size 0 max 16384 2024-04-02T17:31:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:31:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:31:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:31:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 msgr2=0x7fa25c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 0x7fa25c0a6190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa26c09cbb0 tx=0x7fa26c0058d0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 msgr2=0x7fa2741a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00eb10 tx=0x7fa26c00c2d0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 shutdown_connections 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2741099f0 0x7fa2741a1500 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa274075b40 0x7fa27419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa25c0a3cd0 0x7fa25c0a6190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 --2- 172.21.15.59:0/1046166801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa274074560 0x7fa274199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.176+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 >> 172.21.15.59:0/1046166801 conn(0x7fa2740fd0c0 msgr2=0x7fa2741058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.177+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 shutdown_connections 2024-04-02T17:31:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:42.177+0000 7fa27b600640 1 -- 172.21.15.59:0/1046166801 wait complete. 2024-04-02T17:31:42.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 242166 40.90 MB/sec execute 189 sec latency 162.111 ms 2024-04-02T17:31:42.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:42 smithi164 ceph-mon[36499]: pgmap v948: 97 pgs: 97 active+clean; 973 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-02T17:31:42.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:42 smithi154 ceph-mon[36479]: pgmap v948: 97 pgs: 97 active+clean; 973 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-02T17:31:43.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:42 smithi059 ceph-mon[31664]: pgmap v948: 97 pgs: 97 active+clean; 973 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-02T17:31:43.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 242924 40.94 MB/sec execute 190 sec latency 51.084 ms 2024-04-02T17:31:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:44 smithi154 ceph-mon[36479]: pgmap v949: 97 pgs: 97 active+clean; 968 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-04-02T17:31:44.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:44 smithi059 ceph-mon[31664]: pgmap v949: 97 pgs: 97 active+clean; 968 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-04-02T17:31:44.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:44 smithi164 ceph-mon[36499]: pgmap v949: 97 pgs: 97 active+clean; 968 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 650 op/s 2024-04-02T17:31:44.622 INFO:tasks.workunit.client.0.smithi059.stdout: 10 243467 40.88 MB/sec execute 191 sec latency 484.008 ms 2024-04-02T17:31:45.624 INFO:tasks.workunit.client.0.smithi059.stdout: 10 243832 40.70 MB/sec execute 192 sec latency 1037.282 ms 2024-04-02T17:31:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:46 smithi154 ceph-mon[36479]: pgmap v950: 97 pgs: 97 active+clean; 962 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 688 op/s 2024-04-02T17:31:46.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:46 smithi059 ceph-mon[31664]: pgmap v950: 97 pgs: 97 active+clean; 962 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 688 op/s 2024-04-02T17:31:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:46 smithi164 ceph-mon[36499]: pgmap v950: 97 pgs: 97 active+clean; 962 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 688 op/s 2024-04-02T17:31:46.624 INFO:tasks.workunit.client.0.smithi059.stdout: 10 243866 40.50 MB/sec execute 193 sec latency 980.826 ms 2024-04-02T17:31:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: 10 244179 40.36 MB/sec execute 194 sec latency 1253.114 ms 2024-04-02T17:31:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:48 smithi154 ceph-mon[36479]: pgmap v951: 97 pgs: 97 active+clean; 1011 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-02T17:31:48.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:48 smithi059 ceph-mon[31664]: pgmap v951: 97 pgs: 97 active+clean; 1011 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-02T17:31:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:48 smithi164 ceph-mon[36499]: pgmap v951: 97 pgs: 97 active+clean; 1011 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-02T17:31:48.624 INFO:tasks.workunit.client.0.smithi059.stdout: 10 244644 40.29 MB/sec execute 195 sec latency 131.359 ms 2024-04-02T17:31:49.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 245466 40.34 MB/sec execute 196 sec latency 90.095 ms 2024-04-02T17:31:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:50 smithi154 ceph-mon[36479]: pgmap v952: 97 pgs: 97 active+clean; 1017 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 766 op/s 2024-04-02T17:31:50.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:50 smithi059 ceph-mon[31664]: pgmap v952: 97 pgs: 97 active+clean; 1017 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 766 op/s 2024-04-02T17:31:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:50 smithi164 ceph-mon[36499]: pgmap v952: 97 pgs: 97 active+clean; 1017 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 766 op/s 2024-04-02T17:31:50.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 246205 40.37 MB/sec execute 197 sec latency 58.533 ms 2024-04-02T17:31:51.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 246830 40.34 MB/sec execute 198 sec latency 53.208 ms 2024-04-02T17:31:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:52 smithi154 ceph-mon[36479]: pgmap v953: 97 pgs: 97 active+clean; 991 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-04-02T17:31:52.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:52 smithi059 ceph-mon[31664]: pgmap v953: 97 pgs: 97 active+clean; 991 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-04-02T17:31:52.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:52 smithi164 ceph-mon[36499]: pgmap v953: 97 pgs: 97 active+clean; 991 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-04-02T17:31:52.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 247564 40.28 MB/sec execute 199 sec latency 44.640 ms 2024-04-02T17:31:53.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 248391 40.34 MB/sec execute 200 sec latency 79.405 ms 2024-04-02T17:31:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:54 smithi154 ceph-mon[36479]: pgmap v954: 97 pgs: 97 active+clean; 954 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 971 op/s 2024-04-02T17:31:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:54 smithi059 ceph-mon[31664]: pgmap v954: 97 pgs: 97 active+clean; 954 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 971 op/s 2024-04-02T17:31:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:54 smithi164 ceph-mon[36499]: pgmap v954: 97 pgs: 97 active+clean; 954 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 971 op/s 2024-04-02T17:31:54.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 249307 40.37 MB/sec execute 201 sec latency 56.424 ms 2024-04-02T17:31:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.342+0000 7f4308000640 1 -- 172.21.15.59:0/3542338520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43001036e0 msgr2=0x7f4300109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.342+0000 7f4308000640 1 --2- 172.21.15.59:0/3542338520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43001036e0 0x7f4300109f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f42f801a910 tx=0x7f42f803cc60 comp rx=0 tx=0).stop 2024-04-02T17:31:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 -- 172.21.15.59:0/3542338520 shutdown_connections 2024-04-02T17:31:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 --2- 172.21.15.59:0/3542338520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43001036e0 0x7f4300109f70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 --2- 172.21.15.59:0/3542338520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4300102d20 0x7f43001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 --2- 172.21.15.59:0/3542338520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 0x7f4300101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 -- 172.21.15.59:0/3542338520 >> 172.21.15.59:0/3542338520 conn(0x7f43000fd0c0 msgr2=0x7f43000ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 -- 172.21.15.59:0/3542338520 shutdown_connections 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.343+0000 7f4308000640 1 -- 172.21.15.59:0/3542338520 wait complete. 2024-04-02T17:31:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.344+0000 7f4308000640 1 Processor -- start 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 -- start start 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 0x7f4300073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 0x7f4300074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f430006e170 con 0x7f4300101b30 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f430006dff0 con 0x7f4300102d20 2024-04-02T17:31:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4308000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f430006e2f0 con 0x7f43001036e0 2024-04-02T17:31:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4304c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4304c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:48068/0 (socket says 172.21.15.59:48068) 2024-04-02T17:31:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4304c00640 1 -- 172.21.15.59:0/3615577726 learned_addr learned my addr 172.21.15.59:0/3615577726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4305600640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 0x7f4300073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.345+0000 7f4306000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 0x7f4300074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 -- 172.21.15.59:0/3615577726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 msgr2=0x7f4300074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 0x7f4300074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 -- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 msgr2=0x7f4300073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 0x7f4300073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 -- 172.21.15.59:0/3615577726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4300075550 con 0x7f4300102d20 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4306000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 0x7f4300074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:31:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.346+0000 7f4304c00640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9f0 tx=0x7f42ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.347+0000 7f42fe000640 1 -- 172.21.15.59:0/3615577726 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00cd80 con 0x7f4300102d20 2024-04-02T17:31:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.347+0000 7f42fe000640 1 -- 172.21.15.59:0/3615577726 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42ec010910 con 0x7f4300102d20 2024-04-02T17:31:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.347+0000 7f42fe000640 1 -- 172.21.15.59:0/3615577726 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec018990 con 0x7f4300102d20 2024-04-02T17:31:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.347+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4300075840 con 0x7f4300102d20 2024-04-02T17:31:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.348+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4300075c80 con 0x7f4300102d20 2024-04-02T17:31:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.351+0000 7f430ac33740 1 Processor -- start 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 -- start start 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x555849294ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5558491f2d70 con 0x55584928d8c0 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55584919b2a0 con 0x555849292a30 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.352+0000 7f430ac33740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558491eed20 con 0x55584928ced0 2024-04-02T17:31:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52848/0 (socket says 172.21.15.59:52848) 2024-04-02T17:31:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 -- 172.21.15.59:0/3781281209 learned_addr learned my addr 172.21.15.59:0/3781281209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4305600640 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 -- 172.21.15.59:0/3781281209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 msgr2=0x555849294ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x555849294ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 -- 172.21.15.59:0/3781281209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 msgr2=0x5558492923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492923c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4306000640 1 -- 172.21.15.59:0/3781281209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558491af840 con 0x55584928ced0 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f4305600640 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.353+0000 7f42fe000640 1 -- 172.21.15.59:0/3615577726 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f42ec010ac0 con 0x7f4300102d20 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.354+0000 7f4306000640 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42f80047e0 tx=0x7f42f8004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857d9dd5de1634af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.354+0000 7f42f6600640 1 -- 172.21.15.59:0/3781281209 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f803d810 con 0x55584928ced0 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.354+0000 7f42f6600640 1 -- 172.21.15.59:0/3781281209 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f80035f0 con 0x55584928ced0 2024-04-02T17:31:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.354+0000 7f42f6600640 1 -- 172.21.15.59:0/3781281209 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8045740 con 0x55584928ced0 2024-04-02T17:31:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.355+0000 7f430ac33740 1 -- 172.21.15.59:0/3781281209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 msgr2=0x55584928d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.355+0000 7f430ac33740 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42f80047e0 tx=0x7f42f8004950 comp rx=0 tx=0).stop 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 -- 172.21.15.59:0/3781281209 shutdown_connections 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x555849294ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 --2- 172.21.15.59:0/3781281209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x55584928d2f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 -- 172.21.15.59:0/3781281209 >> 172.21.15.59:0/3781281209 conn(0x555849288440 msgr2=0x555849288840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f42fe000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 0x7f42e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f42fe000640 1 -- 172.21.15.59:0/3615577726 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f42ec0d0670 con 0x7f4300102d20 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.356+0000 7f430ac33740 1 -- 172.21.15.59:0/3781281209 shutdown_connections 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.357+0000 7f4305600640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 0x7f42e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.357+0000 7f430ac33740 1 -- 172.21.15.59:0/3781281209 wait complete. 2024-04-02T17:31:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.357+0000 7f430ac33740 1 -- start start 2024-04-02T17:31:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.358+0000 7f430ac33740 1 Processor -- start 2024-04-02T17:31:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.358+0000 7f4305600640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 0x7f42e40a6280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5558492893b0 tx=0x7f42f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.359+0000 7f430ac33740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x5558492a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.359+0000 7f430ac33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.359+0000 7f430ac33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x5558492af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.359+0000 7f4305600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x5558492a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.359+0000 7f4305600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x5558492a8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52862/0 (socket says 172.21.15.59:52862) 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4305600640 1 -- 172.21.15.59:0/4085351295 learned_addr learned my addr 172.21.15.59:0/4085351295 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5558492a6990 con 0x55584928d8c0 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5558491ae660 con 0x555849292a30 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558492a6b10 con 0x55584928ced0 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f430ac33740 10 client.0 _ll_register_callbacks cb 0x5558491ae050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x5558492af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4304c00640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 -- 172.21.15.59:0/4085351295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 msgr2=0x5558492a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x5558492a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 -- 172.21.15.59:0/4085351295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 msgr2=0x5558492a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492a8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.360+0000 7f4306000640 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558491bde80 con 0x555849292a30 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.361+0000 7f4306000640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x5558492af9a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42f803d1a0 tx=0x7f42f8003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10eb8183e75a4f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.361+0000 7f42eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.361+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80454e0 con 0x555849292a30 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.361+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f42f8045680 con 0x555849292a30 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.361+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f80035f0 con 0x555849292a30 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.362+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f42f8068050 con 0x555849292a30 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.362+0000 7f430ac33740 10 client.27371 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.362+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555848f07210 con 0x555849292a30 2024-04-02T17:31:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.362+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f80035f0 con 0x555849292a30 2024-04-02T17:31:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.362+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f42f80638d0 con 0x555849292a30 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 10 client.27371 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 10 client.27371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 20 client.27371 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558492b3e00 0x5558492b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 4 client.27371 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558491a2cd0 con 0x5558492b3e00 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.363+0000 7f4305600640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558492b3e00 0x5558492b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.364+0000 7f4305600640 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558492b3e00 0x5558492b61f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88521543e89e4908 server_cookie=a713dd5d76078e19 in_seq=0 out_seq=0 2024-04-02T17:31:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.364+0000 7f42eaa00640 10 client.27371 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.365+0000 7f42eaa00640 1 -- 172.21.15.59:0/4085351295 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558491a2cd0 con 0x5558492b3e00 2024-04-02T17:31:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f42eaa00640 10 client.27371 handle_command_reply: tid=0 2024-04-02T17:31:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 client.27371 shutdown 2024-04-02T17:31:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 msgr2=0x5558492af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x5558492af9a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42f803d1a0 tx=0x7f42f8003a50 comp rx=0 tx=0).stop 2024-04-02T17:31:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 shutdown_connections 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558492b3e00 0x5558492b61f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928d8c0 0x5558492a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555849292a30 0x5558492af9a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 --2- 172.21.15.59:0/4085351295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928ced0 0x5558492a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.366+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 >> 172.21.15.59:0/4085351295 conn(0x555849288440 msgr2=0x555849287640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.367+0000 7f42eaa00640 0 client.27371 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.368+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 shutdown_connections 2024-04-02T17:31:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.369+0000 7f430ac33740 1 -- 172.21.15.59:0/4085351295 wait complete. 2024-04-02T17:31:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.369+0000 7f430ac33740 20 client.27371 trim_cache size 0 max 16384 2024-04-02T17:31:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 Processor -- start 2024-04-02T17:31:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 -- start start 2024-04-02T17:31:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928ced0 0x5558492a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928d8c0 0x5558492af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f430019e620 con 0x55584928ced0 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43001a0160 con 0x55584928d8c0 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f430ac33740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f430019ecf0 con 0x555849292a30 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f4306000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f4306000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52868/0 (socket says 172.21.15.59:52868) 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.394+0000 7f4306000640 1 -- 172.21.15.59:0/3255477730 learned_addr learned my addr 172.21.15.59:0/3255477730 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 -- 172.21.15.59:0/3255477730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928d8c0 msgr2=0x5558492af960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928d8c0 0x5558492af960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 -- 172.21.15.59:0/3255477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928ced0 msgr2=0x5558492a6ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928ced0 0x5558492a6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 -- 172.21.15.59:0/3255477730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555848fd3620 con 0x555849292a30 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f4306000640 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42f80059d0 tx=0x7f42f8004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3b2ba2666f8487 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f42f5c00640 1 -- 172.21.15.59:0/3255477730 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f80035f0 con 0x555849292a30 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f42f5c00640 1 -- 172.21.15.59:0/3255477730 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f8003e50 con 0x555849292a30 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.395+0000 7f42f5c00640 1 -- 172.21.15.59:0/3255477730 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8045950 con 0x555849292a30 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.396+0000 7f430ac33740 1 -- 172.21.15.59:0/3255477730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 msgr2=0x555849286db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.396+0000 7f430ac33740 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42f80059d0 tx=0x7f42f8004950 comp rx=0 tx=0).stop 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.396+0000 7f430ac33740 1 -- 172.21.15.59:0/3255477730 shutdown_connections 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.396+0000 7f430ac33740 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555849292a30 0x555849286db0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928d8c0 0x5558492af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 --2- 172.21.15.59:0/3255477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55584928ced0 0x5558492a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 -- 172.21.15.59:0/3255477730 >> 172.21.15.59:0/3255477730 conn(0x555849288440 msgr2=0x5558492a8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 -- 172.21.15.59:0/3255477730 shutdown_connections 2024-04-02T17:31:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 -- 172.21.15.59:0/3255477730 wait complete. 2024-04-02T17:31:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.397+0000 7f430ac33740 1 -- start start 2024-04-02T17:31:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 Processor -- start 2024-04-02T17:31:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928ced0 0x5558492ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 0x5558492acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 0x5558493182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f430019fa90 con 0x555849292a30 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f430019f3c0 con 0x55584928ced0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4300196570 con 0x55584928d8c0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.398+0000 7f430ac33740 10 client.0 _ll_register_callbacks cb 0x7f42f8004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4306000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 0x5558493182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4306000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 0x5558493182b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36250/0 (socket says 172.21.15.59:36250) 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4306000640 1 -- 172.21.15.59:0/1014334502 learned_addr learned my addr 172.21.15.59:0/1014334502 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 0x5558492acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4305600640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928ced0 0x5558492ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 -- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928ced0 msgr2=0x5558492ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928ced0 0x5558492ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 -- 172.21.15.59:0/1014334502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 msgr2=0x5558493182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 0x5558493182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.399+0000 7f4304c00640 1 -- 172.21.15.59:0/1014334502 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558493187f0 con 0x55584928d8c0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f4304c00640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 0x5558492acd30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9c0 tx=0x7f42ec00e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80b07abc81d43f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f42eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec018d30 con 0x55584928d8c0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f42ec09bdf0 con 0x55584928d8c0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f8003e50 con 0x55584928d8c0 2024-04-02T17:31:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.400+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f42ec0a0ca0 con 0x55584928d8c0 2024-04-02T17:31:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f430ac33740 10 client.27427 fetch_fsmap learned FSMap version 55 2024-04-02T17:31:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555849319b00 con 0x55584928d8c0 2024-04-02T17:31:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f8003e50 con 0x55584928d8c0 2024-04-02T17:31:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f42ec0bbab0 con 0x55584928d8c0 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f430ac33740 10 client.27427 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f430ac33740 10 client.27427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.401+0000 7f430ac33740 20 client.27427 populate_metadata read hostname 'smithi059' 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.402+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55584931a460 0x55584931c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.402+0000 7f430ac33740 4 client.27427 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.402+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558492002b0 con 0x55584931a460 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.402+0000 7f4305600640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55584931a460 0x55584931c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.403+0000 7f42eaa00640 10 client.27427 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.404+0000 7f4305600640 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55584931a460 0x55584931c880 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c5c45cfea6b1de server_cookie=7854fb364522dc1 in_seq=0 out_seq=0 2024-04-02T17:31:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.601+0000 7f42eaa00640 1 -- 172.21.15.59:0/1014334502 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558492002b0 con 0x55584931a460 2024-04-02T17:31:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.601+0000 7f42eaa00640 10 client.27427 handle_command_reply: tid=0 2024-04-02T17:31:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.601+0000 7f430ac33740 1 client.27427 shutdown 2024-04-02T17:31:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 msgr2=0x5558492acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 0x5558492acd30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9c0 tx=0x7f42ec00e940 comp rx=0 tx=0).stop 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 shutdown_connections 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55584931a460 0x55584931c880 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555849292a30 0x5558493182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55584928d8c0 0x5558492acd30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 --2- 172.21.15.59:0/1014334502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55584928ced0 0x5558492ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 >> 172.21.15.59:0/1014334502 conn(0x555849288440 msgr2=0x55584928b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f42eaa00640 0 client.27427 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.602+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 shutdown_connections 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.603+0000 7f430ac33740 1 -- 172.21.15.59:0/1014334502 wait complete. 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.603+0000 7f430ac33740 20 client.27427 trim_cache size 0 max 16384 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:31:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 msgr2=0x7f42e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 0x7f42e40a6280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5558492893b0 tx=0x7f42f0009500 comp rx=0 tx=0).stop 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 msgr2=0x7f4300073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00e9f0 tx=0x7f42ec00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 shutdown_connections 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43001036e0 0x7f4300074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4300102d20 0x7f4300073ea0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f42e40a3dc0 0x7f42e40a6280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 --2- 172.21.15.59:0/3615577726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300101b30 0x7f4300073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:31:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.604+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 >> 172.21.15.59:0/3615577726 conn(0x7f43000fd0c0 msgr2=0x7f4300105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:31:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.606+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 shutdown_connections 2024-04-02T17:31:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:31:55.606+0000 7f4308000640 1 -- 172.21.15.59:0/3615577726 wait complete. 2024-04-02T17:31:55.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 249927 40.37 MB/sec execute 202 sec latency 127.929 ms 2024-04-02T17:31:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:56 smithi154 ceph-mon[36479]: pgmap v955: 97 pgs: 97 active+clean; 959 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 17 MiB/s wr, 988 op/s 2024-04-02T17:31:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:56 smithi059 ceph-mon[31664]: pgmap v955: 97 pgs: 97 active+clean; 959 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 17 MiB/s wr, 988 op/s 2024-04-02T17:31:56.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:56 smithi164 ceph-mon[36499]: pgmap v955: 97 pgs: 97 active+clean; 959 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 17 MiB/s wr, 988 op/s 2024-04-02T17:31:56.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 250506 40.29 MB/sec execute 203 sec latency 103.603 ms 2024-04-02T17:31:57.625 INFO:tasks.workunit.client.0.smithi059.stdout: 10 251228 40.29 MB/sec execute 204 sec latency 41.949 ms 2024-04-02T17:31:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:31:58 smithi154 ceph-mon[36479]: pgmap v956: 97 pgs: 97 active+clean; 967 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 989 op/s 2024-04-02T17:31:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:31:58 smithi059 ceph-mon[31664]: pgmap v956: 97 pgs: 97 active+clean; 967 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 989 op/s 2024-04-02T17:31:58.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:31:58 smithi164 ceph-mon[36499]: pgmap v956: 97 pgs: 97 active+clean; 967 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 989 op/s 2024-04-02T17:31:58.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 252259 40.37 MB/sec execute 205 sec latency 41.142 ms 2024-04-02T17:31:59.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 253070 40.40 MB/sec execute 206 sec latency 43.374 ms 2024-04-02T17:32:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:00 smithi154 ceph-mon[36479]: pgmap v957: 97 pgs: 97 active+clean; 968 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-04-02T17:32:00.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:00 smithi059 ceph-mon[31664]: pgmap v957: 97 pgs: 97 active+clean; 968 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-04-02T17:32:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:00 smithi164 ceph-mon[36499]: pgmap v957: 97 pgs: 97 active+clean; 968 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 819 op/s 2024-04-02T17:32:00.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 253711 40.39 MB/sec execute 207 sec latency 230.322 ms 2024-04-02T17:32:01.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 254500 40.31 MB/sec execute 208 sec latency 48.188 ms 2024-04-02T17:32:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:02 smithi154 ceph-mon[36479]: pgmap v958: 97 pgs: 97 active+clean; 975 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 886 op/s 2024-04-02T17:32:02.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:02 smithi059 ceph-mon[31664]: pgmap v958: 97 pgs: 97 active+clean; 975 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 886 op/s 2024-04-02T17:32:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:02 smithi164 ceph-mon[36499]: pgmap v958: 97 pgs: 97 active+clean; 975 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 886 op/s 2024-04-02T17:32:02.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 255435 40.41 MB/sec execute 209 sec latency 47.198 ms 2024-04-02T17:32:03.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 256319 40.47 MB/sec execute 210 sec latency 47.707 ms 2024-04-02T17:32:04.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:04 smithi154 ceph-mon[36479]: pgmap v959: 97 pgs: 97 active+clean; 983 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-04-02T17:32:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:04 smithi059 ceph-mon[31664]: pgmap v959: 97 pgs: 97 active+clean; 983 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-04-02T17:32:04.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:04 smithi164 ceph-mon[36499]: pgmap v959: 97 pgs: 97 active+clean; 983 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 667 op/s 2024-04-02T17:32:04.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 256959 40.43 MB/sec execute 211 sec latency 41.370 ms 2024-04-02T17:32:05.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 257686 40.40 MB/sec execute 212 sec latency 58.236 ms 2024-04-02T17:32:06.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:06 smithi154 ceph-mon[36479]: pgmap v960: 97 pgs: 97 active+clean; 989 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:32:06.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:06 smithi059 ceph-mon[31664]: pgmap v960: 97 pgs: 97 active+clean; 989 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:32:06.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:06 smithi164 ceph-mon[36499]: pgmap v960: 97 pgs: 97 active+clean; 989 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:32:06.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 258607 40.44 MB/sec execute 213 sec latency 48.293 ms 2024-04-02T17:32:07.626 INFO:tasks.workunit.client.0.smithi059.stdout: 10 259355 40.48 MB/sec execute 214 sec latency 66.807 ms 2024-04-02T17:32:08.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:08 smithi154 ceph-mon[36479]: pgmap v961: 97 pgs: 97 active+clean; 999 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 646 op/s 2024-04-02T17:32:08.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:08 smithi059 ceph-mon[31664]: pgmap v961: 97 pgs: 97 active+clean; 999 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 646 op/s 2024-04-02T17:32:08.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:08 smithi164 ceph-mon[36499]: pgmap v961: 97 pgs: 97 active+clean; 999 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 646 op/s 2024-04-02T17:32:08.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 259962 40.41 MB/sec execute 215 sec latency 235.859 ms 2024-04-02T17:32:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.780+0000 7f6556a00640 1 -- 172.21.15.59:0/36374259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550102450 msgr2=0x7f6550104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.780+0000 7f6556a00640 1 --2- 172.21.15.59:0/36374259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550102450 0x7f6550104840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402bc60 comp rx=0 tx=0).stop 2024-04-02T17:32:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 -- 172.21.15.59:0/36374259 shutdown_connections 2024-04-02T17:32:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 --2- 172.21.15.59:0/36374259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6550104d80 0x7f6550107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 --2- 172.21.15.59:0/36374259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 0x7f6550109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 --2- 172.21.15.59:0/36374259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550102450 0x7f6550104840 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 -- 172.21.15.59:0/36374259 >> 172.21.15.59:0/36374259 conn(0x7f65500fbde0 msgr2=0x7f65500fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 -- 172.21.15.59:0/36374259 shutdown_connections 2024-04-02T17:32:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.781+0000 7f6556a00640 1 -- 172.21.15.59:0/36374259 wait complete. 2024-04-02T17:32:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.782+0000 7f6556a00640 1 Processor -- start 2024-04-02T17:32:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 -- start start 2024-04-02T17:32:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6550102450 0x7f6550110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 0x7f65501ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65501020b0 con 0x7f6550102450 2024-04-02T17:32:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6550101f30 con 0x7f65501076b0 2024-04-02T17:32:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6550102230 con 0x7f6550104d80 2024-04-02T17:32:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6554c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6554c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57296/0 (socket says 172.21.15.59:57296) 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6554c00640 1 -- 172.21.15.59:0/4156873939 learned_addr learned my addr 172.21.15.59:0/4156873939 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.783+0000 7f6556000640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 0x7f65501ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.784+0000 7f6554c00640 1 -- 172.21.15.59:0/4156873939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 msgr2=0x7f65501ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.784+0000 7f6554c00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 0x7f65501ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.784+0000 7f6554c00640 1 -- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6550102450 msgr2=0x7f6550110dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.784+0000 7f6554c00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6550102450 0x7f6550110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.784+0000 7f6554c00640 1 -- 172.21.15.59:0/4156873939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501ae6d0 con 0x7f6550104d80 2024-04-02T17:32:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.785+0000 7f6554c00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e9a0 tx=0x7f653c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.785+0000 7f654e000640 1 -- 172.21.15.59:0/4156873939 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c00cd20 con 0x7f6550104d80 2024-04-02T17:32:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.785+0000 7f654e000640 1 -- 172.21.15.59:0/4156873939 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c0035f0 con 0x7f6550104d80 2024-04-02T17:32:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.785+0000 7f654e000640 1 -- 172.21.15.59:0/4156873939 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c010690 con 0x7f6550104d80 2024-04-02T17:32:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.786+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501ae9c0 con 0x7f6550104d80 2024-04-02T17:32:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.786+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501aeec0 con 0x7f6550104d80 2024-04-02T17:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.795+0000 7f655ae3a740 1 Processor -- start 2024-04-02T17:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.795+0000 7f654e000640 1 -- 172.21.15.59:0/4156873939 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f653c003b10 con 0x7f6550104d80 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 -- start start 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fbc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fbcc60 0x563f97fbf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f97f1cd70 con 0x563f97fb7100 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563f97ec52a0 con 0x563f97fbcc60 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.796+0000 7f655ae3a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563f97f18d20 con 0x563f97fb7af0 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.797+0000 7f6556000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.797+0000 7f6556000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44192/0 (socket says 172.21.15.59:44192) 2024-04-02T17:32:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.797+0000 7f6556000640 1 -- 172.21.15.59:0/2469992564 learned_addr learned my addr 172.21.15.59:0/2469992564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.798+0000 7f6554c00640 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fbcc60 0x563f97fbf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.798+0000 7f6556000640 1 -- 172.21.15.59:0/2469992564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 msgr2=0x563f97fbc5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:32:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.798+0000 7f6556000640 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fbc5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.798+0000 7f6556000640 1 -- 172.21.15.59:0/2469992564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fbcc60 msgr2=0x563f97fbf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.798+0000 7f6556000640 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fbcc60 0x563f97fbf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.799+0000 7f6556000640 1 -- 172.21.15.59:0/2469992564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f97ed9840 con 0x563f97fb7100 2024-04-02T17:32:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.799+0000 7f654e000640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 0x7f65380a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.800+0000 7f654e000640 1 -- 172.21.15.59:0/4156873939 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f653c0d0670 con 0x7f6550104d80 2024-04-02T17:32:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.800+0000 7f6556000640 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6548009fd0 tx=0x7f65480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e4ff14e0bcea0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.801+0000 7f6542600640 1 -- 172.21.15.59:0/2469992564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548004d20 con 0x563f97fb7100 2024-04-02T17:32:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.801+0000 7f6542600640 1 -- 172.21.15.59:0/2469992564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548004050 con 0x563f97fb7100 2024-04-02T17:32:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.801+0000 7f6542600640 1 -- 172.21.15.59:0/2469992564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548005240 con 0x563f97fb7100 2024-04-02T17:32:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.801+0000 7f6555600640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 0x7f65380a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 -- 172.21.15.59:0/2469992564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 msgr2=0x563f97fb7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6548009fd0 tx=0x7f65480118d0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 -- 172.21.15.59:0/2469992564 shutdown_connections 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fbcc60 0x563f97fbf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fbc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 --2- 172.21.15.59:0/2469992564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fb7520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.805+0000 7f655ae3a740 1 -- 172.21.15.59:0/2469992564 >> 172.21.15.59:0/2469992564 conn(0x563f97fb2670 msgr2=0x563f97fb2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.804+0000 7f6555600640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 0x7f65380a6230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6544009230 tx=0x7f6544000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.806+0000 7f655ae3a740 1 -- 172.21.15.59:0/2469992564 shutdown_connections 2024-04-02T17:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.807+0000 7f655ae3a740 1 -- 172.21.15.59:0/2469992564 wait complete. 2024-04-02T17:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.808+0000 7f655ae3a740 1 -- start start 2024-04-02T17:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.808+0000 7f655ae3a740 1 Processor -- start 2024-04-02T17:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.809+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.809+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f6555600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f6555600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57302/0 (socket says 172.21.15.59:57302) 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f6555600640 1 -- 172.21.15.59:0/3927527885 learned_addr learned my addr 172.21.15.59:0/3927527885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f97fbfc10 con 0x563f97fbcc60 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563f97ed8660 con 0x563f97fb7af0 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563f97fbfd90 con 0x563f97fb7100 2024-04-02T17:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f655ae3a740 10 client.0 _ll_register_callbacks cb 0x563f97ed8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.810+0000 7f6556000640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6554c00640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6555600640 1 -- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 msgr2=0x563f97fc1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6555600640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6555600640 1 -- 172.21.15.59:0/3927527885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 msgr2=0x563f97fc8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6555600640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6555600640 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f97ee7e80 con 0x563f97fb7100 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.811+0000 7f6554c00640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.812+0000 7f6555600640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f6544009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52934b553e59884 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.812+0000 7f6536a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.812+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654404f060 con 0x563f97fb7100 2024-04-02T17:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.812+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f654402cc50 con 0x563f97fb7100 2024-04-02T17:32:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.812+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6548004050 con 0x563f97fb7100 2024-04-02T17:32:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.813+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6544039040 con 0x563f97fb7100 2024-04-02T17:32:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.813+0000 7f655ae3a740 10 client.27436 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.813+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563f97c31210 con 0x563f97fb7100 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.814+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548004050 con 0x563f97fb7100 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.814+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f6544053030 con 0x563f97fb7100 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.814+0000 7f655ae3a740 10 client.27436 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.814+0000 7f655ae3a740 10 client.27436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.814+0000 7f655ae3a740 20 client.27436 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.815+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f97fcd4f0 0x563f97fcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.815+0000 7f655ae3a740 4 client.27436 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:32:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.815+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f97ecccd0 con 0x563f97fcd4f0 2024-04-02T17:32:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.815+0000 7f6554c00640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f97fcd4f0 0x563f97fcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.816+0000 7f6554c00640 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f97fcd4f0 0x563f97fcf970 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4adc60cfe93fbf83 server_cookie=f0e846137f040562 in_seq=0 out_seq=0 2024-04-02T17:32:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.816+0000 7f6536a00640 10 client.27436 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.817+0000 7f6536a00640 1 -- 172.21.15.59:0/3927527885 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f97ecccd0 con 0x563f97fcd4f0 2024-04-02T17:32:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.817+0000 7f6536a00640 10 client.27436 handle_command_reply: tid=0 2024-04-02T17:32:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.817+0000 7f655ae3a740 1 client.27436 shutdown 2024-04-02T17:32:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.818+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 msgr2=0x563f97fc1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.818+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f6544009260 comp rx=0 tx=0).stop 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 shutdown_connections 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f97fcd4f0 0x563f97fcf970 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 --2- 172.21.15.59:0/3927527885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7100 0x563f97fc1670 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 >> 172.21.15.59:0/3927527885 conn(0x563f97fb2670 msgr2=0x563f97fb1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f6536a00640 0 client.27436 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 shutdown_connections 2024-04-02T17:32:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 1 -- 172.21.15.59:0/3927527885 wait complete. 2024-04-02T17:32:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.819+0000 7f655ae3a740 20 client.27436 trim_cache size 0 max 16384 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 Processor -- start 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 -- start start 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 0x563f97fb2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b7ee0 con 0x563f97fbcc60 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b9a20 con 0x563f97fb7100 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.847+0000 7f655ae3a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b85b0 con 0x563f97fb7af0 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6555600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 0x563f97fb2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:57310/0 (socket says 172.21.15.59:57310) 2024-04-02T17:32:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 -- 172.21.15.59:0/1377229754 learned_addr learned my addr 172.21.15.59:0/1377229754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6556000640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 -- 172.21.15.59:0/1377229754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 msgr2=0x563f97fb2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 0x563f97fb2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.848+0000 7f6554c00640 1 -- 172.21.15.59:0/1377229754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 msgr2=0x563f97fc1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6554c00640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6554c00640 1 -- 172.21.15.59:0/1377229754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f97cfd620 con 0x563f97fb7af0 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6555600640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 0x563f97fb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6556000640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6554c00640 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f653c098cf0 tx=0x7f653c0cdf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33eeaf85a6d0c5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6541c00640 1 -- 172.21.15.59:0/1377229754 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c09c890 con 0x563f97fb7af0 2024-04-02T17:32:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6541c00640 1 -- 172.21.15.59:0/1377229754 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c0a0c40 con 0x563f97fb7af0 2024-04-02T17:32:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.849+0000 7f6541c00640 1 -- 172.21.15.59:0/1377229754 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c098f00 con 0x563f97fb7af0 2024-04-02T17:32:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.851+0000 7f655ae3a740 1 -- 172.21.15.59:0/1377229754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 msgr2=0x563f97fc8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.851+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f653c098cf0 tx=0x7f653c0cdf80 comp rx=0 tx=0).stop 2024-04-02T17:32:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 -- 172.21.15.59:0/1377229754 shutdown_connections 2024-04-02T17:32:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fbcc60 0x563f97fc1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fb7af0 0x563f97fc8bb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1377229754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7100 0x563f97fb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 -- 172.21.15.59:0/1377229754 >> 172.21.15.59:0/1377229754 conn(0x563f97fb2670 msgr2=0x563f97fc0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 -- 172.21.15.59:0/1377229754 shutdown_connections 2024-04-02T17:32:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.852+0000 7f655ae3a740 1 -- 172.21.15.59:0/1377229754 wait complete. 2024-04-02T17:32:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.854+0000 7f655ae3a740 1 -- start start 2024-04-02T17:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.854+0000 7f655ae3a740 1 Processor -- start 2024-04-02T17:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 0x563f98031ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b9350 con 0x563f97fb7100 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b8c80 con 0x563f97fb7af0 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65501b5ec0 con 0x563f97fbcc60 2024-04-02T17:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.855+0000 7f655ae3a740 10 client.0 _ll_register_callbacks cb 0x7f6544009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6554c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6554c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51056/0 (socket says 172.21.15.59:51056) 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6554c00640 1 -- 172.21.15.59:0/1494858672 learned_addr learned my addr 172.21.15.59:0/1494858672 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 0x563f98031ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6555600640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 -- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 msgr2=0x563f97fc6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 -- 172.21.15.59:0/1494858672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 msgr2=0x563f97fc5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fc5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.856+0000 7f6556000640 1 -- 172.21.15.59:0/1494858672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f98032020 con 0x563f97fbcc60 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f6554c00640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f6556000640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 0x563f98031ae0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6548006010 tx=0x7f6548003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c878062d5e0944 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f6536a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548003c70 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f6548004050 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.857+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653c0a0c40 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6548035050 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 10 client.27442 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563f98033300 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548003e10 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f65480057e0 con 0x563f97fbcc60 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 10 client.27442 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 10 client.27442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 20 client.27442 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f98033ad0 0x563f98035ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 4 client.27442 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:32:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.858+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f653c0a0c40 con 0x563f98033ad0 2024-04-02T17:32:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.859+0000 7f6555600640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f98033ad0 0x563f98035ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.860+0000 7f6555600640 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f98033ad0 0x563f98035ef0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a44bd4804f74fa server_cookie=f5e31d224b92e1d4 in_seq=0 out_seq=0 2024-04-02T17:32:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:08.860+0000 7f6536a00640 10 client.27442 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.123+0000 7f6536a00640 1 -- 172.21.15.59:0/1494858672 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f653c0a0c40 con 0x563f98033ad0 2024-04-02T17:32:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.124+0000 7f6536a00640 10 client.27442 handle_command_reply: tid=0 2024-04-02T17:32:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.124+0000 7f655ae3a740 1 client.27442 shutdown 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.124+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 msgr2=0x563f98031ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.124+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 0x563f98031ae0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6548006010 tx=0x7f6548003970 comp rx=0 tx=0).stop 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 shutdown_connections 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x563f98033ad0 0x563f98035ef0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563f97fbcc60 0x563f98031ae0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563f97fb7af0 0x563f97fc6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 --2- 172.21.15.59:0/1494858672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f97fb7100 0x563f97fc5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 >> 172.21.15.59:0/1494858672 conn(0x563f97fb2670 msgr2=0x563f97fb2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.125+0000 7f6536a00640 0 client.27442 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.126+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 shutdown_connections 2024-04-02T17:32:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.126+0000 7f655ae3a740 1 -- 172.21.15.59:0/1494858672 wait complete. 2024-04-02T17:32:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.126+0000 7f655ae3a740 20 client.27442 trim_cache size 0 max 16384 2024-04-02T17:32:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:32:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:32:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:32:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:32:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.127+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 msgr2=0x7f65380a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.127+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 0x7f65380a6230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6544009230 tx=0x7f6544000bc0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.127+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 msgr2=0x7f6550111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.127+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e9a0 tx=0x7f653c00ee60 comp rx=0 tx=0).stop 2024-04-02T17:32:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 shutdown_connections 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65501076b0 0x7f65501ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6550104d80 0x7f6550111310 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f65380a3d70 0x7f65380a6230 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 --2- 172.21.15.59:0/4156873939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6550102450 0x7f6550110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 >> 172.21.15.59:0/4156873939 conn(0x7f65500fbde0 msgr2=0x7f6550102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 shutdown_connections 2024-04-02T17:32:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:09.128+0000 7f6556a00640 1 -- 172.21.15.59:0/4156873939 wait complete. 2024-04-02T17:32:09.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 260587 40.41 MB/sec execute 216 sec latency 132.789 ms 2024-04-02T17:32:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:10 smithi154 ceph-mon[36479]: pgmap v962: 97 pgs: 97 active+clean; 1001 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 542 op/s 2024-04-02T17:32:10.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:10 smithi059 ceph-mon[31664]: pgmap v962: 97 pgs: 97 active+clean; 1001 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 542 op/s 2024-04-02T17:32:10.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:10 smithi164 ceph-mon[36499]: pgmap v962: 97 pgs: 97 active+clean; 1001 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.1 MiB/s wr, 542 op/s 2024-04-02T17:32:10.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 261387 40.37 MB/sec execute 217 sec latency 62.496 ms 2024-04-02T17:32:11.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 262196 40.41 MB/sec execute 218 sec latency 66.054 ms 2024-04-02T17:32:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:12 smithi154 ceph-mon[36479]: pgmap v963: 97 pgs: 97 active+clean; 1003 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-02T17:32:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:12 smithi059 ceph-mon[31664]: pgmap v963: 97 pgs: 97 active+clean; 1003 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-02T17:32:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:12 smithi164 ceph-mon[36499]: pgmap v963: 97 pgs: 97 active+clean; 1003 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-02T17:32:12.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 262846 40.41 MB/sec execute 219 sec latency 110.790 ms 2024-04-02T17:32:13.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 263433 40.31 MB/sec execute 220 sec latency 241.732 ms 2024-04-02T17:32:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:14 smithi154 ceph-mon[36479]: pgmap v964: 97 pgs: 97 active+clean; 1016 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 663 op/s 2024-04-02T17:32:14.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:14 smithi059 ceph-mon[31664]: pgmap v964: 97 pgs: 97 active+clean; 1016 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 663 op/s 2024-04-02T17:32:14.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:14 smithi164 ceph-mon[36499]: pgmap v964: 97 pgs: 97 active+clean; 1016 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 663 op/s 2024-04-02T17:32:14.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 264286 40.37 MB/sec execute 221 sec latency 119.208 ms 2024-04-02T17:32:15.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 265209 40.43 MB/sec execute 222 sec latency 51.406 ms 2024-04-02T17:32:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:16 smithi154 ceph-mon[36479]: pgmap v965: 97 pgs: 97 active+clean; 1020 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.2 MiB/s wr, 623 op/s 2024-04-02T17:32:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:16 smithi059 ceph-mon[31664]: pgmap v965: 97 pgs: 97 active+clean; 1020 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.2 MiB/s wr, 623 op/s 2024-04-02T17:32:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:16 smithi164 ceph-mon[36499]: pgmap v965: 97 pgs: 97 active+clean; 1020 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.2 MiB/s wr, 623 op/s 2024-04-02T17:32:16.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 265841 40.41 MB/sec execute 223 sec latency 82.231 ms 2024-04-02T17:32:17.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 266186 40.30 MB/sec execute 224 sec latency 335.920 ms 2024-04-02T17:32:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:18 smithi154 ceph-mon[36479]: pgmap v966: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 690 op/s 2024-04-02T17:32:18.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:18 smithi059 ceph-mon[31664]: pgmap v966: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 690 op/s 2024-04-02T17:32:18.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:18 smithi164 ceph-mon[36499]: pgmap v966: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 690 op/s 2024-04-02T17:32:18.627 INFO:tasks.workunit.client.0.smithi059.stdout: 10 266605 40.19 MB/sec execute 225 sec latency 560.386 ms 2024-04-02T17:32:19.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 267376 40.18 MB/sec execute 226 sec latency 66.743 ms 2024-04-02T17:32:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:20 smithi154 ceph-mon[36479]: pgmap v967: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 598 op/s 2024-04-02T17:32:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:20 smithi059 ceph-mon[31664]: pgmap v967: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 598 op/s 2024-04-02T17:32:20.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:20 smithi164 ceph-mon[36499]: pgmap v967: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 598 op/s 2024-04-02T17:32:20.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 268430 40.29 MB/sec execute 227 sec latency 102.665 ms 2024-04-02T17:32:21.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 269166 40.33 MB/sec execute 228 sec latency 38.277 ms 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 -- 172.21.15.59:0/30222849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 msgr2=0x7f12881053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 --2- 172.21.15.59:0/30222849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f12881053c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1274009900 tx=0x7f127402bc40 comp rx=0 tx=0).stop 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 -- 172.21.15.59:0/30222849 shutdown_connections 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 --2- 172.21.15.59:0/30222849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f128810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 --2- 172.21.15.59:0/30222849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f12881053c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 --2- 172.21.15.59:0/30222849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f1288104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 -- 172.21.15.59:0/30222849 >> 172.21.15.59:0/30222849 conn(0x7f12880ff2c0 msgr2=0x7f1288101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 -- 172.21.15.59:0/30222849 shutdown_connections 2024-04-02T17:32:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.314+0000 7f128cc00640 1 -- 172.21.15.59:0/30222849 wait complete. 2024-04-02T17:32:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.315+0000 7f128cc00640 1 Processor -- start 2024-04-02T17:32:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.315+0000 7f128cc00640 1 -- start start 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f128819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f128819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f12881a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f1285e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f128819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f128810fa00 con 0x7f1288105900 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f128810f880 con 0x7f1288104f40 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f128cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f128810fb80 con 0x7f1288103d50 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f1285e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f128819e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49550/0 (socket says 172.21.15.59:49550) 2024-04-02T17:32:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f1285e00640 1 -- 172.21.15.59:0/2148006810 learned_addr learned my addr 172.21.15.59:0/2148006810 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.316+0000 7f1285400640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f128819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1286800640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f12881a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 -- 172.21.15.59:0/2148006810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 msgr2=0x7f128819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f128819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 -- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 msgr2=0x7f12881a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f12881a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 -- 172.21.15.59:0/2148006810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12881a5fd0 con 0x7f1288104f40 2024-04-02T17:32:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1286800640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f12881a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.317+0000 7f1285400640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f128819e8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1274036040 tx=0x7f127402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.318+0000 7f127ea00640 1 -- 172.21.15.59:0/2148006810 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274041070 con 0x7f1288104f40 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.318+0000 7f127ea00640 1 -- 172.21.15.59:0/2148006810 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1274004450 con 0x7f1288104f40 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.318+0000 7f127ea00640 1 -- 172.21.15.59:0/2148006810 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274003e60 con 0x7f1288104f40 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.318+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881a6260 con 0x7f1288104f40 2024-04-02T17:32:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.318+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a66d0 con 0x7f1288104f40 2024-04-02T17:32:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.322+0000 7f127ea00640 1 -- 172.21.15.59:0/2148006810 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1274044070 con 0x7f1288104f40 2024-04-02T17:32:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.325+0000 7f127ea00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 0x7f126c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.325+0000 7f127ea00640 1 -- 172.21.15.59:0/2148006810 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f12740f1ec0 con 0x7f1288104f40 2024-04-02T17:32:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.326+0000 7f1285e00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 0x7f126c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.326+0000 7f128f82d740 1 Processor -- start 2024-04-02T17:32:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.326+0000 7f128f82d740 1 -- start start 2024-04-02T17:32:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f1285e00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 0x7f126c0a6280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12780058c0 tx=0x7f1278005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b8fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557327aedd70 con 0x557327b87ed0 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557327a962a0 con 0x557327b888c0 2024-04-02T17:32:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.327+0000 7f128f82d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557327ae9d20 con 0x557327b8da30 2024-04-02T17:32:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56930/0 (socket says 172.21.15.59:56930) 2024-04-02T17:32:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 -- 172.21.15.59:0/2397680719 learned_addr learned my addr 172.21.15.59:0/2397680719 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1286800640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b8fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285e00640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 -- 172.21.15.59:0/2397680719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 msgr2=0x557327b8fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b8fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 -- 172.21.15.59:0/2397680719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 msgr2=0x557327b882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285400640 1 -- 172.21.15.59:0/2397680719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557327aaa840 con 0x557327b888c0 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1285e00640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.328+0000 7f1286800640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b8fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.329+0000 7f1285400640 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12740ba580 tx=0x7f1274005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e7b526a6adcb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.329+0000 7f1273000640 1 -- 172.21.15.59:0/2397680719 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127404b040 con 0x557327b888c0 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.329+0000 7f1273000640 1 -- 172.21.15.59:0/2397680719 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12740be120 con 0x557327b888c0 2024-04-02T17:32:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.329+0000 7f1273000640 1 -- 172.21.15.59:0/2397680719 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740c2120 con 0x557327b888c0 2024-04-02T17:32:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 -- 172.21.15.59:0/2397680719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 msgr2=0x557327b8d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f12740ba580 tx=0x7f1274005e50 comp rx=0 tx=0).stop 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 -- 172.21.15.59:0/2397680719 shutdown_connections 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b8fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b888c0 0x557327b8d3c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 --2- 172.21.15.59:0/2397680719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 -- 172.21.15.59:0/2397680719 >> 172.21.15.59:0/2397680719 conn(0x557327b83440 msgr2=0x557327b83840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 -- 172.21.15.59:0/2397680719 shutdown_connections 2024-04-02T17:32:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.330+0000 7f128f82d740 1 -- 172.21.15.59:0/2397680719 wait complete. 2024-04-02T17:32:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.331+0000 7f128f82d740 1 -- start start 2024-04-02T17:32:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.331+0000 7f128f82d740 1 Processor -- start 2024-04-02T17:32:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 0x557327b99630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557327a962a0 con 0x557327b8da30 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557327aa9660 con 0x557327b87ed0 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557327b907a0 con 0x557327b888c0 2024-04-02T17:32:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f128f82d740 10 client.0 _ll_register_callbacks cb 0x557327aa9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f1285400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f1285400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49576/0 (socket says 172.21.15.59:49576) 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f1285400640 1 -- 172.21.15.59:0/3005000238 learned_addr learned my addr 172.21.15.59:0/3005000238 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f1285e00640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.332+0000 7f1286800640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 0x557327b99630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 -- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 msgr2=0x557327b920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b920b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 -- 172.21.15.59:0/3005000238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 msgr2=0x557327b99630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 0x557327b99630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 -- 172.21.15.59:0/3005000238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557327ab8e80 con 0x557327b888c0 2024-04-02T17:32:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1286800640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 0x557327b99630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1285400640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f127402c8f0 tx=0x7f127402bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a03bc7bce87fa1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1267400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740dc060 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12740be120 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f12740c2120 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f127404b040 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f128f82d740 10 client.27454 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.333+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557327802210 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12740be120 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f12740d2bc0 con 0x557327b888c0 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 10 client.27454 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 10 client.27454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 20 client.27454 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327b9dd70 0x557327ba0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 4 client.27454 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:32:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557327a9dcd0 con 0x557327b9dd70 2024-04-02T17:32:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.334+0000 7f1285e00640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327b9dd70 0x557327ba0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.335+0000 7f1285e00640 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327b9dd70 0x557327ba0190 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f001f46860ac0fa server_cookie=d67dc37e5b67aa92 in_seq=0 out_seq=0 2024-04-02T17:32:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.335+0000 7f1267400640 10 client.27454 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.337+0000 7f1267400640 1 -- 172.21.15.59:0/3005000238 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557327a9dcd0 con 0x557327b9dd70 2024-04-02T17:32:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.337+0000 7f1267400640 10 client.27454 handle_command_reply: tid=0 2024-04-02T17:32:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.337+0000 7f128f82d740 1 client.27454 shutdown 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 msgr2=0x557327b925f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f127402c8f0 tx=0x7f127402bc40 comp rx=0 tx=0).stop 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 shutdown_connections 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327b9dd70 0x557327ba0190 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b8da30 0x557327b99630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b925f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 --2- 172.21.15.59:0/3005000238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 >> 172.21.15.59:0/3005000238 conn(0x557327b83440 msgr2=0x557327b82640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f1267400640 0 client.27454 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.338+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 shutdown_connections 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.339+0000 7f128f82d740 1 -- 172.21.15.59:0/3005000238 wait complete. 2024-04-02T17:32:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.339+0000 7f128f82d740 20 client.27454 trim_cache size 0 max 16384 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 Processor -- start 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 -- start start 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b888c0 0x557327b995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12881af710 con 0x557327b888c0 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12881b1250 con 0x557327b87ed0 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.368+0000 7f128f82d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12881afde0 con 0x557327b8da30 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56948/0 (socket says 172.21.15.59:56948) 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 -- 172.21.15.59:0/3646202216 learned_addr learned my addr 172.21.15.59:0/3646202216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1286800640 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285400640 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b888c0 0x557327b995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 -- 172.21.15.59:0/3646202216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 msgr2=0x557327b916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 -- 172.21.15.59:0/3646202216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b888c0 msgr2=0x557327b995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b888c0 0x557327b995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.369+0000 7f1285e00640 1 -- 172.21.15.59:0/3646202216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573278ce620 con 0x557327b87ed0 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.370+0000 7f1285e00640 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1278011ca0 tx=0x7f1278014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a73202ac6bd5c40e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.370+0000 7f1272600640 1 -- 172.21.15.59:0/3646202216 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278005020 con 0x557327b87ed0 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.370+0000 7f1272600640 1 -- 172.21.15.59:0/3646202216 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12780047a0 con 0x557327b87ed0 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.370+0000 7f1272600640 1 -- 172.21.15.59:0/3646202216 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802b470 con 0x557327b87ed0 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.371+0000 7f128f82d740 1 -- 172.21.15.59:0/3646202216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 msgr2=0x557327b9a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.371+0000 7f128f82d740 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1278011ca0 tx=0x7f1278014000 comp rx=0 tx=0).stop 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 -- 172.21.15.59:0/3646202216 shutdown_connections 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b888c0 0x557327b995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b8da30 0x557327b916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 --2- 172.21.15.59:0/3646202216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b87ed0 0x557327b9a790 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 -- 172.21.15.59:0/3646202216 >> 172.21.15.59:0/3646202216 conn(0x557327b83440 msgr2=0x557327b9a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 -- 172.21.15.59:0/3646202216 shutdown_connections 2024-04-02T17:32:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.372+0000 7f128f82d740 1 -- 172.21.15.59:0/3646202216 wait complete. 2024-04-02T17:32:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.373+0000 7f128f82d740 1 -- start start 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.373+0000 7f128f82d740 1 Processor -- start 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 0x557327c02280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12881b0b80 con 0x557327b87ed0 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12881b04b0 con 0x557327b8da30 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ad6f0 con 0x557327b888c0 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f128f82d740 10 client.0 _ll_register_callbacks cb 0x7f12740be8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f1285400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f1285400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49592/0 (socket says 172.21.15.59:49592) 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f1285400640 1 -- 172.21.15.59:0/4074346846 learned_addr learned my addr 172.21.15.59:0/4074346846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.374+0000 7f1285e00640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 -- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 msgr2=0x557327c02280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1286800640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 0x557327c02280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 0x557327c02280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 -- 172.21.15.59:0/4074346846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 msgr2=0x557327b965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 -- 172.21.15.59:0/4074346846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557327c137d0 con 0x557327b888c0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1286800640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 0x557327c02280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285e00640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1285400640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f12740bc3d0 tx=0x7f12740c6bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8633643f180bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.375+0000 7f1267400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.376+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740bda80 con 0x557327b888c0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.376+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f12740cd050 con 0x557327b888c0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.376+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12780047a0 con 0x557327b888c0 2024-04-02T17:32:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12740be120 con 0x557327b888c0 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 10 client.27460 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557327c137d0 con 0x557327b888c0 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12780047a0 con 0x557327b888c0 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f12740035f0 con 0x557327b888c0 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 10 client.27460 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 10 client.27460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 20 client.27460 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327c150b0 0x557327c174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 4 client.27460 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:32:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.377+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557327b91ea0 con 0x557327c150b0 2024-04-02T17:32:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.378+0000 7f1285e00640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327c150b0 0x557327c174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.380+0000 7f1285e00640 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327c150b0 0x557327c174d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db3cae536cd8df7 server_cookie=c509b0876a25769f in_seq=0 out_seq=0 2024-04-02T17:32:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.380+0000 7f1267400640 10 client.27460 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:22 smithi154 ceph-mon[36479]: pgmap v968: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-02T17:32:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:22 smithi059 ceph-mon[31664]: pgmap v968: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-02T17:32:22.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:22 smithi164 ceph-mon[36499]: pgmap v968: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-02T17:32:22.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 269591 40.23 MB/sec execute 229 sec latency 182.273 ms 2024-04-02T17:32:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.758+0000 7f1267400640 1 -- 172.21.15.59:0/4074346846 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557327b91ea0 con 0x557327c150b0 2024-04-02T17:32:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.758+0000 7f1267400640 10 client.27460 handle_command_reply: tid=0 2024-04-02T17:32:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.758+0000 7f128f82d740 1 client.27460 shutdown 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 msgr2=0x557327b96b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f12740bc3d0 tx=0x7f12740c6bc0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 shutdown_connections 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x557327c150b0 0x557327c174d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557327b8da30 0x557327c02280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327b888c0 0x557327b96b30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 --2- 172.21.15.59:0/4074346846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557327b87ed0 0x557327b965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f1267400640 0 client.27460 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 >> 172.21.15.59:0/4074346846 conn(0x557327b83440 msgr2=0x557327b9a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.759+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 shutdown_connections 2024-04-02T17:32:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.760+0000 7f128f82d740 1 -- 172.21.15.59:0/4074346846 wait complete. 2024-04-02T17:32:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.760+0000 7f128f82d740 20 client.27460 trim_cache size 0 max 16384 2024-04-02T17:32:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:32:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:32:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:32:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 msgr2=0x7f126c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 0x7f126c0a6280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12780058c0 tx=0x7f1278005f90 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 msgr2=0x7f128819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f128819e8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1274036040 tx=0x7f127402c660 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 shutdown_connections 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f126c0a3dc0 0x7f126c0a6280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1288105900 0x7f12881a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1288104f40 0x7f128819e8f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.762+0000 7f128cc00640 1 --2- 172.21.15.59:0/2148006810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1288103d50 0x7f128819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.763+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 >> 172.21.15.59:0/2148006810 conn(0x7f12880ff2c0 msgr2=0x7f1288077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.765+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 shutdown_connections 2024-04-02T17:32:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:22.765+0000 7f128cc00640 1 -- 172.21.15.59:0/2148006810 wait complete. 2024-04-02T17:32:23.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 269738 40.09 MB/sec execute 230 sec latency 702.393 ms 2024-04-02T17:32:24.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:24 smithi154 ceph-mon[36479]: pgmap v969: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 833 op/s 2024-04-02T17:32:24.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:24 smithi059 ceph-mon[31664]: pgmap v969: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 833 op/s 2024-04-02T17:32:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:24 smithi164 ceph-mon[36499]: pgmap v969: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 833 op/s 2024-04-02T17:32:24.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 269914 39.94 MB/sec execute 231 sec latency 870.110 ms 2024-04-02T17:32:25.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 270353 39.82 MB/sec execute 232 sec latency 164.071 ms 2024-04-02T17:32:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:26 smithi154 ceph-mon[36479]: pgmap v970: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 839 op/s 2024-04-02T17:32:26.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:26 smithi059 ceph-mon[31664]: pgmap v970: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 839 op/s 2024-04-02T17:32:26.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:26 smithi164 ceph-mon[36499]: pgmap v970: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 839 op/s 2024-04-02T17:32:26.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 271026 39.82 MB/sec execute 233 sec latency 110.481 ms 2024-04-02T17:32:27.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 271723 39.82 MB/sec execute 234 sec latency 445.143 ms 2024-04-02T17:32:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:28 smithi154 ceph-mon[36479]: pgmap v971: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-02T17:32:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:28 smithi059 ceph-mon[31664]: pgmap v971: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-02T17:32:28.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:28 smithi164 ceph-mon[36499]: pgmap v971: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-02T17:32:28.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 272390 39.87 MB/sec execute 235 sec latency 680.403 ms 2024-04-02T17:32:29.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 272991 39.83 MB/sec execute 236 sec latency 37.213 ms 2024-04-02T17:32:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:30 smithi154 ceph-mon[36479]: pgmap v972: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-04-02T17:32:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:30 smithi059 ceph-mon[31664]: pgmap v972: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-04-02T17:32:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:30 smithi164 ceph-mon[36499]: pgmap v972: 97 pgs: 97 active+clean; 1.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 19 MiB/s wr, 967 op/s 2024-04-02T17:32:30.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 273623 39.76 MB/sec execute 237 sec latency 70.281 ms 2024-04-02T17:32:31.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 274722 39.82 MB/sec execute 238 sec latency 41.264 ms 2024-04-02T17:32:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:32 smithi154 ceph-mon[36479]: pgmap v973: 97 pgs: 97 active+clean; 1008 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-02T17:32:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:32 smithi059 ceph-mon[31664]: pgmap v973: 97 pgs: 97 active+clean; 1008 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-02T17:32:32.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:32 smithi164 ceph-mon[36499]: pgmap v973: 97 pgs: 97 active+clean; 1008 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-02T17:32:32.628 INFO:tasks.workunit.client.0.smithi059.stdout: 10 275765 39.96 MB/sec execute 239 sec latency 41.673 ms 2024-04-02T17:32:33.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 276361 39.92 MB/sec execute 240 sec latency 42.479 ms 2024-04-02T17:32:34.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:34 smithi154 ceph-mon[36479]: pgmap v974: 97 pgs: 97 active+clean; 1019 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 947 op/s 2024-04-02T17:32:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:34 smithi059 ceph-mon[31664]: pgmap v974: 97 pgs: 97 active+clean; 1019 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 947 op/s 2024-04-02T17:32:34.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:34 smithi164 ceph-mon[36499]: pgmap v974: 97 pgs: 97 active+clean; 1019 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 947 op/s 2024-04-02T17:32:34.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 276952 39.86 MB/sec execute 241 sec latency 48.099 ms 2024-04-02T17:32:35.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 277949 39.87 MB/sec execute 242 sec latency 46.107 ms 2024-04-02T17:32:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 -- 172.21.15.59:0/604567143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0598107630 msgr2=0x7f0598109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 --2- 172.21.15.59:0/604567143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0598107630 0x7f0598109a80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0590009be0 tx=0x7f059002bef0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 -- 172.21.15.59:0/604567143 shutdown_connections 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 --2- 172.21.15.59:0/604567143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0598107630 0x7f0598109a80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 --2- 172.21.15.59:0/604567143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 0x7f05981070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 --2- 172.21.15.59:0/604567143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05981023d0 0x7f05981047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 -- 172.21.15.59:0/604567143 >> 172.21.15.59:0/604567143 conn(0x7f05980fbde0 msgr2=0x7f05980fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 -- 172.21.15.59:0/604567143 shutdown_connections 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.919+0000 7f059ea00640 1 -- 172.21.15.59:0/604567143 wait complete. 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.920+0000 7f059ea00640 1 Processor -- start 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.920+0000 7f059ea00640 1 -- start start 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05981023d0 0x7f0598197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 0x7f0598198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d5d0 con 0x7f0598107630 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d450 con 0x7f05981023d0 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d750 con 0x7f0598104d00 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05981023d0 0x7f0598197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43856/0 (socket says 172.21.15.59:43856) 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 -- 172.21.15.59:0/2307681267 learned_addr learned my addr 172.21.15.59:0/2307681267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059cc00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 0x7f0598198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 -- 172.21.15.59:0/2307681267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 msgr2=0x7f0598198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 0x7f0598198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 -- 172.21.15.59:0/2307681267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05981023d0 msgr2=0x7f0598197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05981023d0 0x7f0598197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.921+0000 7f059e000640 1 -- 172.21.15.59:0/2307681267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819f840 con 0x7f0598107630 2024-04-02T17:32:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f059e000640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f05900337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f0596000640 1 -- 172.21.15.59:0/2307681267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590043020 con 0x7f0598107630 2024-04-02T17:32:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f0596000640 1 -- 172.21.15.59:0/2307681267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059002ca60 con 0x7f0598107630 2024-04-02T17:32:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819fad0 con 0x7f0598107630 2024-04-02T17:32:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f0596000640 1 -- 172.21.15.59:0/2307681267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590034700 con 0x7f0598107630 2024-04-02T17:32:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.922+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a0070 con 0x7f0598107630 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.925+0000 7f0596000640 1 -- 172.21.15.59:0/2307681267 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0590046020 con 0x7f0598107630 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.925+0000 7f05a2dca740 1 Processor -- start 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.925+0000 7f05a2dca740 1 -- start start 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dc54d0 0x55a874dc7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a874d1cd70 con 0x55a874dc0910 2024-04-02T17:32:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a874cc52a0 con 0x55a874dbff20 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f05a2dca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a874d18d20 con 0x55a874dc54d0 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f059cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f059d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f059cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38296/0 (socket says 172.21.15.59:38296) 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f059cc00640 1 -- 172.21.15.59:0/547106550 learned_addr learned my addr 172.21.15.59:0/547106550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.926+0000 7f059cc00640 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dc54d0 0x55a874dc7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 -- 172.21.15.59:0/547106550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dc54d0 msgr2=0x55a874dc7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dc54d0 0x55a874dc7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 -- 172.21.15.59:0/547106550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 msgr2=0x55a874dc4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 -- 172.21.15.59:0/547106550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a874cd9840 con 0x55a874dbff20 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f0596000640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 0x7f05800a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059cc00640 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05840058c0 tx=0x7f0584005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498b3d1b82860e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f0596000640 1 -- 172.21.15.59:0/2307681267 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f05900f2aa0 con 0x7f0598107630 2024-04-02T17:32:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f058a600640 1 -- 172.21.15.59:0/547106550 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584004030 con 0x55a874dbff20 2024-04-02T17:32:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f058a600640 1 -- 172.21.15.59:0/547106550 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0584005500 con 0x55a874dbff20 2024-04-02T17:32:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f058a600640 1 -- 172.21.15.59:0/547106550 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584004ce0 con 0x55a874dbff20 2024-04-02T17:32:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.927+0000 7f059d600640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 0x7f05800a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- 172.21.15.59:0/547106550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 msgr2=0x55a874dc0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05840058c0 tx=0x7f0584005850 comp rx=0 tx=0).stop 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f059d600640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 0x7f05800a60e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55a874dc0d10 tx=0x7f058c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- 172.21.15.59:0/547106550 shutdown_connections 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dc54d0 0x55a874dc7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 --2- 172.21.15.59:0/547106550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dbff20 0x55a874dc0340 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- 172.21.15.59:0/547106550 >> 172.21.15.59:0/547106550 conn(0x55a874db25f0 msgr2=0x55a874db29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- 172.21.15.59:0/547106550 shutdown_connections 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- 172.21.15.59:0/547106550 wait complete. 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.928+0000 7f05a2dca740 1 -- start start 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 Processor -- start 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc54d0 0x55a874dd1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a874dc8480 con 0x55a874dc54d0 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a874cd8660 con 0x55a874dc0910 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a874dc8600 con 0x55a874dbff20 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f05a2dca740 10 client.0 _ll_register_callbacks cb 0x55a874cd8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f059e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc54d0 0x55a874dd1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f059cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f059cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38304/0 (socket says 172.21.15.59:38304) 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.929+0000 7f059cc00640 1 -- 172.21.15.59:0/450405596 learned_addr learned my addr 172.21.15.59:0/450405596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059d600640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 -- 172.21.15.59:0/450405596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 msgr2=0x55a874dc9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 -- 172.21.15.59:0/450405596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc54d0 msgr2=0x55a874dd1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc54d0 0x55a874dd1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 -- 172.21.15.59:0/450405596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a874ce7e80 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f059cc00640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0584005450 tx=0x7f0584005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9e43a1e05d3dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f057ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584009c70 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0584005500 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f0584019040 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0584033050 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.930+0000 7f05a2dca740 10 client.27413 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a874a31210 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0584005500 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f058401f440 con 0x55a874dc0910 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 10 client.27413 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 10 client.27413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 20 client.27413 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874dd5b40 0x55a874dd7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 4 client.27413 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a874ccccd0 con 0x55a874dd5b40 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.931+0000 7f059d600640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874dd5b40 0x55a874dd7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.932+0000 7f059d600640 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874dd5b40 0x55a874dd7f60 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19545a237c829465 server_cookie=90cab8ff174656c6 in_seq=0 out_seq=0 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.932+0000 7f057ea00640 10 client.27413 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f057ea00640 1 -- 172.21.15.59:0/450405596 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a874ccccd0 con 0x55a874dd5b40 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f057ea00640 10 client.27413 handle_command_reply: tid=0 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 client.27413 shutdown 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 msgr2=0x55a874dca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0584005450 tx=0x7f0584005480 comp rx=0 tx=0).stop 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 shutdown_connections 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874dd5b40 0x55a874dd7f60 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f057ea00640 0 client.27413 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc54d0 0x55a874dd1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc0910 0x55a874dca3f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 --2- 172.21.15.59:0/450405596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 >> 172.21.15.59:0/450405596 conn(0x55a874db25f0 msgr2=0x55a874dbf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 shutdown_connections 2024-04-02T17:32:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 1 -- 172.21.15.59:0/450405596 wait complete. 2024-04-02T17:32:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.933+0000 7f05a2dca740 20 client.27413 trim_cache size 0 max 16384 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 Processor -- start 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 -- start start 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874dd13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a6cd0 con 0x55a874dc0910 2024-04-02T17:32:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a8810 con 0x55a874dc54d0 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f05a2dca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a73a0 con 0x55a874dbff20 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f059cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f059cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43886/0 (socket says 172.21.15.59:43886) 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.949+0000 7f059cc00640 1 -- 172.21.15.59:0/3797243310 learned_addr learned my addr 172.21.15.59:0/3797243310 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059e000640 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874dd13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059d600640 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 -- 172.21.15.59:0/3797243310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 msgr2=0x55a874dc8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 -- 172.21.15.59:0/3797243310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 msgr2=0x55a874dd13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874dd13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 -- 172.21.15.59:0/3797243310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a874afd620 con 0x55a874dc0910 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f059cc00640 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f05840058c0 tx=0x7f0584005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690b61d9305e07b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f0589c00640 1 -- 172.21.15.59:0/3797243310 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584009cf0 con 0x55a874dc0910 2024-04-02T17:32:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f0589c00640 1 -- 172.21.15.59:0/3797243310 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0584005500 con 0x55a874dc0910 2024-04-02T17:32:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.950+0000 7f0589c00640 1 -- 172.21.15.59:0/3797243310 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584011330 con 0x55a874dc0910 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- 172.21.15.59:0/3797243310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 msgr2=0x55a874dc8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f05840058c0 tx=0x7f0584005850 comp rx=0 tx=0).stop 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- 172.21.15.59:0/3797243310 shutdown_connections 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dc8ed0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874dd13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3797243310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dc8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- 172.21.15.59:0/3797243310 >> 172.21.15.59:0/3797243310 conn(0x55a874db25f0 msgr2=0x55a874dc18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- 172.21.15.59:0/3797243310 shutdown_connections 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- 172.21.15.59:0/3797243310 wait complete. 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.951+0000 7f05a2dca740 1 -- start start 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 Processor -- start 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874e3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a8140 con 0x55a874dc0910 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a7a70 con 0x55a874dc54d0 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05981a4cb0 con 0x55a874dbff20 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f05a2dca740 10 client.0 _ll_register_callbacks cb 0x7f0584011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f059cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f059cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43902/0 (socket says 172.21.15.59:43902) 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f059cc00640 1 -- 172.21.15.59:0/3469902550 learned_addr learned my addr 172.21.15.59:0/3469902550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.952+0000 7f059e000640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874e3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 -- 172.21.15.59:0/3469902550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 msgr2=0x55a874dce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 -- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 msgr2=0x55a874e3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874e3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 -- 172.21.15.59:0/3469902550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a874e3a680 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059e000640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874e3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f059cc00640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f058401fa00 tx=0x7f0584005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d64eb9843bb93cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f057ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584009e80 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7f05840110b0 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0584005500 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0584048050 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f05a2dca740 10 client.17301 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.953+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a874e3b9c0 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0584005500 con 0x55a874dc0910 2024-04-02T17:32:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f058403f070 con 0x55a874dc0910 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 10 client.17301 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 20 client.17301 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874e3c200 0x55a874e3e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 4 client.17301 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a874d2a2b0 con 0x55a874e3c200 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.954+0000 7f059d600640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874e3c200 0x55a874e3e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.955+0000 7f059d600640 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874e3c200 0x55a874e3e620 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f034e8f6843936 server_cookie=8e55d1370edffc71 in_seq=0 out_seq=0 2024-04-02T17:32:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:35.955+0000 7f057ea00640 10 client.17301 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.190+0000 7f057ea00640 1 -- 172.21.15.59:0/3469902550 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a874d2a2b0 con 0x55a874e3c200 2024-04-02T17:32:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.190+0000 7f057ea00640 10 client.17301 handle_command_reply: tid=0 2024-04-02T17:32:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.190+0000 7f05a2dca740 1 client.17301 shutdown 2024-04-02T17:32:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.191+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 msgr2=0x55a874dce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.191+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f058401fa00 tx=0x7f0584005720 comp rx=0 tx=0).stop 2024-04-02T17:32:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.191+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 shutdown_connections 2024-04-02T17:32:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.191+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a874e3c200 0x55a874e3e620 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a874dc0910 0x55a874dce9f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a874dc54d0 0x55a874e3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 --2- 172.21.15.59:0/3469902550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a874dbff20 0x55a874dce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 >> 172.21.15.59:0/3469902550 conn(0x55a874db25f0 msgr2=0x55a874dc9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f057ea00640 0 client.17301 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 shutdown_connections 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 1 -- 172.21.15.59:0/3469902550 wait complete. 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.192+0000 7f05a2dca740 20 client.17301 trim_cache size 0 max 16384 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:32:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 msgr2=0x7f05800a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 0x7f05800a60e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55a874dc0d10 tx=0x7f058c009260 comp rx=0 tx=0).stop 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 msgr2=0x7f059819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f05900337b0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 shutdown_connections 2024-04-02T17:32:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f05800a3c20 0x7f05800a60e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0598107630 0x7f059819f1a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0598104d00 0x7f0598198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 --2- 172.21.15.59:0/2307681267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05981023d0 0x7f0598197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 >> 172.21.15.59:0/2307681267 conn(0x7f05980fbde0 msgr2=0x7f0598102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 shutdown_connections 2024-04-02T17:32:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:36.193+0000 7f059ea00640 1 -- 172.21.15.59:0/2307681267 wait complete. 2024-04-02T17:32:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:36 smithi059 ceph-mon[31664]: pgmap v975: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 900 op/s 2024-04-02T17:32:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:36 smithi164 ceph-mon[36499]: pgmap v975: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 900 op/s 2024-04-02T17:32:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 278700 39.91 MB/sec execute 243 sec latency 141.534 ms 2024-04-02T17:32:36.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:36 smithi154 ceph-mon[36479]: pgmap v975: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 900 op/s 2024-04-02T17:32:37.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 279484 39.98 MB/sec execute 244 sec latency 46.205 ms 2024-04-02T17:32:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:38 smithi154 ceph-mon[36479]: pgmap v976: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-04-02T17:32:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:38 smithi059 ceph-mon[31664]: pgmap v976: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-04-02T17:32:38.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:38 smithi164 ceph-mon[36499]: pgmap v976: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-04-02T17:32:38.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 279925 39.87 MB/sec execute 245 sec latency 255.506 ms 2024-04-02T17:32:39.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 280259 39.76 MB/sec execute 246 sec latency 343.711 ms 2024-04-02T17:32:40.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:40 smithi154 ceph-mon[36479]: pgmap v977: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-04-02T17:32:40.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:40 smithi059 ceph-mon[31664]: pgmap v977: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-04-02T17:32:40.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:40 smithi164 ceph-mon[36499]: pgmap v977: 97 pgs: 97 active+clean; 1.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-04-02T17:32:40.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 280992 39.75 MB/sec execute 247 sec latency 63.033 ms 2024-04-02T17:32:41.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 282050 39.82 MB/sec execute 248 sec latency 38.423 ms 2024-04-02T17:32:42.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:41 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:32:42.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:41 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:32:42.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:41 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:32:42.629 INFO:tasks.workunit.client.0.smithi059.stdout: 10 282802 39.89 MB/sec execute 249 sec latency 57.308 ms 2024-04-02T17:32:43.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:42 smithi154 ceph-mon[36479]: pgmap v978: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 696 op/s 2024-04-02T17:32:43.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:42 smithi059 ceph-mon[31664]: pgmap v978: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 696 op/s 2024-04-02T17:32:43.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:42 smithi164 ceph-mon[36499]: pgmap v978: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 696 op/s 2024-04-02T17:32:43.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 283269 39.82 MB/sec execute 250 sec latency 52.153 ms 2024-04-02T17:32:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:32:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:32:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:43 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:32:44.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:32:44.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:32:44.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:43 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:32:44.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:32:44.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:32:44.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:43 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:32:44.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 283443 39.69 MB/sec execute 251 sec latency 580.149 ms 2024-04-02T17:32:45.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:44 smithi154 ceph-mon[36479]: pgmap v979: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-04-02T17:32:45.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:44 smithi059 ceph-mon[31664]: pgmap v979: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-04-02T17:32:45.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:44 smithi164 ceph-mon[36499]: pgmap v979: 97 pgs: 97 active+clean; 1.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-04-02T17:32:45.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 283692 39.57 MB/sec execute 252 sec latency 316.532 ms 2024-04-02T17:32:46.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:46 smithi059 ceph-mon[31664]: pgmap v980: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-02T17:32:46.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:46 smithi164 ceph-mon[36499]: pgmap v980: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-02T17:32:46.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 284245 39.50 MB/sec execute 253 sec latency 88.701 ms 2024-04-02T17:32:46.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:46 smithi154 ceph-mon[36479]: pgmap v980: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-02T17:32:47.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 285385 39.57 MB/sec execute 254 sec latency 42.173 ms 2024-04-02T17:32:48.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:48 smithi059 ceph-mon[31664]: pgmap v981: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-04-02T17:32:48.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:48 smithi164 ceph-mon[36499]: pgmap v981: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-04-02T17:32:48.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 286269 39.69 MB/sec execute 255 sec latency 52.428 ms 2024-04-02T17:32:48.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:48 smithi154 ceph-mon[36479]: pgmap v981: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-04-02T17:32:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 -- 172.21.15.59:0/3990892450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c10fe40 msgr2=0x7fc14c112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 --2- 172.21.15.59:0/3990892450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c10fe40 0x7fc14c112230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc144009ca0 tx=0x7fc14402c0f0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 -- 172.21.15.59:0/3990892450 shutdown_connections 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 --2- 172.21.15.59:0/3990892450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c10fe40 0x7fc14c112230 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 --2- 172.21.15.59:0/3990892450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c073040 0x7fc14c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 --2- 172.21.15.59:0/3990892450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c1096e0 0x7fc14c072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.401+0000 7fc14be00640 1 -- 172.21.15.59:0/3990892450 >> 172.21.15.59:0/3990892450 conn(0x7fc14c06e970 msgr2=0x7fc14c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.402+0000 7fc14be00640 1 -- 172.21.15.59:0/3990892450 shutdown_connections 2024-04-02T17:32:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.402+0000 7fc14be00640 1 -- 172.21.15.59:0/3990892450 wait complete. 2024-04-02T17:32:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.403+0000 7fc14be00640 1 Processor -- start 2024-04-02T17:32:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.403+0000 7fc14be00640 1 -- start start 2024-04-02T17:32:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c073040 0x7fc14c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 0x7fc14c1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113c80 con 0x7fc14c073040 2024-04-02T17:32:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113b00 con 0x7fc14c10fe40 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113e00 con 0x7fc14c1096e0 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 0x7fc14c1a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc143e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc143e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56602/0 (socket says 172.21.15.59:56602) 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 0x7fc14c1a5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42890/0 (socket says 172.21.15.59:42890) 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc143e00640 1 -- 172.21.15.59:0/2612020855 learned_addr learned my addr 172.21.15.59:0/2612020855 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.404+0000 7fc14aa00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c073040 0x7fc14c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 -- 172.21.15.59:0/2612020855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 msgr2=0x7fc14c1a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 0x7fc14c1a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 -- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c073040 msgr2=0x7fc14c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c073040 0x7fc14c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 -- 172.21.15.59:0/2612020855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c1a5e90 con 0x7fc14c1096e0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc143e00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc13400e9c0 tx=0x7fc13400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc150a00640 1 -- 172.21.15.59:0/2612020855 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13400ccc0 con 0x7fc14c1096e0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c1a6180 con 0x7fc14c1096e0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc150a00640 1 -- 172.21.15.59:0/2612020855 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1340035f0 con 0x7fc14c1096e0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.405+0000 7fc150a00640 1 -- 172.21.15.59:0/2612020855 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13400ce60 con 0x7fc14c1096e0 2024-04-02T17:32:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.406+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c1a66d0 con 0x7fc14c1096e0 2024-04-02T17:32:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.410+0000 7fc150a00640 1 -- 172.21.15.59:0/2612020855 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc1340037a0 con 0x7fc14c1096e0 2024-04-02T17:32:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.412+0000 7fc154a06740 1 Processor -- start 2024-04-02T17:32:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.412+0000 7fc154a06740 1 -- start start 2024-04-02T17:32:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.412+0000 7fc150a00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 0x7fc1300aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc150a00640 1 -- 172.21.15.59:0/2612020855 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc1340d07b0 con 0x7fc14c1096e0 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e62831ed70 con 0x55e6283b9100 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e6282c72a0 con 0x55e6283b9af0 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc154a06740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e62831ad20 con 0x55e6283bec60 2024-04-02T17:32:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc14b400640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 0x7fc1300aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46694/0 (socket says 172.21.15.59:46694) 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc14aa00640 1 -- 172.21.15.59:0/3953773757 learned_addr learned my addr 172.21.15.59:0/3953773757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.413+0000 7fc143e00640 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14b400640 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 -- 172.21.15.59:0/3953773757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 msgr2=0x55e6283c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 -- 172.21.15.59:0/3953773757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 msgr2=0x55e6283be5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283be5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 -- 172.21.15.59:0/3953773757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6282db840 con 0x55e6283b9100 2024-04-02T17:32:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.414+0000 7fc14aa00640 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009a70 tx=0x7fc13c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26675ae37d635af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.415+0000 7fc14b400640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 0x7fc1300aa470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc14402c630 tx=0x7fc144032c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.415+0000 7fc13be00640 1 -- 172.21.15.59:0/3953773757 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c017070 con 0x55e6283b9100 2024-04-02T17:32:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.415+0000 7fc13be00640 1 -- 172.21.15.59:0/3953773757 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13c013470 con 0x55e6283b9100 2024-04-02T17:32:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.415+0000 7fc13be00640 1 -- 172.21.15.59:0/3953773757 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0125d0 con 0x55e6283b9100 2024-04-02T17:32:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.416+0000 7fc154a06740 1 -- 172.21.15.59:0/3953773757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 msgr2=0x55e6283b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.416+0000 7fc154a06740 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009a70 tx=0x7fc13c009f30 comp rx=0 tx=0).stop 2024-04-02T17:32:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.416+0000 7fc154a06740 1 -- 172.21.15.59:0/3953773757 shutdown_connections 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.416+0000 7fc154a06740 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.417+0000 7fc154a06740 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283be5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.417+0000 7fc154a06740 1 --2- 172.21.15.59:0/3953773757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9100 0x55e6283b9520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.417+0000 7fc154a06740 1 -- 172.21.15.59:0/3953773757 >> 172.21.15.59:0/3953773757 conn(0x55e6283b4670 msgr2=0x55e6283b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.417+0000 7fc154a06740 1 -- 172.21.15.59:0/3953773757 shutdown_connections 2024-04-02T17:32:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.417+0000 7fc154a06740 1 -- 172.21.15.59:0/3953773757 wait complete. 2024-04-02T17:32:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.418+0000 7fc154a06740 1 -- start start 2024-04-02T17:32:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.418+0000 7fc154a06740 1 Processor -- start 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9af0 0x55e6283c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6283c1c10 con 0x55e6283b9af0 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e6282da660 con 0x55e6283b9100 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e6283c1d90 con 0x55e6283bec60 2024-04-02T17:32:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.419+0000 7fc154a06740 10 client.0 _ll_register_callbacks cb 0x55e6282da050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42908/0 (socket says 172.21.15.59:42908) 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 -- 172.21.15.59:0/2695123663 learned_addr learned my addr 172.21.15.59:0/2695123663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 -- 172.21.15.59:0/2695123663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 msgr2=0x55e6283cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 -- 172.21.15.59:0/2695123663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9af0 msgr2=0x55e6283c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9af0 0x55e6283c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.420+0000 7fc14aa00640 1 -- 172.21.15.59:0/2695123663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6282e9e80 con 0x55e6283b9100 2024-04-02T17:32:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.421+0000 7fc14aa00640 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc13c006a50 tx=0x7fc13c00ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be941df0e23de65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.421+0000 7fc138c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:32:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.422+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c017050 con 0x55e6283b9100 2024-04-02T17:32:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.422+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13c013470 con 0x55e6283b9100 2024-04-02T17:32:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.423+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc13c013aa0 con 0x55e6283b9100 2024-04-02T17:32:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.423+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc13c02f050 con 0x55e6283b9100 2024-04-02T17:32:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.424+0000 7fc154a06740 10 client.27431 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.424+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e628033210 con 0x55e6283b9100 2024-04-02T17:32:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.424+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13c013470 con 0x55e6283b9100 2024-04-02T17:32:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fc13c026640 con 0x55e6283b9100 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 10 client.27431 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 10 client.27431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 20 client.27431 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e6283cf100 0x55e6283d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 4 client.27431 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6282cecd0 con 0x55e6283cf100 2024-04-02T17:32:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.425+0000 7fc143e00640 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e6283cf100 0x55e6283d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.426+0000 7fc143e00640 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e6283cf100 0x55e6283d14f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=501ba8aacf4f9b33 server_cookie=7bafc46532507cf5 in_seq=0 out_seq=0 2024-04-02T17:32:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.426+0000 7fc138c00640 10 client.27431 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.427+0000 7fc138c00640 1 -- 172.21.15.59:0/2695123663 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6282cecd0 con 0x55e6283cf100 2024-04-02T17:32:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.428+0000 7fc138c00640 10 client.27431 handle_command_reply: tid=0 2024-04-02T17:32:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.428+0000 7fc154a06740 1 client.27431 shutdown 2024-04-02T17:32:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 msgr2=0x55e6283c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc13c006a50 tx=0x7fc13c00ef50 comp rx=0 tx=0).stop 2024-04-02T17:32:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 shutdown_connections 2024-04-02T17:32:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e6283cf100 0x55e6283d14f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283b9af0 0x55e6283c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283bec60 0x55e6283cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 --2- 172.21.15.59:0/2695123663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283c35e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 >> 172.21.15.59:0/2695123663 conn(0x55e6283b4670 msgr2=0x55e6283b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.429+0000 7fc138c00640 0 client.27431 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.430+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 shutdown_connections 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.430+0000 7fc154a06740 1 -- 172.21.15.59:0/2695123663 wait complete. 2024-04-02T17:32:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.430+0000 7fc154a06740 20 client.27431 trim_cache size 0 max 16384 2024-04-02T17:32:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.456+0000 7fc154a06740 1 Processor -- start 2024-04-02T17:32:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.456+0000 7fc154a06740 1 -- start start 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1af670 con 0x55e6283bec60 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b11b0 con 0x55e6283b9100 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc154a06740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1afd40 con 0x55e6283b9af0 2024-04-02T17:32:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc143e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc143e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56638/0 (socket says 172.21.15.59:56638) 2024-04-02T17:32:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.457+0000 7fc143e00640 1 -- 172.21.15.59:0/483962301 learned_addr learned my addr 172.21.15.59:0/483962301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc14b400640 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc143e00640 1 -- 172.21.15.59:0/483962301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 msgr2=0x55e6283b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc143e00640 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc143e00640 1 -- 172.21.15.59:0/483962301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 msgr2=0x55e6283c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc143e00640 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.458+0000 7fc143e00640 1 -- 172.21.15.59:0/483962301 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6280ff620 con 0x55e6283b9af0 2024-04-02T17:32:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.459+0000 7fc14aa00640 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:32:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.460+0000 7fc143e00640 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1340ce250 tx=0x7fc13409d1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d7704144ba5ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.461+0000 7fc13b400640 1 -- 172.21.15.59:0/483962301 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340a56a0 con 0x55e6283b9af0 2024-04-02T17:32:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.461+0000 7fc13b400640 1 -- 172.21.15.59:0/483962301 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13409aad0 con 0x55e6283b9af0 2024-04-02T17:32:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.461+0000 7fc13b400640 1 -- 172.21.15.59:0/483962301 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340a5210 con 0x55e6283b9af0 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.465+0000 7fc154a06740 1 -- 172.21.15.59:0/483962301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 msgr2=0x55e6283cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.465+0000 7fc154a06740 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1340ce250 tx=0x7fc13409d1f0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.465+0000 7fc154a06740 1 -- 172.21.15.59:0/483962301 shutdown_connections 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.466+0000 7fc154a06740 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.466+0000 7fc154a06740 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9af0 0x55e6283cab20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.466+0000 7fc154a06740 1 --2- 172.21.15.59:0/483962301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9100 0x55e6283b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.466+0000 7fc154a06740 1 -- 172.21.15.59:0/483962301 >> 172.21.15.59:0/483962301 conn(0x55e6283b4670 msgr2=0x55e6283b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.466+0000 7fc154a06740 1 -- 172.21.15.59:0/483962301 shutdown_connections 2024-04-02T17:32:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.467+0000 7fc154a06740 1 -- 172.21.15.59:0/483962301 wait complete. 2024-04-02T17:32:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.467+0000 7fc154a06740 1 -- start start 2024-04-02T17:32:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.468+0000 7fc154a06740 1 Processor -- start 2024-04-02T17:32:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b0ae0 con 0x55e6283bec60 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b0410 con 0x55e6283b9af0 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1ad650 con 0x55e6283b9100 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc154a06740 10 client.0 _ll_register_callbacks cb 0x7fc13c00eb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc14aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56646/0 (socket says 172.21.15.59:56646) 2024-04-02T17:32:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc14aa00640 1 -- 172.21.15.59:0/1860030094 learned_addr learned my addr 172.21.15.59:0/1860030094 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:32:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.469+0000 7fc143e00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14aa00640 1 -- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 msgr2=0x55e6283c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14aa00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14aa00640 1 -- 172.21.15.59:0/1860030094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 msgr2=0x55e6283c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:32:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14b400640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14aa00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc14aa00640 1 -- 172.21.15.59:0/1860030094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6283c9b20 con 0x55e6283b9100 2024-04-02T17:32:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.470+0000 7fc143e00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.471+0000 7fc14b400640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.471+0000 7fc14aa00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc13c000c00 tx=0x7fc13c009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de059d850d411f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.471+0000 7fc138c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.471+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c037020 con 0x55e6283b9100 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.472+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13409aad0 con 0x55e6283b9100 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.472+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8335+0+0 (secure 0 0 0) 0x7fc13c0265e0 con 0x55e6283b9100 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.472+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc13c042050 con 0x55e6283b9100 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.472+0000 7fc154a06740 10 client.27490 fetch_fsmap learned FSMap version 55 2024-04-02T17:32:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.472+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e6283cae60 con 0x55e6283b9100 2024-04-02T17:32:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.473+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13409aad0 con 0x55e6283b9100 2024-04-02T17:32:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.473+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fc13c033430 con 0x55e6283b9100 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 10 client.27490 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 10 client.27490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 20 client.27490 populate_metadata read hostname 'smithi059' 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e62843e660 0x55e628440a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 4 client.27490 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:32:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e62832c2b0 con 0x55e62843e660 2024-04-02T17:32:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.474+0000 7fc143e00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e62843e660 0x55e628440a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:32:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.475+0000 7fc143e00640 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e62843e660 0x55e628440a80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd6afaa27f6db3f server_cookie=33c0abc60d98b19e in_seq=0 out_seq=0 2024-04-02T17:32:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.475+0000 7fc138c00640 10 client.27490 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:49.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 286723 39.62 MB/sec execute 256 sec latency 135.608 ms 2024-04-02T17:32:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.709+0000 7fc138c00640 1 -- 172.21.15.59:0/1860030094 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e62832c2b0 con 0x55e62843e660 2024-04-02T17:32:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.709+0000 7fc138c00640 10 client.27490 handle_command_reply: tid=0 2024-04-02T17:32:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.709+0000 7fc154a06740 1 client.27490 shutdown 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.709+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 msgr2=0x55e6283c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.709+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc13c000c00 tx=0x7fc13c009960 comp rx=0 tx=0).stop 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 shutdown_connections 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55e62843e660 0x55e628440a80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6283bec60 0x55e6283c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e6283b9af0 0x55e6283c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 --2- 172.21.15.59:0/1860030094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e6283b9100 0x55e6283c6070 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 >> 172.21.15.59:0/1860030094 conn(0x55e6283b4670 msgr2=0x55e6283b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc138c00640 0 client.27490 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:32:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.710+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 shutdown_connections 2024-04-02T17:32:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.711+0000 7fc154a06740 1 -- 172.21.15.59:0/1860030094 wait complete. 2024-04-02T17:32:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.711+0000 7fc154a06740 20 client.27490 trim_cache size 0 max 16384 2024-04-02T17:32:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:32:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:32:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:32:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 msgr2=0x7fc1300aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 0x7fc1300aa470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc14402c630 tx=0x7fc144032c90 comp rx=0 tx=0).stop 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 msgr2=0x7fc14c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc13400e9c0 tx=0x7fc13400edc0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 shutdown_connections 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc14c10fe40 0x7fc14c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc14c1096e0 0x7fc14c19e7b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc1300a7fb0 0x7fc1300aa470 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 --2- 172.21.15.59:0/2612020855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc14c073040 0x7fc14c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:32:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 >> 172.21.15.59:0/2612020855 conn(0x7fc14c06e970 msgr2=0x7fc14c10be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:32:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 shutdown_connections 2024-04-02T17:32:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:32:49.713+0000 7fc14be00640 1 -- 172.21.15.59:0/2612020855 wait complete. 2024-04-02T17:32:50.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:50 smithi059 ceph-mon[31664]: pgmap v982: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 770 op/s 2024-04-02T17:32:50.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:50 smithi059 ceph-mon[31664]: daemon mds.f restarted 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-mon[36499]: pgmap v982: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 770 op/s 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-mon[36499]: daemon mds.f restarted 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-mds[66927]: -- 172.21.15.164:0/3799054413 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d56584e0 con 0x5576d562e800 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-mds[66927]: -- 172.21.15.164:0/3799054413 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576d565ba40 con 0x5576d562e800 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setuser ceph since I am not root 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: starting mds.f at 2024-04-02T17:32:50.572 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:32:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setgroup ceph since I am not root 2024-04-02T17:32:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: 10 287114 39.51 MB/sec execute 257 sec latency 186.066 ms 2024-04-02T17:32:50.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:50 smithi154 ceph-mon[36479]: pgmap v982: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 770 op/s 2024-04-02T17:32:50.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:50 smithi154 ceph-mon[36479]: daemon mds.f restarted 2024-04-02T17:32:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:51 smithi059 ceph-mon[31664]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T17:32:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:51 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:32:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:51 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6834/1035281632,v1:172.21.15.164:6835/1035281632] up:boot 2024-04-02T17:32:51.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:51 smithi059 ceph-mon[31664]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:32:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:51 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:32:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:51 smithi164 ceph-mon[36499]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T17:32:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:51 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:32:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:51 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6834/1035281632,v1:172.21.15.164:6835/1035281632] up:boot 2024-04-02T17:32:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:51 smithi164 ceph-mon[36499]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:32:51.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:51 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:32:51.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 287818 39.47 MB/sec execute 258 sec latency 82.466 ms 2024-04-02T17:32:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:51 smithi154 ceph-mon[36479]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T17:32:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:51 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:32:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:51 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6834/1035281632,v1:172.21.15.164:6835/1035281632] up:boot 2024-04-02T17:32:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:51 smithi154 ceph-mon[36479]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T17:32:51.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:51 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:32:52.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 288965 39.56 MB/sec execute 259 sec latency 89.808 ms 2024-04-02T17:32:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:53 smithi059 ceph-mon[31664]: pgmap v984: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 820 op/s 2024-04-02T17:32:53.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:53 smithi164 ceph-mon[36499]: pgmap v984: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 820 op/s 2024-04-02T17:32:53.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 289886 39.67 MB/sec execute 260 sec latency 35.737 ms 2024-04-02T17:32:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:53 smithi154 ceph-mon[36479]: pgmap v984: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 820 op/s 2024-04-02T17:32:54.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 290424 39.62 MB/sec execute 261 sec latency 57.188 ms 2024-04-02T17:32:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:55 smithi059 ceph-mon[31664]: pgmap v985: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 888 op/s 2024-04-02T17:32:55.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:55 smithi164 ceph-mon[36499]: pgmap v985: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 888 op/s 2024-04-02T17:32:55.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 291021 39.54 MB/sec execute 262 sec latency 90.042 ms 2024-04-02T17:32:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:55 smithi154 ceph-mon[36479]: pgmap v985: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 888 op/s 2024-04-02T17:32:56.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 291814 39.57 MB/sec execute 263 sec latency 146.003 ms 2024-04-02T17:32:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:57 smithi059 ceph-mon[31664]: pgmap v986: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 772 op/s 2024-04-02T17:32:57.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:57 smithi164 ceph-mon[36499]: pgmap v986: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 772 op/s 2024-04-02T17:32:57.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 292898 39.64 MB/sec execute 264 sec latency 43.758 ms 2024-04-02T17:32:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:57 smithi154 ceph-mon[36479]: pgmap v986: 97 pgs: 97 active+clean; 1.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.4 MiB/s wr, 772 op/s 2024-04-02T17:32:58.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 293671 39.69 MB/sec execute 265 sec latency 50.974 ms 2024-04-02T17:32:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:32:59 smithi059 ceph-mon[31664]: pgmap v987: 97 pgs: 97 active+clean; 1.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.9 MiB/s wr, 682 op/s 2024-04-02T17:32:59.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:32:59 smithi164 ceph-mon[36499]: pgmap v987: 97 pgs: 97 active+clean; 1.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.9 MiB/s wr, 682 op/s 2024-04-02T17:32:59.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 294279 39.63 MB/sec execute 266 sec latency 41.004 ms 2024-04-02T17:32:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:32:59 smithi154 ceph-mon[36479]: pgmap v987: 97 pgs: 97 active+clean; 1.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.9 MiB/s wr, 682 op/s 2024-04-02T17:33:00.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 295152 39.64 MB/sec execute 267 sec latency 43.757 ms 2024-04-02T17:33:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:01 smithi059 ceph-mon[31664]: pgmap v988: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-04-02T17:33:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:01 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:01 smithi164 ceph-mon[36499]: pgmap v988: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-04-02T17:33:01.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:01 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:01.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 296073 39.67 MB/sec execute 268 sec latency 68.661 ms 2024-04-02T17:33:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:01 smithi154 ceph-mon[36479]: pgmap v988: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-04-02T17:33:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:01 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:02.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 296923 39.75 MB/sec execute 269 sec latency 43.174 ms 2024-04-02T17:33:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 -- 172.21.15.59:0/2006567663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84102d40 msgr2=0x7efe841031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 --2- 172.21.15.59:0/2006567663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84102d40 0x7efe841031c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efe74009900 tx=0x7efe7402bc30 comp rx=0 tx=0).stop 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 -- 172.21.15.59:0/2006567663 shutdown_connections 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 --2- 172.21.15.59:0/2006567663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84103700 0x7efe84109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 --2- 172.21.15.59:0/2006567663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84102d40 0x7efe841031c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 --2- 172.21.15.59:0/2006567663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe84101b50 0x7efe84101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.899+0000 7efe8a400640 1 -- 172.21.15.59:0/2006567663 >> 172.21.15.59:0/2006567663 conn(0x7efe840fd0c0 msgr2=0x7efe840ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.900+0000 7efe8a400640 1 -- 172.21.15.59:0/2006567663 shutdown_connections 2024-04-02T17:33:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.900+0000 7efe8a400640 1 -- 172.21.15.59:0/2006567663 wait complete. 2024-04-02T17:33:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.901+0000 7efe8a400640 1 Processor -- start 2024-04-02T17:33:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 -- start start 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe84102d40 0x7efe84112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84103700 0x7efe84110cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410b5f0 con 0x7efe84102d40 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410b470 con 0x7efe84103700 2024-04-02T17:33:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe8a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410b770 con 0x7efe84101b50 2024-04-02T17:33:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe83400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.902+0000 7efe83400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39432/0 (socket says 172.21.15.59:39432) 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 -- 172.21.15.59:0/207753062 learned_addr learned my addr 172.21.15.59:0/207753062 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83e00640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84103700 0x7efe84110cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 -- 172.21.15.59:0/207753062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84103700 msgr2=0x7efe84110cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84103700 0x7efe84110cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 -- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe84102d40 msgr2=0x7efe84112b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe84102d40 0x7efe84112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.903+0000 7efe83400640 1 -- 172.21.15.59:0/207753062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe84111350 con 0x7efe84101b50 2024-04-02T17:33:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.904+0000 7efe83400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00ea80 tx=0x7efe7c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.904+0000 7efe7be00640 1 -- 172.21.15.59:0/207753062 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00cd80 con 0x7efe84101b50 2024-04-02T17:33:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.904+0000 7efe7be00640 1 -- 172.21.15.59:0/207753062 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c011950 con 0x7efe84101b50 2024-04-02T17:33:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.905+0000 7efe7be00640 1 -- 172.21.15.59:0/207753062 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01aac0 con 0x7efe84101b50 2024-04-02T17:33:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.905+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe84111640 con 0x7efe84101b50 2024-04-02T17:33:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.906+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841af190 con 0x7efe84101b50 2024-04-02T17:33:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.910+0000 7efe7be00640 1 -- 172.21.15.59:0/207753062 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efe7c011b00 con 0x7efe84101b50 2024-04-02T17:33:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.913+0000 7efe7be00640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 0x7efe680a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.913+0000 7efe7be00640 1 -- 172.21.15.59:0/207753062 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7efe7c0d1000 con 0x7efe84101b50 2024-04-02T17:33:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.913+0000 7efe82a00640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 0x7efe680a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.914+0000 7efe8d216740 1 Processor -- start 2024-04-02T17:33:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.915+0000 7efe8d216740 1 -- start start 2024-04-02T17:33:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.915+0000 7efe8d216740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.915+0000 7efe8d216740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bd83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.915+0000 7efe8d216740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd8a30 0x559603bdaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe8d216740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559603b38d70 con 0x559603bd2ed0 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe8d216740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559603ae12a0 con 0x559603bd8a30 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe8d216740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559603b34d20 con 0x559603bd38c0 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48092/0 (socket says 172.21.15.59:48092) 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe83400640 1 -- 172.21.15.59:0/2978272473 learned_addr learned my addr 172.21.15.59:0/2978272473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.916+0000 7efe82a00640 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bd83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 -- 172.21.15.59:0/2978272473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 msgr2=0x559603bd83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bd83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 -- 172.21.15.59:0/2978272473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd8a30 msgr2=0x559603bdaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd8a30 0x559603bdaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 -- 172.21.15.59:0/2978272473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559603af5840 con 0x559603bd2ed0 2024-04-02T17:33:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe82a00640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 0x7efe680a61b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe74036040 tx=0x7efe74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.917+0000 7efe83400640 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe7c099b60 tx=0x7efe7c09cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1840d8b831a5695d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.918+0000 7efe78c00640 1 -- 172.21.15.59:0/2978272473 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0b5030 con 0x559603bd2ed0 2024-04-02T17:33:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.918+0000 7efe78c00640 1 -- 172.21.15.59:0/2978272473 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c09d220 con 0x559603bd2ed0 2024-04-02T17:33:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.918+0000 7efe78c00640 1 -- 172.21.15.59:0/2978272473 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0a1170 con 0x559603bd2ed0 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 -- 172.21.15.59:0/2978272473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 msgr2=0x559603bd32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe7c099b60 tx=0x7efe7c09cb80 comp rx=0 tx=0).stop 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 -- 172.21.15.59:0/2978272473 shutdown_connections 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd8a30 0x559603bdaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bd83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 --2- 172.21.15.59:0/2978272473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd2ed0 0x559603bd32f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 -- 172.21.15.59:0/2978272473 >> 172.21.15.59:0/2978272473 conn(0x559603bce440 msgr2=0x559603bce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 -- 172.21.15.59:0/2978272473 shutdown_connections 2024-04-02T17:33:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.919+0000 7efe8d216740 1 -- 172.21.15.59:0/2978272473 wait complete. 2024-04-02T17:33:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.920+0000 7efe8d216740 1 -- start start 2024-04-02T17:33:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.920+0000 7efe8d216740 1 Processor -- start 2024-04-02T17:33:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bdd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603be2bc0 0x559603be4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559603bdb9e0 con 0x559603bd8a30 2024-04-02T17:33:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559603af4660 con 0x559603be2bc0 2024-04-02T17:33:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559603bdbb60 con 0x559603bd38c0 2024-04-02T17:33:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.921+0000 7efe8d216740 10 client.0 _ll_register_callbacks cb 0x559603af4050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.922+0000 7efe82a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.922+0000 7efe82a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48102/0 (socket says 172.21.15.59:48102) 2024-04-02T17:33:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.922+0000 7efe82a00640 1 -- 172.21.15.59:0/454586997 learned_addr learned my addr 172.21.15.59:0/454586997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.923+0000 7efe82a00640 1 -- 172.21.15.59:0/454586997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 msgr2=0x559603bdd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.923+0000 7efe82a00640 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bdd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.923+0000 7efe82a00640 1 -- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603be2bc0 msgr2=0x559603be4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.923+0000 7efe82a00640 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603be2bc0 0x559603be4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.923+0000 7efe82a00640 1 -- 172.21.15.59:0/454586997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559603b03e80 con 0x559603bd8a30 2024-04-02T17:33:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.924+0000 7efe82a00640 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe740057c0 tx=0x7efe74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31a9887c1c128c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.924+0000 7efe6d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:33:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.924+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7404d070 con 0x559603bd8a30 2024-04-02T17:33:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.924+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7efe74003d40 con 0x559603bd8a30 2024-04-02T17:33:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.924+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7c09d220 con 0x559603bd8a30 2024-04-02T17:33:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.925+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efe74059050 con 0x559603bd8a30 2024-04-02T17:33:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.925+0000 7efe8d216740 10 client.17322 fetch_fsmap learned FSMap version 58 2024-04-02T17:33:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.925+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55960384d210 con 0x559603bd8a30 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.925+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7c09d220 con 0x559603bd8a30 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.925+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 58) ==== 8358+0+0 (secure 0 0 0) 0x7efe74048070 con 0x559603bd8a30 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 10 client.17322 fetch_fsmap finished waiting for FSMap version 58 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 20 client.17322 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603be98d0 0x559603bebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 4 client.17322 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559603ae8cd0 con 0x559603be98d0 2024-04-02T17:33:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.926+0000 7efe83400640 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603be98d0 0x559603bebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.927+0000 7efe83400640 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603be98d0 0x559603bebd90 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2670d06cd811d8 server_cookie=e41713d5bdd5b850 in_seq=0 out_seq=0 2024-04-02T17:33:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.927+0000 7efe6d600640 10 client.17322 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe6d600640 1 -- 172.21.15.59:0/454586997 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559603ae8cd0 con 0x559603be98d0 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe6d600640 10 client.17322 handle_command_reply: tid=0 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 client.17322 shutdown 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 msgr2=0x559603bddb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe740057c0 tx=0x7efe74009260 comp rx=0 tx=0).stop 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 shutdown_connections 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603be98d0 0x559603bebd90 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bddb90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603be2bc0 0x559603be4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 --2- 172.21.15.59:0/454586997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd38c0 0x559603bdd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 >> 172.21.15.59:0/454586997 conn(0x559603bce440 msgr2=0x559603bcd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.928+0000 7efe6d600640 0 client.17322 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.929+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 shutdown_connections 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.929+0000 7efe8d216740 1 -- 172.21.15.59:0/454586997 wait complete. 2024-04-02T17:33:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.929+0000 7efe8d216740 20 client.17322 trim_cache size 0 max 16384 2024-04-02T17:33:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.957+0000 7efe8d216740 1 Processor -- start 2024-04-02T17:33:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 -- start start 2024-04-02T17:33:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b7f70 con 0x559603bd8a30 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b9ab0 con 0x559603bd38c0 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.958+0000 7efe8d216740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b8640 con 0x559603be2bc0 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.959+0000 7efe83e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.959+0000 7efe83e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:39444/0 (socket says 172.21.15.59:39444) 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.959+0000 7efe83e00640 1 -- 172.21.15.59:0/2291454468 learned_addr learned my addr 172.21.15.59:0/2291454468 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.959+0000 7efe82a00640 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.959+0000 7efe83e00640 1 -- 172.21.15.59:0/2291454468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 msgr2=0x559603bebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.960+0000 7efe83e00640 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.960+0000 7efe83e00640 1 -- 172.21.15.59:0/2291454468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 msgr2=0x559603bccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.960+0000 7efe83e00640 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.960+0000 7efe83e00640 1 -- 172.21.15.59:0/2291454468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559603919620 con 0x559603be2bc0 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.960+0000 7efe83e00640 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe70009c40 tx=0x7efe70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22580b2dfb2c6bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.961+0000 7efe6fe00640 1 -- 172.21.15.59:0/2291454468 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7001e070 con 0x559603be2bc0 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.961+0000 7efe6fe00640 1 -- 172.21.15.59:0/2291454468 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70009d30 con 0x559603be2bc0 2024-04-02T17:33:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.961+0000 7efe6fe00640 1 -- 172.21.15.59:0/2291454468 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70019430 con 0x559603be2bc0 2024-04-02T17:33:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 -- 172.21.15.59:0/2291454468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 msgr2=0x559603bcd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe70009c40 tx=0x7efe70011810 comp rx=0 tx=0).stop 2024-04-02T17:33:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 -- 172.21.15.59:0/2291454468 shutdown_connections 2024-04-02T17:33:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603bd8a30 0x559603bccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603be2bc0 0x559603bcd2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 --2- 172.21.15.59:0/2291454468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 -- 172.21.15.59:0/2291454468 >> 172.21.15.59:0/2291454468 conn(0x559603bce440 msgr2=0x559603beb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 -- 172.21.15.59:0/2291454468 shutdown_connections 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.962+0000 7efe8d216740 1 -- 172.21.15.59:0/2291454468 wait complete. 2024-04-02T17:33:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.963+0000 7efe8d216740 1 -- start start 2024-04-02T17:33:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.964+0000 7efe8d216740 1 Processor -- start 2024-04-02T17:33:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.965+0000 7efe8d216740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bfaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.965+0000 7efe8d216740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 0x559603bfaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.966+0000 7efe8d216740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.966+0000 7efe8d216740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b93e0 con 0x559603be2bc0 2024-04-02T17:33:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.966+0000 7efe8d216740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b8d10 con 0x559603bd38c0 2024-04-02T17:33:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.966+0000 7efe8d216740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b5f50 con 0x559603bd8a30 2024-04-02T17:33:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.966+0000 7efe8d216740 10 client.0 _ll_register_callbacks cb 0x7efe7402cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.968+0000 7efe83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.968+0000 7efe83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48118/0 (socket says 172.21.15.59:48118) 2024-04-02T17:33:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.968+0000 7efe83e00640 1 -- 172.21.15.59:0/2838055313 learned_addr learned my addr 172.21.15.59:0/2838055313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.968+0000 7efe83400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bfaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe82a00640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 0x559603bfaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83400640 1 -- 172.21.15.59:0/2838055313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 msgr2=0x559603bfaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83400640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 0x559603bfaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83400640 1 -- 172.21.15.59:0/2838055313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 msgr2=0x559603bfbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83400640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83400640 1 -- 172.21.15.59:0/2838055313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559603bfc580 con 0x559603bd38c0 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe83e00640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:33:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.970+0000 7efe82a00640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 0x559603bfaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:33:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.972+0000 7efe83400640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bfaa40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559603bcf3e0 tx=0x7efe7c09cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed755103ac98dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.972+0000 7efe6d600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.972+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0b5040 con 0x559603bd38c0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.972+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70009d30 con 0x559603bd38c0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.972+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7efe7c01ac60 con 0x559603bd38c0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.973+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efe7c005df0 con 0x559603bd38c0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.973+0000 7efe8d216740 10 client.27449 fetch_fsmap learned FSMap version 58 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.973+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559603c4fba0 con 0x559603bd38c0 2024-04-02T17:33:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.973+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70009d30 con 0x559603bd38c0 2024-04-02T17:33:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 58) ==== 8358+0+0 (secure 0 0 0) 0x7efe7c022070 con 0x559603bd38c0 2024-04-02T17:33:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 10 client.27449 fetch_fsmap finished waiting for FSMap version 58 2024-04-02T17:33:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 10 client.27449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 20 client.27449 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603c4ffd0 0x559603c523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 4 client.27449 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:33:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.977+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559603b462b0 con 0x559603c4ffd0 2024-04-02T17:33:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.980+0000 7efe82a00640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603c4ffd0 0x559603c523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.981+0000 7efe82a00640 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603c4ffd0 0x559603c523c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386ee0a18d00505e server_cookie=96cc9483c724e062 in_seq=0 out_seq=0 2024-04-02T17:33:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:02.982+0000 7efe6d600640 10 client.27449 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.265+0000 7efe6d600640 1 -- 172.21.15.59:0/2838055313 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559603b462b0 con 0x559603c4ffd0 2024-04-02T17:33:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.265+0000 7efe6d600640 10 client.27449 handle_command_reply: tid=0 2024-04-02T17:33:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.265+0000 7efe8d216740 1 client.27449 shutdown 2024-04-02T17:33:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 msgr2=0x559603bfaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bfaa40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559603bcf3e0 tx=0x7efe7c09cb80 comp rx=0 tx=0).stop 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 shutdown_connections 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559603c4ffd0 0x559603c523c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559603be2bc0 0x559603bfbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559603bd8a30 0x559603bfaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 --2- 172.21.15.59:0/2838055313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559603bd38c0 0x559603bfaa40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.266+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 >> 172.21.15.59:0/2838055313 conn(0x559603bce440 msgr2=0x559603bd16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.267+0000 7efe6d600640 0 client.27449 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.267+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 shutdown_connections 2024-04-02T17:33:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.267+0000 7efe8d216740 1 -- 172.21.15.59:0/2838055313 wait complete. 2024-04-02T17:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.271+0000 7efe8d216740 20 client.27449 trim_cache size 0 max 16384 2024-04-02T17:33:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:33:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:33:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:33:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:33:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.273+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 msgr2=0x7efe680a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.273+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 0x7efe680a61b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe74036040 tx=0x7efe74000bc0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.273+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 msgr2=0x7efe84112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.273+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00ea80 tx=0x7efe7c00ee80 comp rx=0 tx=0).stop 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 shutdown_connections 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7efe680a3cf0 0x7efe680a61b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe84102d40 0x7efe84112b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe84103700 0x7efe84110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 --2- 172.21.15.59:0/207753062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe84101b50 0x7efe84112620 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 >> 172.21.15.59:0/207753062 conn(0x7efe840fd0c0 msgr2=0x7efe84105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 shutdown_connections 2024-04-02T17:33:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:03.274+0000 7efe8a400640 1 -- 172.21.15.59:0/207753062 wait complete. 2024-04-02T17:33:03.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:03 smithi059 ceph-mon[31664]: pgmap v989: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 650 op/s 2024-04-02T17:33:03.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:03 smithi164 ceph-mon[36499]: pgmap v989: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 650 op/s 2024-04-02T17:33:03.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 297413 39.68 MB/sec execute 270 sec latency 178.917 ms 2024-04-02T17:33:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:03 smithi154 ceph-mon[36479]: pgmap v989: 97 pgs: 97 active+clean; 1.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 650 op/s 2024-04-02T17:33:04.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 298059 39.64 MB/sec execute 271 sec latency 88.225 ms 2024-04-02T17:33:05.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:05 smithi059 ceph-mon[31664]: pgmap v990: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 661 op/s 2024-04-02T17:33:05.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:05 smithi164 ceph-mon[36499]: pgmap v990: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 661 op/s 2024-04-02T17:33:05.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 298944 39.67 MB/sec execute 272 sec latency 55.222 ms 2024-04-02T17:33:05.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:05 smithi154 ceph-mon[36479]: pgmap v990: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 661 op/s 2024-04-02T17:33:06.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 299804 39.68 MB/sec execute 273 sec latency 68.963 ms 2024-04-02T17:33:07.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:07 smithi154 ceph-mon[36479]: pgmap v991: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 577 op/s 2024-04-02T17:33:07.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:07 smithi059 ceph-mon[31664]: pgmap v991: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 577 op/s 2024-04-02T17:33:07.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:07 smithi164 ceph-mon[36499]: pgmap v991: 97 pgs: 97 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.5 MiB/s wr, 577 op/s 2024-04-02T17:33:07.631 INFO:tasks.workunit.client.0.smithi059.stdout: 10 300539 39.73 MB/sec execute 274 sec latency 226.624 ms 2024-04-02T17:33:08.632 INFO:tasks.workunit.client.0.smithi059.stdout: 10 301163 39.69 MB/sec execute 275 sec latency 90.918 ms 2024-04-02T17:33:09.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:09 smithi059 ceph-mon[31664]: pgmap v992: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 628 op/s 2024-04-02T17:33:09.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:09 smithi164 ceph-mon[36499]: pgmap v992: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 628 op/s 2024-04-02T17:33:09.632 INFO:tasks.workunit.client.0.smithi059.stdout: 10 301870 39.67 MB/sec execute 276 sec latency 54.873 ms 2024-04-02T17:33:09.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:09 smithi154 ceph-mon[36479]: pgmap v992: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.6 MiB/s wr, 628 op/s 2024-04-02T17:33:10.632 INFO:tasks.workunit.client.0.smithi059.stdout: 10 302627 39.67 MB/sec execute 277 sec latency 207.488 ms 2024-04-02T17:33:11.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:11 smithi059 ceph-mon[31664]: pgmap v993: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 617 op/s 2024-04-02T17:33:11.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:11 smithi164 ceph-mon[36499]: pgmap v993: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 617 op/s 2024-04-02T17:33:11.632 INFO:tasks.workunit.client.0.smithi059.stdout: 10 303542 39.71 MB/sec execute 278 sec latency 55.542 ms 2024-04-02T17:33:11.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:11 smithi154 ceph-mon[36479]: pgmap v993: 97 pgs: 97 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 617 op/s 2024-04-02T17:33:12.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 304341 39.74 MB/sec execute 279 sec latency 74.497 ms 2024-04-02T17:33:13.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:13 smithi059 ceph-mon[31664]: pgmap v994: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 608 op/s 2024-04-02T17:33:13.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:13 smithi164 ceph-mon[36499]: pgmap v994: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 608 op/s 2024-04-02T17:33:13.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 304923 39.71 MB/sec execute 280 sec latency 97.405 ms 2024-04-02T17:33:13.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:13 smithi154 ceph-mon[36479]: pgmap v994: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 608 op/s 2024-04-02T17:33:14.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 305621 39.69 MB/sec execute 281 sec latency 71.238 ms 2024-04-02T17:33:15.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:15 smithi059 ceph-mon[31664]: pgmap v995: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 697 op/s 2024-04-02T17:33:15.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:15 smithi164 ceph-mon[36499]: pgmap v995: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 697 op/s 2024-04-02T17:33:15.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 306053 39.61 MB/sec execute 282 sec latency 446.347 ms 2024-04-02T17:33:15.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:15 smithi154 ceph-mon[36479]: pgmap v995: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 697 op/s 2024-04-02T17:33:16.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 306381 39.53 MB/sec execute 283 sec latency 556.848 ms 2024-04-02T17:33:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 -- 172.21.15.59:0/2086956012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001001c0 msgr2=0x7f64001025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 --2- 172.21.15.59:0/2086956012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001001c0 0x7f64001025f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009860 tx=0x7f63f002bb70 comp rx=0 tx=0).stop 2024-04-02T17:33:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 -- 172.21.15.59:0/2086956012 shutdown_connections 2024-04-02T17:33:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 --2- 172.21.15.59:0/2086956012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64001054e0 0x7f6400105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 --2- 172.21.15.59:0/2086956012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400102b30 0x7f6400104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 --2- 172.21.15.59:0/2086956012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001001c0 0x7f64001025f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 -- 172.21.15.59:0/2086956012 >> 172.21.15.59:0/2086956012 conn(0x7f64000fbde0 msgr2=0x7f64000fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 -- 172.21.15.59:0/2086956012 shutdown_connections 2024-04-02T17:33:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.686+0000 7f6405800640 1 -- 172.21.15.59:0/2086956012 wait complete. 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 Processor -- start 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 -- start start 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001001c0 0x7f640019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 0x7f64001a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6400113bf0 con 0x7f64001054e0 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6400113a70 con 0x7f6400102b30 2024-04-02T17:33:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.687+0000 7f6405800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6400113d70 con 0x7f64001001c0 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33776/0 (socket says 172.21.15.59:33776) 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 -- 172.21.15.59:0/688246384 learned_addr learned my addr 172.21.15.59:0/688246384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f6404e00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 0x7f64001a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ffe00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001001c0 0x7f640019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 -- 172.21.15.59:0/688246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001001c0 msgr2=0x7f640019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001001c0 0x7f640019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 -- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 msgr2=0x7f64001a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 0x7f64001a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f63ff400640 1 -- 172.21.15.59:0/688246384 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64001a5e50 con 0x7f6400102b30 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.688+0000 7f6404e00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 0x7f64001a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f63ff400640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00e9c0 tx=0x7f63ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f63fcc00640 1 -- 172.21.15.59:0/688246384 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec00ccc0 con 0x7f6400102b30 2024-04-02T17:33:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64001a6140 con 0x7f6400102b30 2024-04-02T17:33:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f63fcc00640 1 -- 172.21.15.59:0/688246384 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec005500 con 0x7f6400102b30 2024-04-02T17:33:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f63fcc00640 1 -- 172.21.15.59:0/688246384 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec010430 con 0x7f6400102b30 2024-04-02T17:33:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.689+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a66e0 con 0x7f6400102b30 2024-04-02T17:33:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.693+0000 7f63fcc00640 1 -- 172.21.15.59:0/688246384 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f63ec0046c0 con 0x7f6400102b30 2024-04-02T17:33:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.694+0000 7f63fcc00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 0x7f63e40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.694+0000 7f63fcc00640 1 -- 172.21.15.59:0/688246384 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f63ec0d0730 con 0x7f6400102b30 2024-04-02T17:33:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.694+0000 7f63ffe00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 0x7f63e40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.695+0000 7f63ffe00640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 0x7f63e40a60e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63f002c180 tx=0x7f63f0034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 Processor -- start 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 -- start start 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e75520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556390ddad70 con 0x556390e75af0 2024-04-02T17:33:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556390d832a0 con 0x556390e75100 2024-04-02T17:33:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.702+0000 7f6409bf4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556390dd6d20 con 0x556390e7ac60 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46712/0 (socket says 172.21.15.59:46712) 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 -- 172.21.15.59:0/1170947082 learned_addr learned my addr 172.21.15.59:0/1170947082 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ffe00640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e75520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f6404e00640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 -- 172.21.15.59:0/1170947082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 msgr2=0x556390e7d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e7d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 -- 172.21.15.59:0/1170947082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 msgr2=0x556390e75520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e75520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 -- 172.21.15.59:0/1170947082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556390d97840 con 0x556390e75af0 2024-04-02T17:33:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f6404e00640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e7d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:33:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.703+0000 7f63ff400640 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f63ec099320 tx=0x7f63ec09c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925f174678682f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.704+0000 7f63f5600640 1 -- 172.21.15.59:0/1170947082 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0a0d00 con 0x556390e75af0 2024-04-02T17:33:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.704+0000 7f63f5600640 1 -- 172.21.15.59:0/1170947082 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec0a55c0 con 0x556390e75af0 2024-04-02T17:33:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.704+0000 7f63f5600640 1 -- 172.21.15.59:0/1170947082 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0870c0 con 0x556390e75af0 2024-04-02T17:33:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.704+0000 7f6409bf4740 1 -- 172.21.15.59:0/1170947082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 msgr2=0x556390e7a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.704+0000 7f6409bf4740 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f63ec099320 tx=0x7f63ec09c550 comp rx=0 tx=0).stop 2024-04-02T17:33:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 -- 172.21.15.59:0/1170947082 shutdown_connections 2024-04-02T17:33:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e7a5f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e7d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 --2- 172.21.15.59:0/1170947082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e75520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 -- 172.21.15.59:0/1170947082 >> 172.21.15.59:0/1170947082 conn(0x556390e70670 msgr2=0x556390e70a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 -- 172.21.15.59:0/1170947082 shutdown_connections 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 -- 172.21.15.59:0/1170947082 wait complete. 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 -- start start 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.705+0000 7f6409bf4740 1 Processor -- start 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75100 0x556390e7f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e86bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556390e7dc10 con 0x556390e75100 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556390d96660 con 0x556390e75af0 2024-04-02T17:33:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556390e7dd90 con 0x556390e7ac60 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6409bf4740 10 client.0 _ll_register_callbacks cb 0x556390d96050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75100 0x556390e7f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6404e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e86bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f63ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33798/0 (socket says 172.21.15.59:33798) 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f6404e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e86bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36366/0 (socket says 172.21.15.59:36366) 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.706+0000 7f63ff400640 1 -- 172.21.15.59:0/4225058774 learned_addr learned my addr 172.21.15.59:0/4225058774 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 -- 172.21.15.59:0/4225058774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 msgr2=0x556390e86bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e86bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 -- 172.21.15.59:0/4225058774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75100 msgr2=0x556390e7f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75100 0x556390e7f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 -- 172.21.15.59:0/4225058774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556390da5e80 con 0x556390e75af0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ff400640 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f63ec09c8e0 tx=0x7f63ec09ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2f1fa58cd384c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ea000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:33:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0b8030 con 0x556390e75af0 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f63ec09a9f0 con 0x556390e75af0 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ec0a55c0 con 0x556390e75af0 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f63ec0a0d00 con 0x556390e75af0 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f6409bf4740 10 client.27461 fetch_fsmap learned FSMap version 58 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.707+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556390aef210 con 0x556390e75af0 2024-04-02T17:33:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ec0a55c0 con 0x556390e75af0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 58) ==== 8358+0+0 (secure 0 0 0) 0x7f63ec083030 con 0x556390e75af0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 10 client.27461 fetch_fsmap finished waiting for FSMap version 58 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 10 client.27461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 20 client.27461 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390e8b0e0 0x556390e8d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 4 client.27461 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556390d8acd0 con 0x556390e8b0e0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.708+0000 7f63ffe00640 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390e8b0e0 0x556390e8d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f63ffe00640 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390e8b0e0 0x556390e8d4d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c115bb17f15e35 server_cookie=3603445f48ef5ccf in_seq=0 out_seq=0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f63ea000640 10 client.27461 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f63ea000640 1 -- 172.21.15.59:0/4225058774 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556390d8acd0 con 0x556390e8b0e0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f63ea000640 10 client.27461 handle_command_reply: tid=0 2024-04-02T17:33:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f6409bf4740 1 client.27461 shutdown 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 msgr2=0x556390e7fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.709+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f63ec09c8e0 tx=0x7f63ec09ca10 comp rx=0 tx=0).stop 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 shutdown_connections 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390e8b0e0 0x556390e8d4d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e86bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75af0 0x556390e7fbb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f63ea000640 0 client.27461 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4225058774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75100 0x556390e7f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 >> 172.21.15.59:0/4225058774 conn(0x556390e70670 msgr2=0x556390e6f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 shutdown_connections 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 1 -- 172.21.15.59:0/4225058774 wait complete. 2024-04-02T17:33:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.710+0000 7f6409bf4740 20 client.27461 trim_cache size 0 max 16384 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 Processor -- start 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 -- start start 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e7ac60 0x556390e91ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64001af6b0 con 0x556390e75af0 2024-04-02T17:33:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b11f0 con 0x556390e7ac60 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.727+0000 7f6409bf4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64001afd80 con 0x556390e75100 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36378/0 (socket says 172.21.15.59:36378) 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 -- 172.21.15.59:0/4148905816 learned_addr learned my addr 172.21.15.59:0/4148905816 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f6404e00640 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e7ac60 0x556390e91ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ff400640 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 -- 172.21.15.59:0/4148905816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e7ac60 msgr2=0x556390e91ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e7ac60 0x556390e91ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 -- 172.21.15.59:0/4148905816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 msgr2=0x556390e86bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e86bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 -- 172.21.15.59:0/4148905816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556390bbb620 con 0x556390e75100 2024-04-02T17:33:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.728+0000 7f63ffe00640 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f63f00057c0 tx=0x7f63f003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49e7794ad024479 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f63f4c00640 1 -- 172.21.15.59:0/4148905816 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0039040 con 0x556390e75100 2024-04-02T17:33:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f63f4c00640 1 -- 172.21.15.59:0/4148905816 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0054300 con 0x556390e75100 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f63f4c00640 1 -- 172.21.15.59:0/4148905816 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f002c870 con 0x556390e75100 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 -- 172.21.15.59:0/4148905816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 msgr2=0x556390e73fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f63f00057c0 tx=0x7f63f003e000 comp rx=0 tx=0).stop 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 -- 172.21.15.59:0/4148905816 shutdown_connections 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e86bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e7ac60 0x556390e91ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 --2- 172.21.15.59:0/4148905816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e75100 0x556390e73fe0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.729+0000 7f6409bf4740 1 -- 172.21.15.59:0/4148905816 >> 172.21.15.59:0/4148905816 conn(0x556390e70670 msgr2=0x556390e738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- 172.21.15.59:0/4148905816 shutdown_connections 2024-04-02T17:33:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- 172.21.15.59:0/4148905816 wait complete. 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- start start 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 Processor -- start 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b0b20 con 0x556390e75af0 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b0450 con 0x556390e75100 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.730+0000 7f6409bf4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64001ad690 con 0x556390e7ac60 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33810/0 (socket says 172.21.15.59:33810) 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 -- 172.21.15.59:0/3888415724 learned_addr learned my addr 172.21.15.59:0/3888415724 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ff400640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f6404e00640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f6409bf4740 10 client.0 _ll_register_callbacks cb 0x7f63ec09c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 -- 172.21.15.59:0/3888415724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 msgr2=0x556390e854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 -- 172.21.15.59:0/3888415724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 msgr2=0x556390e844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 -- 172.21.15.59:0/3888415724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556390e85aa0 con 0x556390e75100 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ff400640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ffe00640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f63f0037040 tx=0x7f63f002bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98989e9bb54a152a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.731+0000 7f63ea000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0063060 con 0x556390e75100 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f63f00544b0 con 0x556390e75100 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f0054300 con 0x556390e75100 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f63f0039050 con 0x556390e75100 2024-04-02T17:33:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f6409bf4740 10 client.27467 fetch_fsmap learned FSMap version 58 2024-04-02T17:33:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556390e86d90 con 0x556390e75100 2024-04-02T17:33:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.732+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0054300 con 0x556390e75100 2024-04-02T17:33:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 58) ==== 8358+0+0 (secure 0 0 0) 0x7f63f0032780 con 0x556390e75100 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 10 client.27467 fetch_fsmap finished waiting for FSMap version 58 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 10 client.27467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 20 client.27467 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390ef1a50 0x556390ef3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 4 client.27467 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.733+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556390de82b0 con 0x556390ef1a50 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.734+0000 7f63ff400640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390ef1a50 0x556390ef3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.734+0000 7f63ff400640 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390ef1a50 0x556390ef3e70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f271e0de5b02b3e server_cookie=748140ad51773a67 in_seq=0 out_seq=0 2024-04-02T17:33:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:16.734+0000 7f63ea000640 10 client.27467 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.158+0000 7f63ea000640 1 -- 172.21.15.59:0/3888415724 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556390de82b0 con 0x556390ef1a50 2024-04-02T17:33:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.158+0000 7f63ea000640 10 client.27467 handle_command_reply: tid=0 2024-04-02T17:33:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.158+0000 7f6409bf4740 1 client.27467 shutdown 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 msgr2=0x556390e83f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f63f0037040 tx=0x7f63f002bb70 comp rx=0 tx=0).stop 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 shutdown_connections 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556390ef1a50 0x556390ef3e70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556390e75af0 0x556390e844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556390e7ac60 0x556390e854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 --2- 172.21.15.59:0/3888415724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556390e75100 0x556390e83f60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 >> 172.21.15.59:0/3888415724 conn(0x556390e70670 msgr2=0x556390e74160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.159+0000 7f63ea000640 0 client.27467 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.160+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 shutdown_connections 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.160+0000 7f6409bf4740 1 -- 172.21.15.59:0/3888415724 wait complete. 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.160+0000 7f6409bf4740 20 client.27467 trim_cache size 0 max 16384 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:33:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:33:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 msgr2=0x7f63e40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 0x7f63e40a60e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63f002c180 tx=0x7f63f0034ad0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 msgr2=0x7f640019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00e9c0 tx=0x7f63ec00ee80 comp rx=0 tx=0).stop 2024-04-02T17:33:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 shutdown_connections 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f63e40a3c20 0x7f63e40a60e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64001054e0 0x7f64001a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6400102b30 0x7f640019e770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 --2- 172.21.15.59:0/688246384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64001001c0 0x7f640019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 >> 172.21.15.59:0/688246384 conn(0x7f64000fbde0 msgr2=0x7f64001034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 shutdown_connections 2024-04-02T17:33:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:17.162+0000 7f6405800640 1 -- 172.21.15.59:0/688246384 wait complete. 2024-04-02T17:33:17.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:17 smithi059 ceph-mon[31664]: pgmap v996: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-04-02T17:33:17.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:17 smithi164 ceph-mon[36499]: pgmap v996: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-04-02T17:33:17.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 306737 39.46 MB/sec execute 284 sec latency 177.176 ms 2024-04-02T17:33:17.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:17 smithi154 ceph-mon[36479]: pgmap v996: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-04-02T17:33:18.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 307638 39.54 MB/sec execute 285 sec latency 49.710 ms 2024-04-02T17:33:19.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:19 smithi059 ceph-mon[31664]: pgmap v997: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 687 op/s 2024-04-02T17:33:19.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:19 smithi164 ceph-mon[36499]: pgmap v997: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 687 op/s 2024-04-02T17:33:19.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 308314 39.52 MB/sec execute 286 sec latency 58.366 ms 2024-04-02T17:33:19.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:19 smithi154 ceph-mon[36479]: pgmap v997: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 687 op/s 2024-04-02T17:33:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:20 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:20.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:20 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:20.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 308941 39.49 MB/sec execute 287 sec latency 63.738 ms 2024-04-02T17:33:20.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:20 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:33:21.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:21 smithi059 ceph-mon[31664]: pgmap v998: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-04-02T17:33:21.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:21 smithi164 ceph-mon[36499]: pgmap v998: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-04-02T17:33:21.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 309082 39.37 MB/sec execute 288 sec latency 791.522 ms 2024-04-02T17:33:21.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:21 smithi154 ceph-mon[36479]: pgmap v998: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-04-02T17:33:22.633 INFO:tasks.workunit.client.0.smithi059.stdout: 10 309347 39.28 MB/sec execute 289 sec latency 909.890 ms 2024-04-02T17:33:23.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:23 smithi059 ceph-mon[31664]: pgmap v999: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-04-02T17:33:23.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:23 smithi164 ceph-mon[36499]: pgmap v999: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-04-02T17:33:23.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 309822 39.21 MB/sec execute 290 sec latency 105.939 ms 2024-04-02T17:33:23.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:23 smithi154 ceph-mon[36479]: pgmap v999: 97 pgs: 97 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-04-02T17:33:24.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 310559 39.23 MB/sec execute 291 sec latency 97.616 ms 2024-04-02T17:33:25.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:25 smithi059 ceph-mon[31664]: pgmap v1000: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 986 op/s 2024-04-02T17:33:25.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:25 smithi164 ceph-mon[36499]: pgmap v1000: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 986 op/s 2024-04-02T17:33:25.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 311413 39.26 MB/sec execute 292 sec latency 55.489 ms 2024-04-02T17:33:25.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:25 smithi154 ceph-mon[36479]: pgmap v1000: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 986 op/s 2024-04-02T17:33:26.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 312165 39.29 MB/sec execute 293 sec latency 35.589 ms 2024-04-02T17:33:27.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:27 smithi164 ceph-mon[36499]: pgmap v1001: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 969 op/s 2024-04-02T17:33:27.635 INFO:tasks.workunit.client.0.smithi059.stdout: 10 312609 39.22 MB/sec execute 294 sec latency 282.239 ms 2024-04-02T17:33:27.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:27 smithi154 ceph-mon[36479]: pgmap v1001: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 969 op/s 2024-04-02T17:33:27.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:27 smithi059 ceph-mon[31664]: pgmap v1001: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 969 op/s 2024-04-02T17:33:28.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 313327 39.19 MB/sec execute 295 sec latency 70.586 ms 2024-04-02T17:33:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:29 smithi164 ceph-mon[36499]: pgmap v1002: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-04-02T17:33:29.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 314208 39.23 MB/sec execute 296 sec latency 61.753 ms 2024-04-02T17:33:29.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:29 smithi154 ceph-mon[36479]: pgmap v1002: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-04-02T17:33:29.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:29 smithi059 ceph-mon[31664]: pgmap v1002: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-04-02T17:33:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 -- 172.21.15.59:0/3764270752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0109d30 msgr2=0x7fb9b01105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/3764270752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0109d30 0x7fb9b01105c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009ca0 tx=0x7fb9a802c100 comp rx=0 tx=0).stop 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 -- 172.21.15.59:0/3764270752 shutdown_connections 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/3764270752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0109d30 0x7fb9b01105c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/3764270752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/3764270752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9b0108180 0x7fb9b01085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.418+0000 7fb9b7a00640 1 -- 172.21.15.59:0/3764270752 >> 172.21.15.59:0/3764270752 conn(0x7fb9b0077150 msgr2=0x7fb9b0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.419+0000 7fb9b7a00640 1 -- 172.21.15.59:0/3764270752 shutdown_connections 2024-04-02T17:33:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.419+0000 7fb9b7a00640 1 -- 172.21.15.59:0/3764270752 wait complete. 2024-04-02T17:33:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 Processor -- start 2024-04-02T17:33:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 -- start start 2024-04-02T17:33:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0108180 0x7fb9b019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9b0109d30 0x7fb9b01a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113de0 con 0x7fb9b0109d30 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113c60 con 0x7fb9b0108180 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.420+0000 7fb9b7a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113f60 con 0x7fb9b0109370 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.421+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0108180 0x7fb9b019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.421+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.421+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32908/0 (socket says 172.21.15.59:32908) 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.421+0000 7fb9afe00640 1 -- 172.21.15.59:0/1800121291 learned_addr learned my addr 172.21.15.59:0/1800121291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.422+0000 7fb9afe00640 1 -- 172.21.15.59:0/1800121291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0108180 msgr2=0x7fb9b019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.422+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0108180 0x7fb9b019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.422+0000 7fb9afe00640 1 -- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9b0109d30 msgr2=0x7fb9b01a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.422+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9b0109d30 0x7fb9b01a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.422+0000 7fb9afe00640 1 -- 172.21.15.59:0/1800121291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b01a6040 con 0x7fb9b0109370 2024-04-02T17:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.423+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00ea60 tx=0x7fb99c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.424+0000 7fb9ad600640 1 -- 172.21.15.59:0/1800121291 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00cde0 con 0x7fb9b0109370 2024-04-02T17:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.424+0000 7fb9ad600640 1 -- 172.21.15.59:0/1800121291 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c0035f0 con 0x7fb9b0109370 2024-04-02T17:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.424+0000 7fb9ad600640 1 -- 172.21.15.59:0/1800121291 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c010670 con 0x7fb9b0109370 2024-04-02T17:33:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.425+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a6330 con 0x7fb9b0109370 2024-04-02T17:33:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.425+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a67a0 con 0x7fb9b0109370 2024-04-02T17:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.430+0000 7fb9ad600640 1 -- 172.21.15.59:0/1800121291 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb99c0037a0 con 0x7fb9b0109370 2024-04-02T17:33:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.432+0000 7fb9ad600640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 0x7fb9980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.433+0000 7fb9ad600640 1 -- 172.21.15.59:0/1800121291 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fb99c0d0a30 con 0x7fb9b0109370 2024-04-02T17:33:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.433+0000 7fb9b5000640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 0x7fb9980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.436+0000 7fb9b5000640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 0x7fb9980a6190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9b01091d0 tx=0x7fb9a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 Processor -- start 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 -- start start 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d09c8d0 0x56526d0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56526cff0d70 con 0x56526d09bee0 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56526cf992a0 con 0x56526d09c8d0 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.446+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56526cfecd20 con 0x56526d0a1a40 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39606/0 (socket says 172.21.15.59:39606) 2024-04-02T17:33:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 -- 172.21.15.59:0/3949432523 learned_addr learned my addr 172.21.15.59:0/3949432523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5a00640 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 -- 172.21.15.59:0/3949432523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 msgr2=0x56526d0a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 -- 172.21.15.59:0/3949432523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d09c8d0 msgr2=0x56526d0a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d09c8d0 0x56526d0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.447+0000 7fb9b5000640 1 -- 172.21.15.59:0/3949432523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526cfad840 con 0x56526d09bee0 2024-04-02T17:33:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.448+0000 7fb9b5a00640 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:33:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.448+0000 7fb9b5000640 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40058a0 tx=0x7fb9a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f619b1544cbc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.449+0000 7fb9a2000640 1 -- 172.21.15.59:0/3949432523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004220 con 0x56526d09bee0 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.449+0000 7fb9a2000640 1 -- 172.21.15.59:0/3949432523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40043c0 con 0x56526d09bee0 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.449+0000 7fb9a2000640 1 -- 172.21.15.59:0/3949432523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a401ea00 con 0x56526d09bee0 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- 172.21.15.59:0/3949432523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 msgr2=0x56526d09c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40058a0 tx=0x7fb9a4013000 comp rx=0 tx=0).stop 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- 172.21.15.59:0/3949432523 shutdown_connections 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d09c8d0 0x56526d0a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/3949432523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09bee0 0x56526d09c300 secure :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40058a0 tx=0x7fb9a4013000 comp rx=0 tx=0).stop 2024-04-02T17:33:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- 172.21.15.59:0/3949432523 >> 172.21.15.59:0/3949432523 conn(0x56526d086440 msgr2=0x56526d086840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- 172.21.15.59:0/3949432523 shutdown_connections 2024-04-02T17:33:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- 172.21.15.59:0/3949432523 wait complete. 2024-04-02T17:33:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.450+0000 7fb9ba79c740 1 -- start start 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 Processor -- start 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d0abba0 0x56526d0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56526d0a49f0 con 0x56526d0abba0 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56526cfac660 con 0x56526d0a1a40 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56526d0a4b70 con 0x56526d09c8d0 2024-04-02T17:33:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.451+0000 7fb9ba79c740 10 client.0 _ll_register_callbacks cb 0x56526cfac050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32924/0 (socket says 172.21.15.59:32924) 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d0abba0 0x56526d0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 -- 172.21.15.59:0/1636687505 learned_addr learned my addr 172.21.15.59:0/1636687505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 -- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 msgr2=0x56526d0a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 -- 172.21.15.59:0/1636687505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d0abba0 msgr2=0x56526d0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.452+0000 7fb9b5000640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d0abba0 0x56526d0adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb9b5000640 1 -- 172.21.15.59:0/1636687505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526cfbbe80 con 0x56526d09c8d0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb9b5000640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4011f50 tx=0x7fb9a4004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5a8a5da6fb8eea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb996a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4003a00 con 0x56526d09c8d0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fb9a401e700 con 0x56526d09c8d0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.453+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a40043c0 con 0x56526d09c8d0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb9a4010ca0 con 0x56526d09c8d0 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb9ba79c740 10 client.27535 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56526cd05210 con 0x56526d09c8d0 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40043c0 con 0x56526d09c8d0 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fb9a4032850 con 0x56526d09c8d0 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb9ba79c740 10 client.27535 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb9ba79c740 10 client.27535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.454+0000 7fb9ba79c740 20 client.27535 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.455+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d0b2550 0x56526d0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.455+0000 7fb9ba79c740 4 client.27535 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:33:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.455+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56526cfa0cd0 con 0x56526d0b2550 2024-04-02T17:33:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.455+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d0b2550 0x56526d0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.456+0000 7fb9afe00640 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d0b2550 0x56526d0b4940 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e46ce005bece33 server_cookie=6ede33cf40bfa0cd in_seq=0 out_seq=0 2024-04-02T17:33:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.457+0000 7fb996a00640 10 client.27535 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.460+0000 7fb996a00640 1 -- 172.21.15.59:0/1636687505 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56526cfa0cd0 con 0x56526d0b2550 2024-04-02T17:33:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.460+0000 7fb996a00640 10 client.27535 handle_command_reply: tid=0 2024-04-02T17:33:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.460+0000 7fb9ba79c740 1 client.27535 shutdown 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 msgr2=0x56526d0a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4011f50 tx=0x7fb9a4004160 comp rx=0 tx=0).stop 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 shutdown_connections 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d0b2550 0x56526d0b4940 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d0abba0 0x56526d0adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/1636687505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d09c8d0 0x56526d0a6630 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 >> 172.21.15.59:0/1636687505 conn(0x56526d086440 msgr2=0x56526d085640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb996a00640 0 client.27535 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 shutdown_connections 2024-04-02T17:33:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.461+0000 7fb9ba79c740 1 -- 172.21.15.59:0/1636687505 wait complete. 2024-04-02T17:33:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.462+0000 7fb9ba79c740 20 client.27535 trim_cache size 0 max 16384 2024-04-02T17:33:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.485+0000 7fb9ba79c740 1 Processor -- start 2024-04-02T17:33:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.485+0000 7fb9ba79c740 1 -- start start 2024-04-02T17:33:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b007aa20 con 0x56526d09c8d0 2024-04-02T17:33:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b007c560 con 0x56526d0a1a40 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.486+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b007b0f0 con 0x56526d0abba0 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32936/0 (socket says 172.21.15.59:32936) 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 -- 172.21.15.59:0/79240897 learned_addr learned my addr 172.21.15.59:0/79240897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5000640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9afe00640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 -- 172.21.15.59:0/79240897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 msgr2=0x56526d0a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 -- 172.21.15.59:0/79240897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 msgr2=0x56526d0a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.487+0000 7fb9b5a00640 1 -- 172.21.15.59:0/79240897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526cdd1620 con 0x56526d0abba0 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.488+0000 7fb9b5000640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.488+0000 7fb9b5a00640 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8004af0 tx=0x7fb9a8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4950032522fa2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.488+0000 7fb9a1600640 1 -- 172.21.15.59:0/79240897 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8055070 con 0x56526d0abba0 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.488+0000 7fb9a1600640 1 -- 172.21.15.59:0/79240897 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8003970 con 0x56526d0abba0 2024-04-02T17:33:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.488+0000 7fb9a1600640 1 -- 172.21.15.59:0/79240897 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8039040 con 0x56526d0abba0 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 -- 172.21.15.59:0/79240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 msgr2=0x56526d084db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8004af0 tx=0x7fb9a8009800 comp rx=0 tx=0).stop 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 -- 172.21.15.59:0/79240897 shutdown_connections 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0abba0 0x56526d084db0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0a1a40 0x56526d0a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/79240897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 -- 172.21.15.59:0/79240897 >> 172.21.15.59:0/79240897 conn(0x56526d086440 msgr2=0x56526d09dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 -- 172.21.15.59:0/79240897 shutdown_connections 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.489+0000 7fb9ba79c740 1 -- 172.21.15.59:0/79240897 wait complete. 2024-04-02T17:33:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 -- start start 2024-04-02T17:33:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 Processor -- start 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 0x56526d116540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b007be90 con 0x56526d09c8d0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b007b7c0 con 0x56526d0abba0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b00722d0 con 0x56526d0a1a40 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.490+0000 7fb9ba79c740 10 client.0 _ll_register_callbacks cb 0x7fb9a400b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9b5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9b5a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 0x56526d116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9b5a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 0x56526d116540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58592/0 (socket says 172.21.15.59:58592) 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9b5a00640 1 -- 172.21.15.59:0/4145145818 learned_addr learned my addr 172.21.15.59:0/4145145818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 -- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 msgr2=0x56526d116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 0x56526d116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 -- 172.21.15.59:0/4145145818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 msgr2=0x56526d0c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 -- 172.21.15.59:0/4145145818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56526d116a80 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb9afe00640 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0c3110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x56526d09d340 tx=0x7fb99c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8f2ab7e441e381 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.491+0000 7fb996a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0b5070 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fb99c09af50 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a8003970 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb99c0a58c0 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb9ba79c740 10 client.27541 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56526d117cb0 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8003970 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.492+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fb99c020070 con 0x56526d0a1a40 2024-04-02T17:33:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 10 client.27541 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 10 client.27541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 20 client.27541 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d118590 0x56526d11a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 4 client.27541 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:33:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.493+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56526cffe2b0 con 0x56526d118590 2024-04-02T17:33:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.519+0000 7fb9b5000640 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d118590 0x56526d11a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.519+0000 7fb9b5000640 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d118590 0x56526d11a9b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b8f2aecba8cd3e server_cookie=f3b2ecd080eaf1e1 in_seq=0 out_seq=0 2024-04-02T17:33:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.519+0000 7fb996a00640 10 client.27541 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:30.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 314969 39.27 MB/sec execute 297 sec latency 77.172 ms 2024-04-02T17:33:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.794+0000 7fb996a00640 1 -- 172.21.15.59:0/4145145818 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56526cffe2b0 con 0x56526d118590 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.794+0000 7fb996a00640 10 client.27541 handle_command_reply: tid=0 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.794+0000 7fb9ba79c740 1 client.27541 shutdown 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 msgr2=0x56526d0c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0c3110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x56526d09d340 tx=0x7fb99c003a90 comp rx=0 tx=0).stop 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 shutdown_connections 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56526d118590 0x56526d11a9b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56526d0abba0 0x56526d116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb996a00640 0 client.27541 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56526d0a1a40 0x56526d0c3110 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 --2- 172.21.15.59:0/4145145818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56526d09c8d0 0x56526d0c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 >> 172.21.15.59:0/4145145818 conn(0x56526d086440 msgr2=0x56526d0a6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 shutdown_connections 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 1 -- 172.21.15.59:0/4145145818 wait complete. 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.795+0000 7fb9ba79c740 20 client.27541 trim_cache size 0 max 16384 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:33:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:33:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 msgr2=0x7fb9980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 0x7fb9980a6190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9b01091d0 tx=0x7fb9a400b390 comp rx=0 tx=0).stop 2024-04-02T17:33:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 msgr2=0x7fb9b019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00ea60 tx=0x7fb99c00ef20 comp rx=0 tx=0).stop 2024-04-02T17:33:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 shutdown_connections 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb9980a3cd0 0x7fb9980a6190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9b0109d30 0x7fb9b01a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9b0109370 0x7fb9b019e960 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 --2- 172.21.15.59:0/1800121291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9b0108180 0x7fb9b019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.797+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 >> 172.21.15.59:0/1800121291 conn(0x7fb9b0077150 msgr2=0x7fb9b010c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.798+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 shutdown_connections 2024-04-02T17:33:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:30.798+0000 7fb9b7a00640 1 -- 172.21.15.59:0/1800121291 wait complete. 2024-04-02T17:33:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:31 smithi164 ceph-mon[36499]: pgmap v1003: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 983 op/s 2024-04-02T17:33:31.634 INFO:tasks.workunit.client.0.smithi059.stdout: 10 315550 39.23 MB/sec execute 298 sec latency 174.072 ms 2024-04-02T17:33:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:31 smithi154 ceph-mon[36479]: pgmap v1003: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 983 op/s 2024-04-02T17:33:31.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:31 smithi059 ceph-mon[31664]: pgmap v1003: 97 pgs: 97 active+clean; 1.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 983 op/s 2024-04-02T17:33:32.635 INFO:tasks.workunit.client.0.smithi059.stdout: 10 316186 39.21 MB/sec execute 299 sec latency 49.736 ms 2024-04-02T17:33:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:33 smithi164 ceph-mon[36499]: pgmap v1004: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-02T17:33:33.635 INFO:tasks.workunit.client.0.smithi059.stdout: 10 316927 39.19 MB/sec execute 300 sec latency 53.913 ms 2024-04-02T17:33:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:33 smithi154 ceph-mon[36479]: pgmap v1004: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-02T17:33:33.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:33 smithi059 ceph-mon[31664]: pgmap v1004: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-02T17:33:34.636 INFO:tasks.workunit.client.0.smithi059.stdout: 10 317871 39.24 MB/sec execute 301 sec latency 81.262 ms 2024-04-02T17:33:35.636 INFO:tasks.workunit.client.0.smithi059.stdout: 10 318519 39.24 MB/sec execute 302 sec latency 109.390 ms 2024-04-02T17:33:36.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:35 smithi154 ceph-mon[36479]: pgmap v1005: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-04-02T17:33:36.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:35 smithi059 ceph-mon[31664]: pgmap v1005: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-04-02T17:33:36.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:35 smithi164 ceph-mon[36499]: pgmap v1005: 97 pgs: 97 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 859 op/s 2024-04-02T17:33:36.637 INFO:tasks.workunit.client.0.smithi059.stdout: 10 319157 39.22 MB/sec execute 303 sec latency 38.921 ms 2024-04-02T17:33:37.638 INFO:tasks.workunit.client.0.smithi059.stdout: 10 319837 39.18 MB/sec execute 304 sec latency 159.954 ms 2024-04-02T17:33:38.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:37 smithi154 ceph-mon[36479]: pgmap v1006: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 771 op/s 2024-04-02T17:33:38.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:37 smithi059 ceph-mon[31664]: pgmap v1006: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 771 op/s 2024-04-02T17:33:38.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:37 smithi164 ceph-mon[36499]: pgmap v1006: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 771 op/s 2024-04-02T17:33:38.638 INFO:tasks.workunit.client.0.smithi059.stdout: 10 320732 39.23 MB/sec execute 305 sec latency 86.017 ms 2024-04-02T17:33:39.638 INFO:tasks.workunit.client.0.smithi059.stdout: 10 321572 39.26 MB/sec execute 306 sec latency 72.250 ms 2024-04-02T17:33:40.101 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:39 smithi059 ceph-mon[31664]: pgmap v1007: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.3 MiB/s wr, 728 op/s 2024-04-02T17:33:40.124 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:39 smithi164 ceph-mon[36499]: pgmap v1007: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.3 MiB/s wr, 728 op/s 2024-04-02T17:33:40.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:39 smithi154 ceph-mon[36479]: pgmap v1007: 97 pgs: 97 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.3 MiB/s wr, 728 op/s 2024-04-02T17:33:40.638 INFO:tasks.workunit.client.0.smithi059.stdout: 10 322187 39.26 MB/sec execute 307 sec latency 52.462 ms 2024-04-02T17:33:41.639 INFO:tasks.workunit.client.0.smithi059.stdout: 10 322671 39.19 MB/sec execute 308 sec latency 72.214 ms 2024-04-02T17:33:42.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:41 smithi154 ceph-mon[36479]: pgmap v1008: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 565 op/s 2024-04-02T17:33:42.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:41 smithi059 ceph-mon[31664]: pgmap v1008: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 565 op/s 2024-04-02T17:33:42.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:41 smithi164 ceph-mon[36499]: pgmap v1008: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.9 MiB/s wr, 565 op/s 2024-04-02T17:33:42.639 INFO:tasks.workunit.client.0.smithi059.stdout: 10 323447 39.19 MB/sec execute 309 sec latency 47.183 ms 2024-04-02T17:33:43.639 INFO:tasks.workunit.client.0.smithi059.stdout: 10 324414 39.24 MB/sec execute 310 sec latency 59.212 ms 2024-04-02T17:33:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:43 smithi154 ceph-mon[36479]: pgmap v1009: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:33:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:43 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:33:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.229+0000 7f097d200640 1 -- 172.21.15.59:0/3501089153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978104f40 msgr2=0x7f09781053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.229+0000 7f097d200640 1 --2- 172.21.15.59:0/3501089153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978104f40 0x7f09781053c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0964009900 tx=0x7f096402bc40 comp rx=0 tx=0).stop 2024-04-02T17:33:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 -- 172.21.15.59:0/3501089153 shutdown_connections 2024-04-02T17:33:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 --2- 172.21.15.59:0/3501089153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978105900 0x7f097810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 --2- 172.21.15.59:0/3501089153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978104f40 0x7f09781053c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 --2- 172.21.15.59:0/3501089153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978103d50 0x7f0978104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 -- 172.21.15.59:0/3501089153 >> 172.21.15.59:0/3501089153 conn(0x7f09780ff2c0 msgr2=0x7f0978101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 -- 172.21.15.59:0/3501089153 shutdown_connections 2024-04-02T17:33:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.230+0000 7f097d200640 1 -- 172.21.15.59:0/3501089153 wait complete. 2024-04-02T17:33:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 Processor -- start 2024-04-02T17:33:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 -- start start 2024-04-02T17:33:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978103d50 0x7f0978195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 0x7f0978196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 0x7f097819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f097810fa00 con 0x7f0978103d50 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f097810f880 con 0x7f0978104f40 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.232+0000 7f097d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f097810fb80 con 0x7f0978105900 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0975400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 0x7f0978196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0975e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978103d50 0x7f0978195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0975400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 0x7f0978196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42112/0 (socket says 172.21.15.59:42112) 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0975400640 1 -- 172.21.15.59:0/2045411717 learned_addr learned my addr 172.21.15.59:0/2045411717 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 0x7f097819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 -- 172.21.15.59:0/2045411717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 msgr2=0x7f0978196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 0x7f0978196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 -- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978103d50 msgr2=0x7f0978195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978103d50 0x7f0978195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.233+0000 7f0976800640 1 -- 172.21.15.59:0/2045411717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781ae900 con 0x7f0978105900 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 0x7f097819d1d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f097000ec50 tx=0x7f097000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f096ea00640 1 -- 172.21.15.59:0/2045411717 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970003b70 con 0x7f0978105900 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f096ea00640 1 -- 172.21.15.59:0/2045411717 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970003d10 con 0x7f0978105900 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f096ea00640 1 -- 172.21.15.59:0/2045411717 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970010890 con 0x7f0978105900 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781aeb90 con 0x7f0978105900 2024-04-02T17:33:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.234+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781af060 con 0x7f0978105900 2024-04-02T17:33:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.238+0000 7f097fe6f740 1 Processor -- start 2024-04-02T17:33:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.242+0000 7f096ea00640 1 -- 172.21.15.59:0/2045411717 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0970003fc0 con 0x7f0978105900 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.242+0000 7f097fe6f740 1 -- start start 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea359e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cea3503d70 con 0x55cea359e8c0 2024-04-02T17:33:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cea34ac2a0 con 0x55cea359ded0 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f097fe6f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cea34ffd20 con 0x55cea35a3a30 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f0975400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea359e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f0975400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea359e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42118/0 (socket says 172.21.15.59:42118) 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f0975400640 1 -- 172.21.15.59:0/84975921 learned_addr learned my addr 172.21.15.59:0/84975921 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.243+0000 7f0975400640 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.244+0000 7f0975e00640 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f096ea00640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 0x7f095c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f096ea00640 1 -- 172.21.15.59:0/2045411717 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f09700d0970 con 0x7f0978105900 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f0975e00640 1 -- 172.21.15.59:0/84975921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 msgr2=0x55cea35a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f0975e00640 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f0975e00640 1 -- 172.21.15.59:0/84975921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 msgr2=0x55cea359e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f0975e00640 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea359e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.245+0000 7f0975e00640 1 -- 172.21.15.59:0/84975921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea34c0840 con 0x55cea359e8c0 2024-04-02T17:33:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.246+0000 7f0975e00640 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35a33c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0968009bf0 tx=0x7f0968011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4159b0d6496be97d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.247+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 0x7f095c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.247+0000 7f0963400640 1 -- 172.21.15.59:0/84975921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968009da0 con 0x55cea359e8c0 2024-04-02T17:33:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.247+0000 7f0963400640 1 -- 172.21.15.59:0/84975921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968005020 con 0x55cea359e8c0 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.247+0000 7f0963400640 1 -- 172.21.15.59:0/84975921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968004b60 con 0x55cea359e8c0 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 -- 172.21.15.59:0/84975921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 msgr2=0x55cea35a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35a33c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0968009bf0 tx=0x7f0968011810 comp rx=0 tx=0).stop 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f0976800640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 0x7f095c0a6190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f097009cb80 tx=0x7f097000ce70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 -- 172.21.15.59:0/84975921 shutdown_connections 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35a33c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 --2- 172.21.15.59:0/84975921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea359e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 -- 172.21.15.59:0/84975921 >> 172.21.15.59:0/84975921 conn(0x55cea3599440 msgr2=0x55cea3599840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 -- 172.21.15.59:0/84975921 shutdown_connections 2024-04-02T17:33:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.248+0000 7f097fe6f740 1 -- 172.21.15.59:0/84975921 wait complete. 2024-04-02T17:33:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.249+0000 7f097fe6f740 1 -- start start 2024-04-02T17:33:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.249+0000 7f097fe6f740 1 Processor -- start 2024-04-02T17:33:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cea35a69e0 con 0x55cea359ded0 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cea34bf660 con 0x55cea359e8c0 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cea35a6b60 con 0x55cea35a3a30 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.250+0000 7f097fe6f740 10 client.0 _ll_register_callbacks cb 0x55cea34bf050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.251+0000 7f0975e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.251+0000 7f0975400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.252+0000 7f0975400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40904/0 (socket says 172.21.15.59:40904) 2024-04-02T17:33:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.252+0000 7f0975400640 1 -- 172.21.15.59:0/3457387411 learned_addr learned my addr 172.21.15.59:0/3457387411 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.252+0000 7f0975400640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975400640 1 -- 172.21.15.59:0/3457387411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 msgr2=0x55cea35afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975400640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975400640 1 -- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 msgr2=0x55cea35a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975400640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975400640 1 -- 172.21.15.59:0/3457387411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea34cee80 con 0x55cea359ded0 2024-04-02T17:33:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.253+0000 7f0975e00640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:33:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.254+0000 7f0975400640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0964006010 tx=0x7f0964004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5dc4d91ad69012 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.254+0000 7f0957e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:33:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.255+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964004f10 con 0x55cea359ded0 2024-04-02T17:33:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.255+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f09640050b0 con 0x55cea359ded0 2024-04-02T17:33:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.255+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0968005020 con 0x55cea359ded0 2024-04-02T17:33:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.255+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f09640052b0 con 0x55cea359ded0 2024-04-02T17:33:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.256+0000 7f097fe6f740 10 client.17373 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.256+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cea3218210 con 0x55cea359ded0 2024-04-02T17:33:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.258+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968005020 con 0x55cea359ded0 2024-04-02T17:33:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.258+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f0964003bf0 con 0x55cea359ded0 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 10 client.17373 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 20 client.17373 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea35b4300 0x55cea35b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 4 client.17373 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:33:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.259+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cea34b3cd0 con 0x55cea35b4300 2024-04-02T17:33:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.260+0000 7f0975e00640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea35b4300 0x55cea35b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.260+0000 7f0975e00640 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea35b4300 0x55cea35b6750 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291f3da7df95780b server_cookie=cee86de5bf6f9990 in_seq=0 out_seq=0 2024-04-02T17:33:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.260+0000 7f0957e00640 10 client.17373 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.263+0000 7f0957e00640 1 -- 172.21.15.59:0/3457387411 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cea34b3cd0 con 0x55cea35b4300 2024-04-02T17:33:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.263+0000 7f0957e00640 10 client.17373 handle_command_reply: tid=0 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.263+0000 7f097fe6f740 1 client.17373 shutdown 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.263+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 msgr2=0x55cea35a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.263+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0964006010 tx=0x7f0964004e50 comp rx=0 tx=0).stop 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 shutdown_connections 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea35b4300 0x55cea35b6750 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea35afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 --2- 172.21.15.59:0/3457387411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359ded0 0x55cea35a84a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f0957e00640 0 client.17373 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.264+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 >> 172.21.15.59:0/3457387411 conn(0x55cea3599440 msgr2=0x55cea3598640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.265+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 shutdown_connections 2024-04-02T17:33:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.265+0000 7f097fe6f740 1 -- 172.21.15.59:0/3457387411 wait complete. 2024-04-02T17:33:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.265+0000 7f097fe6f740 20 client.17373 trim_cache size 0 max 16384 2024-04-02T17:33:44.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:43 smithi059 ceph-mon[31664]: pgmap v1009: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:33:44.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:43 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:33:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 Processor -- start 2024-04-02T17:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 -- start start 2024-04-02T17:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea35a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b8000 con 0x55cea359e8c0 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b9b40 con 0x55cea359ded0 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.302+0000 7f097fe6f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b86d0 con 0x55cea35a3a30 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.303+0000 7f0975e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.303+0000 7f0975400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.303+0000 7f0975400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:59176/0 (socket says 172.21.15.59:59176) 2024-04-02T17:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.303+0000 7f0975400640 1 -- 172.21.15.59:0/2856256813 learned_addr learned my addr 172.21.15.59:0/2856256813 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.303+0000 7f0975400640 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea35a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.304+0000 7f0975400640 1 -- 172.21.15.59:0/2856256813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 msgr2=0x55cea35a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.304+0000 7f0975400640 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea35a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.304+0000 7f0975400640 1 -- 172.21.15.59:0/2856256813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 msgr2=0x55cea35af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.305+0000 7f0975400640 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.305+0000 7f0975400640 1 -- 172.21.15.59:0/2856256813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea32e4620 con 0x55cea35a3a30 2024-04-02T17:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.305+0000 7f0975e00640 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.306+0000 7f0975400640 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0964009340 tx=0x7f096402c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8685a30c45129aff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.307+0000 7f0962a00640 1 -- 172.21.15.59:0/2856256813 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964004f10 con 0x55cea35a3a30 2024-04-02T17:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.307+0000 7f0962a00640 1 -- 172.21.15.59:0/2856256813 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096405e070 con 0x55cea35a3a30 2024-04-02T17:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.307+0000 7f0962a00640 1 -- 172.21.15.59:0/2856256813 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964003ad0 con 0x55cea35a3a30 2024-04-02T17:33:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 -- 172.21.15.59:0/2856256813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 msgr2=0x55cea359cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0964009340 tx=0x7f096402c890 comp rx=0 tx=0).stop 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 -- 172.21.15.59:0/2856256813 shutdown_connections 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea359e8c0 0x55cea35af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea35a3a30 0x55cea359cdb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 --2- 172.21.15.59:0/2856256813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359ded0 0x55cea35a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.308+0000 7f097fe6f740 1 -- 172.21.15.59:0/2856256813 >> 172.21.15.59:0/2856256813 conn(0x55cea3599440 msgr2=0x55cea359c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.309+0000 7f097fe6f740 1 -- 172.21.15.59:0/2856256813 shutdown_connections 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.309+0000 7f097fe6f740 1 -- 172.21.15.59:0/2856256813 wait complete. 2024-04-02T17:33:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.309+0000 7f097fe6f740 1 -- start start 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 Processor -- start 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea359ded0 0x55cea35af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea35a3a30 0x55cea35ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b9470 con 0x55cea35a3a30 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b8da0 con 0x55cea359e8c0 2024-04-02T17:33:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f09781b5fe0 con 0x55cea359ded0 2024-04-02T17:33:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.310+0000 7f097fe6f740 10 client.0 _ll_register_callbacks cb 0x7f09640044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea359ded0 0x55cea35af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42152/0 (socket says 172.21.15.59:42152) 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 -- 172.21.15.59:0/1986404846 learned_addr learned my addr 172.21.15.59:0/1986404846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 -- 172.21.15.59:0/1986404846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea359ded0 msgr2=0x55cea35af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea359ded0 0x55cea35af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.311+0000 7f0975e00640 1 -- 172.21.15.59:0/1986404846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea35a3a30 msgr2=0x55cea35ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f0975e00640 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea35a3a30 0x55cea35ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f0975e00640 1 -- 172.21.15.59:0/1986404846 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea35ae190 con 0x55cea359e8c0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f0975e00640 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0968009bf0 tx=0x7f0968005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c742c544c4583c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f0957e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968005370 con 0x55cea359e8c0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.312+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096405e070 con 0x55cea359e8c0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.313+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f0968016040 con 0x55cea359e8c0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.313+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f0968005510 con 0x55cea359e8c0 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.313+0000 7f097fe6f740 10 client.27500 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.313+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cea361b350 con 0x55cea359e8c0 2024-04-02T17:33:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.313+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096405e070 con 0x55cea359e8c0 2024-04-02T17:33:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f0968036590 con 0x55cea359e8c0 2024-04-02T17:33:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 10 client.27500 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 10 client.27500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 20 client.27500 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea361bb40 0x55cea361df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 4 client.27500 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cea35112b0 con 0x55cea361bb40 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.314+0000 7f0975400640 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea361bb40 0x55cea361df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.315+0000 7f0975400640 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea361bb40 0x55cea361df60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168f6525e63607c7 server_cookie=6bd773e86c993192 in_seq=0 out_seq=0 2024-04-02T17:33:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.315+0000 7f0957e00640 10 client.27500 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:44.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:43 smithi164 ceph-mon[36499]: pgmap v1009: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 645 op/s 2024-04-02T17:33:44.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:43 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:33:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.592+0000 7f0957e00640 1 -- 172.21.15.59:0/1986404846 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cea35112b0 con 0x55cea361bb40 2024-04-02T17:33:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.592+0000 7f0957e00640 10 client.27500 handle_command_reply: tid=0 2024-04-02T17:33:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.592+0000 7f097fe6f740 1 client.27500 shutdown 2024-04-02T17:33:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.592+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 msgr2=0x55cea35af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.592+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0968009bf0 tx=0x7f0968005d30 comp rx=0 tx=0).stop 2024-04-02T17:33:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.593+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 shutdown_connections 2024-04-02T17:33:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.593+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cea361bb40 0x55cea361df60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.593+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cea35a3a30 0x55cea35ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.593+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cea359e8c0 0x55cea35af900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.593+0000 7f097fe6f740 1 --2- 172.21.15.59:0/1986404846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cea359ded0 0x55cea35af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.594+0000 7f0957e00640 0 client.27500 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.594+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 >> 172.21.15.59:0/1986404846 conn(0x55cea3599440 msgr2=0x55cea359cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.595+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 shutdown_connections 2024-04-02T17:33:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.595+0000 7f097fe6f740 1 -- 172.21.15.59:0/1986404846 wait complete. 2024-04-02T17:33:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.595+0000 7f097fe6f740 20 client.27500 trim_cache size 0 max 16384 2024-04-02T17:33:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:33:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:33:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:33:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 msgr2=0x7f095c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 0x7f095c0a6190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f097009cb80 tx=0x7f097000ce70 comp rx=0 tx=0).stop 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 msgr2=0x7f097819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 0x7f097819d1d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f097000ec50 tx=0x7f097000c6a0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 shutdown_connections 2024-04-02T17:33:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0978105900 0x7f097819d1d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0978104f40 0x7f0978196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f095c0a3cd0 0x7f095c0a6190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 --2- 172.21.15.59:0/2045411717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0978103d50 0x7f0978195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.597+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 >> 172.21.15.59:0/2045411717 conn(0x7f09780ff2c0 msgr2=0x7f0978077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.598+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 shutdown_connections 2024-04-02T17:33:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:44.598+0000 7f097d200640 1 -- 172.21.15.59:0/2045411717 wait complete. 2024-04-02T17:33:44.639 INFO:tasks.workunit.client.0.smithi059.stdout: 10 325040 39.25 MB/sec execute 311 sec latency 208.995 ms 2024-04-02T17:33:45.639 INFO:tasks.workunit.client.0.smithi059.stdout: 10 325527 39.20 MB/sec execute 312 sec latency 292.124 ms 2024-04-02T17:33:46.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:45 smithi154 ceph-mon[36479]: pgmap v1010: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-04-02T17:33:46.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:45 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:45 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:45 smithi059 ceph-mon[31664]: pgmap v1010: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-04-02T17:33:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:45 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:45 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:45 smithi164 ceph-mon[36499]: pgmap v1010: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-04-02T17:33:46.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:45 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:45 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:46.640 INFO:tasks.workunit.client.0.smithi059.stdout: 10 325582 39.09 MB/sec execute 313 sec latency 871.796 ms 2024-04-02T17:33:47.640 INFO:tasks.workunit.client.0.smithi059.stdout: 10 325614 38.97 MB/sec execute 314 sec latency 1663.103 ms 2024-04-02T17:33:48.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:47 smithi154 ceph-mon[36479]: pgmap v1011: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-04-02T17:33:48.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:47 smithi059 ceph-mon[31664]: pgmap v1011: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-04-02T17:33:48.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:47 smithi164 ceph-mon[36499]: pgmap v1011: 97 pgs: 97 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-04-02T17:33:48.641 INFO:tasks.workunit.client.0.smithi059.stdout: 10 325881 38.88 MB/sec execute 315 sec latency 555.771 ms 2024-04-02T17:33:49.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 326100 38.78 MB/sec execute 316 sec latency 348.978 ms 2024-04-02T17:33:50.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:49 smithi164 ceph-mon[36499]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2024-04-02T17:33:50.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:49 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:49 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:49 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:49 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:49 smithi154 ceph-mon[36479]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2024-04-02T17:33:50.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:49 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:49 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:49 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:49 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:49 smithi059 ceph-mon[31664]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2024-04-02T17:33:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:49 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:49 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:49 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:49 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:50.641 INFO:tasks.workunit.client.0.smithi059.stdout: 10 326356 38.71 MB/sec execute 317 sec latency 363.901 ms 2024-04-02T17:33:51.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 327114 38.69 MB/sec execute 318 sec latency 69.537 ms 2024-04-02T17:33:52.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:51 smithi154 ceph-mon[36479]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:33:52.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:51 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:51 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:51 smithi059 ceph-mon[31664]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:33:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:51 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:51 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:51 smithi164 ceph-mon[36499]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:33:52.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:51 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:51 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:52.641 INFO:tasks.workunit.client.0.smithi059.stdout: 10 327792 38.69 MB/sec execute 319 sec latency 102.396 ms 2024-04-02T17:33:53.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 328613 38.74 MB/sec execute 320 sec latency 116.877 ms 2024-04-02T17:33:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:54 smithi154 ceph-mon[36479]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 908 op/s 2024-04-02T17:33:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:54 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:54 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:54 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:54 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:54 smithi059 ceph-mon[31664]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 908 op/s 2024-04-02T17:33:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:54 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:54 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:54 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:54 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:54 smithi164 ceph-mon[36499]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 908 op/s 2024-04-02T17:33:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:54 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:54 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:54 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:54 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:54.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 328947 38.67 MB/sec execute 321 sec latency 323.640 ms 2024-04-02T17:33:55.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 329514 38.63 MB/sec execute 322 sec latency 61.995 ms 2024-04-02T17:33:56.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: pgmap v1015: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-04-02T17:33:56.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:33:56.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:33:56.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:55 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: pgmap v1015: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-04-02T17:33:56.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:33:56.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:33:56.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:55 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: pgmap v1015: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-04-02T17:33:56.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:33:56.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:33:56.150 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:55 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:33:56.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 330282 38.64 MB/sec execute 323 sec latency 46.418 ms 2024-04-02T17:33:57.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 331126 38.64 MB/sec execute 324 sec latency 62.770 ms 2024-04-02T17:33:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.834+0000 7f7555e00640 1 -- 172.21.15.59:0/2839339575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550103d50 msgr2=0x7f7550104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.834+0000 7f7555e00640 1 --2- 172.21.15.59:0/2839339575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550103d50 0x7f7550104170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7540009900 tx=0x7f754002bc30 comp rx=0 tx=0).stop 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 -- 172.21.15.59:0/2839339575 shutdown_connections 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 --2- 172.21.15.59:0/2839339575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7550105900 0x7f755010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 --2- 172.21.15.59:0/2839339575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550104f40 0x7f75501053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 --2- 172.21.15.59:0/2839339575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550103d50 0x7f7550104170 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7540009900 tx=0x7f754002bc30 comp rx=0 tx=0).stop 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 -- 172.21.15.59:0/2839339575 >> 172.21.15.59:0/2839339575 conn(0x7f75500ff2c0 msgr2=0x7f7550101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 -- 172.21.15.59:0/2839339575 shutdown_connections 2024-04-02T17:33:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.835+0000 7f7555e00640 1 -- 172.21.15.59:0/2839339575 wait complete. 2024-04-02T17:33:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.836+0000 7f7555e00640 1 Processor -- start 2024-04-02T17:33:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.836+0000 7f7555e00640 1 -- start start 2024-04-02T17:33:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550104f40 0x7f755019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550105900 0x7f755019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f755010f9f0 con 0x7f7550104f40 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f755010f870 con 0x7f755019f750 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.837+0000 7f7555e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f755010fb70 con 0x7f7550105900 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550104f40 0x7f755019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550105900 0x7f755019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53346/0 (socket says 172.21.15.59:53346) 2024-04-02T17:33:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 -- 172.21.15.59:0/642738758 learned_addr learned my addr 172.21.15.59:0/642738758 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 -- 172.21.15.59:0/642738758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550105900 msgr2=0x7f755019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550105900 0x7f755019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 -- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550104f40 msgr2=0x7f755019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550104f40 0x7f755019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.838+0000 7f754f400640 1 -- 172.21.15.59:0/642738758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75501a21b0 con 0x7f755019f750 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f754f400640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f754800eb10 tx=0x7f754800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f7554a00640 1 -- 172.21.15.59:0/642738758 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548004b20 con 0x7f755019f750 2024-04-02T17:33:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7550075fa0 con 0x7f755019f750 2024-04-02T17:33:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f7554a00640 1 -- 172.21.15.59:0/642738758 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548004cc0 con 0x7f755019f750 2024-04-02T17:33:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f7554a00640 1 -- 172.21.15.59:0/642738758 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548010760 con 0x7f755019f750 2024-04-02T17:33:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.839+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75500764e0 con 0x7f755019f750 2024-04-02T17:33:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.845+0000 7f7558a71740 1 Processor -- start 2024-04-02T17:33:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.847+0000 7f7554a00640 1 -- 172.21.15.59:0/642738758 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7548005020 con 0x7f755019f750 2024-04-02T17:33:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.848+0000 7f7558a71740 1 -- start start 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.848+0000 7f7558a71740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e0e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.848+0000 7f7558a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 0x55cbf2e133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f7558a71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e13a30 0x55cbf2e15ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f7558a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2d73d70 con 0x55cbf2e0e8c0 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f7558a71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2d1c2a0 con 0x55cbf2e13a30 2024-04-02T17:33:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f7558a71740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2d6fd20 con 0x55cbf2e0ded0 2024-04-02T17:33:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f754e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e0e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f754e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e0e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55326/0 (socket says 172.21.15.59:55326) 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f754e000640 1 -- 172.21.15.59:0/3511515351 learned_addr learned my addr 172.21.15.59:0/3511515351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f754e000640 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e13a30 0x55cbf2e15ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.849+0000 7f754ea00640 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 0x55cbf2e133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f7554a00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 0x7f75340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f754ea00640 1 -- 172.21.15.59:0/3511515351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 msgr2=0x55cbf2e0e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f754ea00640 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e0e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f754ea00640 1 -- 172.21.15.59:0/3511515351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e13a30 msgr2=0x55cbf2e15ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f754ea00640 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e13a30 0x55cbf2e15ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f754ea00640 1 -- 172.21.15.59:0/3511515351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf2d30840 con 0x55cbf2e0e8c0 2024-04-02T17:33:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.850+0000 7f7554a00640 1 -- 172.21.15.59:0/642738758 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f75480d0c50 con 0x7f755019f750 2024-04-02T17:33:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.852+0000 7f754f400640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 0x7f75340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.853+0000 7f754ea00640 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 0x55cbf2e133c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7550103890 tx=0x7f7540005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f583e925f873a4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.854+0000 7f754f400640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 0x7f75340a61e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7548005ba0 tx=0x7f75480058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.855+0000 7f753be00640 1 -- 172.21.15.59:0/3511515351 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540043020 con 0x55cbf2e0e8c0 2024-04-02T17:33:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.855+0000 7f753be00640 1 -- 172.21.15.59:0/3511515351 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7540004040 con 0x55cbf2e0e8c0 2024-04-02T17:33:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.855+0000 7f753be00640 1 -- 172.21.15.59:0/3511515351 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540005440 con 0x55cbf2e0e8c0 2024-04-02T17:33:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 -- 172.21.15.59:0/3511515351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 msgr2=0x55cbf2e133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 0x55cbf2e133c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7550103890 tx=0x7f7540005b50 comp rx=0 tx=0).stop 2024-04-02T17:33:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 -- 172.21.15.59:0/3511515351 shutdown_connections 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0e8c0 0x55cbf2e133c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e13a30 0x55cbf2e15ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.856+0000 7f7558a71740 1 --2- 172.21.15.59:0/3511515351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e0e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.857+0000 7f7558a71740 1 -- 172.21.15.59:0/3511515351 >> 172.21.15.59:0/3511515351 conn(0x55cbf2e09440 msgr2=0x55cbf2e09840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.857+0000 7f7558a71740 1 -- 172.21.15.59:0/3511515351 shutdown_connections 2024-04-02T17:33:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.857+0000 7f7558a71740 1 -- 172.21.15.59:0/3511515351 wait complete. 2024-04-02T17:33:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.858+0000 7f7558a71740 1 -- start start 2024-04-02T17:33:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.858+0000 7f7558a71740 1 Processor -- start 2024-04-02T17:33:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e18530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e13a30 0x55cbf2e18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2e16980 con 0x55cbf2e13a30 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2d2f660 con 0x55cbf2e0ded0 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2e16b00 con 0x55cbf2e1daa0 2024-04-02T17:33:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.859+0000 7f7558a71740 10 client.0 _ll_register_callbacks cb 0x55cbf2d2f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.860+0000 7f754e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.860+0000 7f754e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e18530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.860+0000 7f754e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55336/0 (socket says 172.21.15.59:55336) 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.860+0000 7f754e000640 1 -- 172.21.15.59:0/3472823072 learned_addr learned my addr 172.21.15.59:0/3472823072 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 -- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 msgr2=0x55cbf2e18530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754ea00640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e13a30 0x55cbf2e18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e18530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 -- 172.21.15.59:0/3472823072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e13a30 msgr2=0x55cbf2e18a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e13a30 0x55cbf2e18a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 -- 172.21.15.59:0/3472823072 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf2d3ee80 con 0x55cbf2e1daa0 2024-04-02T17:33:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f754e000640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f753c0058c0 tx=0x7f753c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bb915e719bc40b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.861+0000 7f7538c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:33:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.862+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c005020 con 0x55cbf2e1daa0 2024-04-02T17:33:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.862+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f753c0051c0 con 0x55cbf2e1daa0 2024-04-02T17:33:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.863+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7540004040 con 0x55cbf2e1daa0 2024-04-02T17:33:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.863+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f753c0053c0 con 0x55cbf2e1daa0 2024-04-02T17:33:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.863+0000 7f7558a71740 10 client.27565 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.863+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cbf2a88210 con 0x55cbf2e1daa0 2024-04-02T17:33:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.863+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7540004040 con 0x55cbf2e1daa0 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f753c00f040 con 0x55cbf2e1daa0 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 10 client.27565 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 20 client.27565 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2e35730 0x55cbf2e37bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 4 client.27565 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.864+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf2d23cd0 con 0x55cbf2e35730 2024-04-02T17:33:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.865+0000 7f754ea00640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2e35730 0x55cbf2e37bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.868+0000 7f754ea00640 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2e35730 0x55cbf2e37bc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35159ce0e78735ca server_cookie=dcb532fa857a5eca in_seq=0 out_seq=0 2024-04-02T17:33:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.868+0000 7f7538c00640 10 client.27565 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.872+0000 7f7538c00640 1 -- 172.21.15.59:0/3472823072 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbf2d23cd0 con 0x55cbf2e35730 2024-04-02T17:33:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.872+0000 7f7538c00640 10 client.27565 handle_command_reply: tid=0 2024-04-02T17:33:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.872+0000 7f7558a71740 1 client.27565 shutdown 2024-04-02T17:33:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 msgr2=0x55cbf2e1fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f753c0058c0 tx=0x7f753c0046c0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 shutdown_connections 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2e35730 0x55cbf2e37bc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e13a30 0x55cbf2e18a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e1fe90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 --2- 172.21.15.59:0/3472823072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e18530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.873+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 >> 172.21.15.59:0/3472823072 conn(0x55cbf2e09440 msgr2=0x55cbf2e08640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.874+0000 7f7538c00640 0 client.27565 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.874+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 shutdown_connections 2024-04-02T17:33:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.874+0000 7f7558a71740 1 -- 172.21.15.59:0/3472823072 wait complete. 2024-04-02T17:33:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.874+0000 7f7558a71740 20 client.27565 trim_cache size 0 max 16384 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 Processor -- start 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 -- start start 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e07db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e08830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75500706f0 con 0x55cbf2e1daa0 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.899+0000 7f7558a71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7550072230 con 0x55cbf2e0ded0 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f7558a71740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7550070dc0 con 0x55cbf2e13a30 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f754ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f754e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e07db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f754ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:55350/0 (socket says 172.21.15.59:55350) 2024-04-02T17:33:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f754ea00640 1 -- 172.21.15.59:0/381481601 learned_addr learned my addr 172.21.15.59:0/381481601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.900+0000 7f754e000640 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e08830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 -- 172.21.15.59:0/381481601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 msgr2=0x55cbf2e07db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e07db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 -- 172.21.15.59:0/381481601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 msgr2=0x55cbf2e08830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e08830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 -- 172.21.15.59:0/381481601 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf2b54620 con 0x55cbf2e13a30 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754e000640 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e08830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.901+0000 7f754ea00640 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7540005c40 tx=0x7f7540000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5166969e06bbbdc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.902+0000 7f753b400640 1 -- 172.21.15.59:0/381481601 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540043040 con 0x55cbf2e13a30 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.902+0000 7f753b400640 1 -- 172.21.15.59:0/381481601 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75400044b0 con 0x55cbf2e13a30 2024-04-02T17:33:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.902+0000 7f753b400640 1 -- 172.21.15.59:0/381481601 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400055e0 con 0x55cbf2e13a30 2024-04-02T17:33:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.902+0000 7f7558a71740 1 -- 172.21.15.59:0/381481601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 msgr2=0x55cbf2e082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.902+0000 7f7558a71740 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7540005c40 tx=0x7f7540000bc0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 -- 172.21.15.59:0/381481601 shutdown_connections 2024-04-02T17:33:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e08830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e082f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 --2- 172.21.15.59:0/381481601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e07db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 -- 172.21.15.59:0/381481601 >> 172.21.15.59:0/381481601 conn(0x55cbf2e09440 msgr2=0x55cbf2e37580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 -- 172.21.15.59:0/381481601 shutdown_connections 2024-04-02T17:33:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 -- 172.21.15.59:0/381481601 wait complete. 2024-04-02T17:33:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.903+0000 7f7558a71740 1 -- start start 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 Processor -- start 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e46550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e47aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7550071b60 con 0x55cbf2e0ded0 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7550071490 con 0x55cbf2e1daa0 2024-04-02T17:33:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.904+0000 7f7558a71740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f755006e6b0 con 0x55cbf2e13a30 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f7558a71740 10 client.0 _ll_register_callbacks cb 0x7f753c014b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f754e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e46550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f754ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f754e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e46550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43968/0 (socket says 172.21.15.59:43968) 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f754e000640 1 -- 172.21.15.59:0/3176190353 learned_addr learned my addr 172.21.15.59:0/3176190353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.905+0000 7f754e000640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e47aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.906+0000 7f754ea00640 1 -- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e1daa0 msgr2=0x55cbf2e47aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.906+0000 7f754ea00640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e47aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.906+0000 7f754ea00640 1 -- 172.21.15.59:0/3176190353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 msgr2=0x55cbf2e46550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.906+0000 7f754ea00640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e46550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.906+0000 7f754ea00640 1 -- 172.21.15.59:0/3176190353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf2e48090 con 0x55cbf2e13a30 2024-04-02T17:33:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.907+0000 7f754ea00640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e46a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f754002c7b0 tx=0x7f75400043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfcb9dd5e2d764f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.908+0000 7f7538c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.908+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540043070 con 0x55cbf2e13a30 2024-04-02T17:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.908+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f7540005360 con 0x55cbf2e13a30 2024-04-02T17:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.909+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75400044b0 con 0x55cbf2e13a30 2024-04-02T17:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.909+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7540043070 con 0x55cbf2e13a30 2024-04-02T17:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.909+0000 7f7558a71740 10 client.27571 fetch_fsmap learned FSMap version 59 2024-04-02T17:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.909+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cbf2e493a0 con 0x55cbf2e13a30 2024-04-02T17:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.910+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7540056070 con 0x55cbf2e13a30 2024-04-02T17:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.910+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f7540052430 con 0x55cbf2e13a30 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.910+0000 7f7558a71740 10 client.27571 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.910+0000 7f7558a71740 10 client.27571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.910+0000 7f7558a71740 20 client.27571 populate_metadata read hostname 'smithi059' 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.911+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2eacb00 0x55cbf2eaef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.911+0000 7f7558a71740 4 client.27571 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.911+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f75400044b0 con 0x55cbf2eacb00 2024-04-02T17:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.911+0000 7f754e000640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2eacb00 0x55cbf2eaef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.912+0000 7f7538c00640 10 client.27571 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:57.913+0000 7f754e000640 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2eacb00 0x55cbf2eaef20 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be30b2af65864582 server_cookie=3f44ffeb18e840c6 in_seq=0 out_seq=0 2024-04-02T17:33:58.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:57 smithi059 ceph-mon[31664]: pgmap v1016: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 935 op/s 2024-04-02T17:33:58.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:57 smithi164 ceph-mon[36499]: pgmap v1016: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 935 op/s 2024-04-02T17:33:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.072+0000 7f7538c00640 1 -- 172.21.15.59:0/3176190353 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f75400044b0 con 0x55cbf2eacb00 2024-04-02T17:33:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.072+0000 7f7538c00640 10 client.27571 handle_command_reply: tid=0 2024-04-02T17:33:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.072+0000 7f7558a71740 1 client.27571 shutdown 2024-04-02T17:33:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.073+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 msgr2=0x55cbf2e46a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.073+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e46a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f754002c7b0 tx=0x7f75400043a0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 shutdown_connections 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cbf2eacb00 0x55cbf2eaef20 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cbf2e1daa0 0x55cbf2e47aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbf2e13a30 0x55cbf2e46a90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 --2- 172.21.15.59:0/3176190353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cbf2e0ded0 0x55cbf2e46550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 >> 172.21.15.59:0/3176190353 conn(0x55cbf2e09440 msgr2=0x55cbf2e0c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7538c00640 0 client.27571 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:33:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 shutdown_connections 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 1 -- 172.21.15.59:0/3176190353 wait complete. 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.076+0000 7f7558a71740 20 client.27571 trim_cache size 0 max 16384 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:33:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:33:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 msgr2=0x7f75340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 0x7f75340a61e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7548005ba0 tx=0x7f75480058d0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 msgr2=0x7f75501a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:33:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f754800eb10 tx=0x7f754800c2d0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 shutdown_connections 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f755019f750 0x7f75501a1b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7550105900 0x7f755019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f75340a3d20 0x7f75340a61e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.077+0000 7f7555e00640 1 --2- 172.21.15.59:0/642738758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7550104f40 0x7f755019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.078+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 >> 172.21.15.59:0/642738758 conn(0x7f75500ff2c0 msgr2=0x7f75501082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.078+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 shutdown_connections 2024-04-02T17:33:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:33:58.078+0000 7f7555e00640 1 -- 172.21.15.59:0/642738758 wait complete. 2024-04-02T17:33:58.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:57 smithi154 ceph-mon[36479]: pgmap v1016: 97 pgs: 97 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 935 op/s 2024-04-02T17:33:58.642 INFO:tasks.workunit.client.0.smithi059.stdout: 10 331868 38.69 MB/sec execute 325 sec latency 103.062 ms 2024-04-02T17:33:59.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 332251 38.63 MB/sec execute 326 sec latency 391.406 ms 2024-04-02T17:34:00.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:33:59 smithi164 ceph-mon[36499]: pgmap v1017: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 972 op/s 2024-04-02T17:34:00.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:33:59 smithi154 ceph-mon[36479]: pgmap v1017: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 972 op/s 2024-04-02T17:34:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:33:59 smithi059 ceph-mon[31664]: pgmap v1017: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 972 op/s 2024-04-02T17:34:00.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 332868 38.59 MB/sec execute 327 sec latency 86.341 ms 2024-04-02T17:34:01.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 333314 38.55 MB/sec execute 328 sec latency 354.083 ms 2024-04-02T17:34:02.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:01 smithi164 ceph-mon[36499]: pgmap v1018: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-04-02T17:34:02.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:01 smithi154 ceph-mon[36479]: pgmap v1018: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-04-02T17:34:02.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:01 smithi059 ceph-mon[31664]: pgmap v1018: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-04-02T17:34:02.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 334125 38.57 MB/sec execute 329 sec latency 50.385 ms 2024-04-02T17:34:03.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 334553 38.52 MB/sec execute 330 sec latency 402.327 ms 2024-04-02T17:34:04.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:03 smithi164 ceph-mon[36499]: pgmap v1019: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-02T17:34:04.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:03 smithi154 ceph-mon[36479]: pgmap v1019: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-02T17:34:04.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:03 smithi059 ceph-mon[31664]: pgmap v1019: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-02T17:34:04.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 335416 38.55 MB/sec execute 331 sec latency 49.433 ms 2024-04-02T17:34:05.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 336097 38.55 MB/sec execute 332 sec latency 76.107 ms 2024-04-02T17:34:06.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:05 smithi164 ceph-mon[36499]: pgmap v1020: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.6 MiB/s wr, 768 op/s 2024-04-02T17:34:06.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:05 smithi154 ceph-mon[36479]: pgmap v1020: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.6 MiB/s wr, 768 op/s 2024-04-02T17:34:06.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:05 smithi059 ceph-mon[31664]: pgmap v1020: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.6 MiB/s wr, 768 op/s 2024-04-02T17:34:06.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 336797 38.55 MB/sec execute 333 sec latency 38.644 ms 2024-04-02T17:34:07.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 337692 38.56 MB/sec execute 334 sec latency 38.883 ms 2024-04-02T17:34:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:07 smithi164 ceph-mon[36499]: pgmap v1021: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-04-02T17:34:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:07 smithi154 ceph-mon[36479]: pgmap v1021: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-04-02T17:34:08.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:07 smithi059 ceph-mon[31664]: pgmap v1021: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 679 op/s 2024-04-02T17:34:08.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 338705 38.62 MB/sec execute 335 sec latency 46.123 ms 2024-04-02T17:34:09.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 339412 38.63 MB/sec execute 336 sec latency 45.629 ms 2024-04-02T17:34:10.100 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:09 smithi059 ceph-mon[31664]: pgmap v1022: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2024-04-02T17:34:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:09 smithi154 ceph-mon[36479]: pgmap v1022: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2024-04-02T17:34:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:09 smithi164 ceph-mon[36499]: pgmap v1022: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2024-04-02T17:34:10.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 340058 38.60 MB/sec execute 337 sec latency 51.748 ms 2024-04-02T17:34:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.272+0000 7f90f5e00640 1 -- 172.21.15.59:0/1716538559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 msgr2=0x7f90f0104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.272+0000 7f90f5e00640 1 --2- 172.21.15.59:0/1716538559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 0x7f90f0104fa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90dc01a910 tx=0x7f90dc03cc70 comp rx=0 tx=0).stop 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.273+0000 7f90f5e00640 1 -- 172.21.15.59:0/1716538559 shutdown_connections 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.273+0000 7f90f5e00640 1 --2- 172.21.15.59:0/1716538559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01054e0 0x7f90f0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.273+0000 7f90f5e00640 1 --2- 172.21.15.59:0/1716538559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 0x7f90f0104fa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.273+0000 7f90f5e00640 1 --2- 172.21.15.59:0/1716538559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01001c0 0x7f90f01025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.273+0000 7f90f5e00640 1 -- 172.21.15.59:0/1716538559 >> 172.21.15.59:0/1716538559 conn(0x7f90f00fbde0 msgr2=0x7f90f00fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.274+0000 7f90f5e00640 1 -- 172.21.15.59:0/1716538559 shutdown_connections 2024-04-02T17:34:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.274+0000 7f90f5e00640 1 -- 172.21.15.59:0/1716538559 wait complete. 2024-04-02T17:34:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 Processor -- start 2024-04-02T17:34:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 -- start start 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 0x7f90f0195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01054e0 0x7f90f019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0109420 con 0x7f90f01054e0 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01092a0 con 0x7f90f0102b30 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.276+0000 7f90f5e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01095a0 con 0x7f90f01001c0 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44748/0 (socket says 172.21.15.59:44748) 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 -- 172.21.15.59:0/810331117 learned_addr learned my addr 172.21.15.59:0/810331117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f5400640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01054e0 0x7f90f019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 -- 172.21.15.59:0/810331117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 msgr2=0x7f90f0195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 0x7f90f0195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 -- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01054e0 msgr2=0x7f90f019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01054e0 0x7f90f019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.277+0000 7f90f4a00640 1 -- 172.21.15.59:0/810331117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019d590 con 0x7f90f01001c0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.278+0000 7f90f4a00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f90e400db80 tx=0x7f90e400b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.278+0000 7f90ed600640 1 -- 172.21.15.59:0/810331117 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4017070 con 0x7f90f01001c0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.278+0000 7f90ed600640 1 -- 172.21.15.59:0/810331117 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e4003b70 con 0x7f90f01001c0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.278+0000 7f90ed600640 1 -- 172.21.15.59:0/810331117 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e40106c0 con 0x7f90f01001c0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.278+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019d880 con 0x7f90f01001c0 2024-04-02T17:34:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.279+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f019dd80 con 0x7f90f01001c0 2024-04-02T17:34:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.283+0000 7f90ed600640 1 -- 172.21.15.59:0/810331117 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90e4004020 con 0x7f90f01001c0 2024-04-02T17:34:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 Processor -- start 2024-04-02T17:34:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 -- start start 2024-04-02T17:34:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55586613bd70 con 0x5558661d6af0 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5558660e42a0 con 0x5558661dbc60 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.284+0000 7f90fa177740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555866137d20 con 0x5558661d6100 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.285+0000 7f90f5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.285+0000 7f90f5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44756/0 (socket says 172.21.15.59:44756) 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.285+0000 7f90f5400640 1 -- 172.21.15.59:0/561481693 learned_addr learned my addr 172.21.15.59:0/561481693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.285+0000 7f90efe00640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.285+0000 7f90f4a00640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f5400640 1 -- 172.21.15.59:0/561481693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 msgr2=0x5558661de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f5400640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f5400640 1 -- 172.21.15.59:0/561481693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 msgr2=0x5558661db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f5400640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f5400640 1 -- 172.21.15.59:0/561481693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558660f8840 con 0x5558661d6100 2024-04-02T17:34:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.287+0000 7f90f4a00640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.288+0000 7f90f5400640 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b6b0 tx=0x7f90e8004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7bc65d1e4aa35a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.288+0000 7f90e1c00640 1 -- 172.21.15.59:0/561481693 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003b20 con 0x5558661d6100 2024-04-02T17:34:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.289+0000 7f90e1c00640 1 -- 172.21.15.59:0/561481693 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e8015070 con 0x5558661d6100 2024-04-02T17:34:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.289+0000 7f90e1c00640 1 -- 172.21.15.59:0/561481693 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80116a0 con 0x5558661d6100 2024-04-02T17:34:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.289+0000 7f90fa177740 1 -- 172.21.15.59:0/561481693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 msgr2=0x5558661d6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.289+0000 7f90fa177740 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b6b0 tx=0x7f90e8004610 comp rx=0 tx=0).stop 2024-04-02T17:34:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.289+0000 7f90ed600640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 0x7f90d80a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90fa177740 1 -- 172.21.15.59:0/561481693 shutdown_connections 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90fa177740 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90fa177740 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90fa177740 1 --2- 172.21.15.59:0/561481693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6100 0x5558661d6520 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90fa177740 1 -- 172.21.15.59:0/561481693 >> 172.21.15.59:0/561481693 conn(0x5558661d1670 msgr2=0x5558661d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90ed600640 1 -- 172.21.15.59:0/810331117 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f90e40d0230 con 0x7f90f01001c0 2024-04-02T17:34:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.290+0000 7f90f4a00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 0x7f90d80a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.291+0000 7f90fa177740 1 -- 172.21.15.59:0/561481693 shutdown_connections 2024-04-02T17:34:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.291+0000 7f90fa177740 1 -- 172.21.15.59:0/561481693 wait complete. 2024-04-02T17:34:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.292+0000 7f90f4a00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 0x7f90d80a6120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f90e40ce040 tx=0x7f90e400de80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.292+0000 7f90fa177740 1 -- start start 2024-04-02T17:34:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.292+0000 7f90fa177740 1 Processor -- start 2024-04-02T17:34:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661dbc60 0x5558661e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5558661dec10 con 0x5558661d6af0 2024-04-02T17:34:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5558660f7660 con 0x5558661d6100 2024-04-02T17:34:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558661ded90 con 0x5558661dbc60 2024-04-02T17:34:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.293+0000 7f90fa177740 10 client.0 _ll_register_callbacks cb 0x5558660f7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40750/0 (socket says 172.21.15.59:40750) 2024-04-02T17:34:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 -- 172.21.15.59:0/2221930663 learned_addr learned my addr 172.21.15.59:0/2221930663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661dbc60 0x5558661e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 -- 172.21.15.59:0/2221930663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661dbc60 msgr2=0x5558661e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661dbc60 0x5558661e7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 -- 172.21.15.59:0/2221930663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 msgr2=0x5558661e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661e0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.294+0000 7f90f5400640 1 -- 172.21.15.59:0/2221930663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555866106e80 con 0x5558661d6100 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.295+0000 7f90f5400640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90e801c040 tx=0x7f90e801bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41cc6b109fe1ae5c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.295+0000 7f90d6000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.295+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e802e060 con 0x5558661d6100 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.295+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f90e8011840 con 0x5558661d6100 2024-04-02T17:34:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.295+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8015070 con 0x5558661d6100 2024-04-02T17:34:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.296+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f90e8011a40 con 0x5558661d6100 2024-04-02T17:34:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 10 client.27527 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555865e50210 con 0x5558661d6100 2024-04-02T17:34:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e8015070 con 0x5558661d6100 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f90e8029500 con 0x5558661d6100 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 10 client.27527 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 10 client.27527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 20 client.27527 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558661ec140 0x5558661ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 4 client.27527 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:34:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.297+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558660ebcd0 con 0x5558661ec140 2024-04-02T17:34:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.298+0000 7f90efe00640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558661ec140 0x5558661ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.298+0000 7f90efe00640 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558661ec140 0x5558661ee530 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbf0e089d696c3d server_cookie=cf4ddf7945e12ac1 in_seq=0 out_seq=0 2024-04-02T17:34:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.298+0000 7f90d6000640 10 client.27527 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.299+0000 7f90d6000640 1 -- 172.21.15.59:0/2221930663 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558660ebcd0 con 0x5558661ec140 2024-04-02T17:34:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.299+0000 7f90d6000640 10 client.27527 handle_command_reply: tid=0 2024-04-02T17:34:11.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 client.27527 shutdown 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 msgr2=0x5558661e0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90e801c040 tx=0x7f90e801bfd0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 shutdown_connections 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5558661ec140 0x5558661ee530 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90d6000640 0 client.27527 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6af0 0x5558661e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661dbc60 0x5558661e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.300+0000 7f90fa177740 1 --2- 172.21.15.59:0/2221930663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661d6100 0x5558661e0620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.301+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 >> 172.21.15.59:0/2221930663 conn(0x5558661d1670 msgr2=0x5558661d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.301+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 shutdown_connections 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.301+0000 7f90fa177740 1 -- 172.21.15.59:0/2221930663 wait complete. 2024-04-02T17:34:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.301+0000 7f90fa177740 20 client.27527 trim_cache size 0 max 16384 2024-04-02T17:34:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.332+0000 7f90fa177740 1 Processor -- start 2024-04-02T17:34:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.332+0000 7f90fa177740 1 -- start start 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a6e20 con 0x5558661d6100 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a8960 con 0x5558661dbc60 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90fa177740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a74f0 con 0x5558661d6af0 2024-04-02T17:34:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.333+0000 7f90f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36242/0 (socket says 172.21.15.59:36242) 2024-04-02T17:34:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 -- 172.21.15.59:0/2575864228 learned_addr learned my addr 172.21.15.59:0/2575864228 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 -- 172.21.15.59:0/2575864228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 msgr2=0x5558661e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 -- 172.21.15.59:0/2575864228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 msgr2=0x5558661cffe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661cffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.334+0000 7f90f5400640 1 -- 172.21.15.59:0/2575864228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555865f1c620 con 0x5558661d6100 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.335+0000 7f90f5400640 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90e8003670 tx=0x7f90e8011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cd779efd9eacc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.336+0000 7f90e1200640 1 -- 172.21.15.59:0/2575864228 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e802e040 con 0x5558661d6100 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.336+0000 7f90e1200640 1 -- 172.21.15.59:0/2575864228 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e8015070 con 0x5558661d6100 2024-04-02T17:34:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.336+0000 7f90e1200640 1 -- 172.21.15.59:0/2575864228 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8029b90 con 0x5558661d6100 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 -- 172.21.15.59:0/2575864228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 msgr2=0x5558661e0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90e8003670 tx=0x7f90e8011560 comp rx=0 tx=0).stop 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 -- 172.21.15.59:0/2575864228 shutdown_connections 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x5558661cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 --2- 172.21.15.59:0/2575864228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e0940 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.337+0000 7f90fa177740 1 -- 172.21.15.59:0/2575864228 >> 172.21.15.59:0/2575864228 conn(0x5558661d1670 msgr2=0x5558661e0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.338+0000 7f90fa177740 1 -- 172.21.15.59:0/2575864228 shutdown_connections 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.338+0000 7f90fa177740 1 -- 172.21.15.59:0/2575864228 wait complete. 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.338+0000 7f90fa177740 1 -- start start 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.339+0000 7f90fa177740 1 Processor -- start 2024-04-02T17:34:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x555866250570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a8290 con 0x5558661d6100 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a7bc0 con 0x5558661dbc60 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01a4e00 con 0x5558661d6af0 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90fa177740 10 client.0 _ll_register_callbacks cb 0x7f90e8011700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.340+0000 7f90f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36252/0 (socket says 172.21.15.59:36252) 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90f5400640 1 -- 172.21.15.59:0/1373022182 learned_addr learned my addr 172.21.15.59:0/1373022182 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90f5400640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x555866250570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90efe00640 1 -- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 msgr2=0x555866250570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90efe00640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x555866250570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90efe00640 1 -- 172.21.15.59:0/1373022182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 msgr2=0x5558661e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90efe00640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.341+0000 7f90efe00640 1 -- 172.21.15.59:0/1373022182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555866250ab0 con 0x5558661d6af0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90f5400640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90efe00640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0049b0 tx=0x7f90dc004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f895598c4bb97a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90d6000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc004250 con 0x5558661d6af0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f90dc0043f0 con 0x5558661d6af0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.342+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8015070 con 0x5558661d6af0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.343+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f90dc066050 con 0x5558661d6af0 2024-04-02T17:34:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.343+0000 7f90fa177740 10 client.27589 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.343+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555866251de0 con 0x5558661d6af0 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.343+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e8015070 con 0x5558661d6af0 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f90dc005350 con 0x5558661d6af0 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 10 client.27589 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 10 client.27589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 20 client.27589 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555866252790 0x555866254bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 4 client.27589 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.344+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558661492b0 con 0x555866252790 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.345+0000 7f90f5400640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555866252790 0x555866254bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.346+0000 7f90d6000640 10 client.27589 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.346+0000 7f90f5400640 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555866252790 0x555866254bb0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a1dc1abe6d84cd server_cookie=9b3cd11b7e97ef3a in_seq=0 out_seq=0 2024-04-02T17:34:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.638+0000 7f90d6000640 1 -- 172.21.15.59:0/1373022182 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558661492b0 con 0x555866252790 2024-04-02T17:34:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.638+0000 7f90d6000640 10 client.27589 handle_command_reply: tid=0 2024-04-02T17:34:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.638+0000 7f90fa177740 1 client.27589 shutdown 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 msgr2=0x5558661e50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0049b0 tx=0x7f90dc004c10 comp rx=0 tx=0).stop 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 shutdown_connections 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555866252790 0x555866254bb0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5558661dbc60 0x555866250570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5558661d6af0 0x5558661e50f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 --2- 172.21.15.59:0/1373022182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558661d6100 0x5558661e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 >> 172.21.15.59:0/1373022182 conn(0x5558661d1670 msgr2=0x5558661d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90d6000640 0 client.27589 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 shutdown_connections 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 1 -- 172.21.15.59:0/1373022182 wait complete. 2024-04-02T17:34:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.639+0000 7f90fa177740 20 client.27589 trim_cache size 0 max 16384 2024-04-02T17:34:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:34:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:34:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:34:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:34:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.640+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 msgr2=0x7f90d80a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.640+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 0x7f90d80a6120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f90e40ce040 tx=0x7f90e400de80 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.640+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 msgr2=0x7f90f0195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.640+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f90e400db80 tx=0x7f90e400b630 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 shutdown_connections 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f90d80a3c60 0x7f90d80a6120 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f01054e0 0x7f90f019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90f0102b30 0x7f90f0195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 --2- 172.21.15.59:0/810331117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f01001c0 0x7f90f0195970 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 >> 172.21.15.59:0/810331117 conn(0x7f90f00fbde0 msgr2=0x7f90f00fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 shutdown_connections 2024-04-02T17:34:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:11.641+0000 7f90f5e00640 1 -- 172.21.15.59:0/810331117 wait complete. 2024-04-02T17:34:11.643 INFO:tasks.workunit.client.0.smithi059.stdout: 10 340753 38.58 MB/sec execute 338 sec latency 186.596 ms 2024-04-02T17:34:12.124 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:11 smithi164 ceph-mon[36499]: pgmap v1023: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-04-02T17:34:12.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:11 smithi154 ceph-mon[36479]: pgmap v1023: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-04-02T17:34:12.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:11 smithi059 ceph-mon[31664]: pgmap v1023: 97 pgs: 97 active+clean; 1.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-04-02T17:34:12.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 341402 38.58 MB/sec execute 339 sec latency 194.627 ms 2024-04-02T17:34:13.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 342298 38.61 MB/sec execute 340 sec latency 47.162 ms 2024-04-02T17:34:14.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:13 smithi164 ceph-mon[36499]: pgmap v1024: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.1 MiB/s wr, 679 op/s 2024-04-02T17:34:14.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:13 smithi154 ceph-mon[36479]: pgmap v1024: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.1 MiB/s wr, 679 op/s 2024-04-02T17:34:14.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:13 smithi059 ceph-mon[31664]: pgmap v1024: 97 pgs: 97 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.1 MiB/s wr, 679 op/s 2024-04-02T17:34:14.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 342987 38.63 MB/sec execute 341 sec latency 80.714 ms 2024-04-02T17:34:15.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 343596 38.60 MB/sec execute 342 sec latency 111.804 ms 2024-04-02T17:34:16.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:15 smithi154 ceph-mon[36479]: pgmap v1025: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 671 op/s 2024-04-02T17:34:16.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:15 smithi059 ceph-mon[31664]: pgmap v1025: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 671 op/s 2024-04-02T17:34:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:15 smithi164 ceph-mon[36499]: pgmap v1025: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 671 op/s 2024-04-02T17:34:16.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 344235 38.58 MB/sec execute 343 sec latency 181.401 ms 2024-04-02T17:34:17.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 344924 38.57 MB/sec execute 344 sec latency 426.643 ms 2024-04-02T17:34:18.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:17 smithi154 ceph-mon[36479]: pgmap v1026: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 564 op/s 2024-04-02T17:34:18.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:17 smithi059 ceph-mon[31664]: pgmap v1026: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 564 op/s 2024-04-02T17:34:18.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:17 smithi164 ceph-mon[36499]: pgmap v1026: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 564 op/s 2024-04-02T17:34:18.644 INFO:tasks.workunit.client.0.smithi059.stdout: 10 345797 38.59 MB/sec execute 345 sec latency 46.188 ms 2024-04-02T17:34:19.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 346485 38.60 MB/sec execute 346 sec latency 92.462 ms 2024-04-02T17:34:20.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:19 smithi154 ceph-mon[36479]: pgmap v1027: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-02T17:34:20.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:19 smithi059 ceph-mon[31664]: pgmap v1027: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-02T17:34:20.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:19 smithi164 ceph-mon[36499]: pgmap v1027: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-02T17:34:20.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 347301 38.63 MB/sec execute 347 sec latency 61.182 ms 2024-04-02T17:34:21.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 347366 38.53 MB/sec execute 348 sec latency 878.425 ms 2024-04-02T17:34:22.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:21 smithi154 ceph-mon[36479]: pgmap v1028: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-04-02T17:34:22.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:21 smithi059 ceph-mon[31664]: pgmap v1028: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-04-02T17:34:22.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:21 smithi164 ceph-mon[36499]: pgmap v1028: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-04-02T17:34:22.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 347654 38.45 MB/sec execute 349 sec latency 1270.339 ms 2024-04-02T17:34:23.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 348017 38.40 MB/sec execute 350 sec latency 105.879 ms 2024-04-02T17:34:24.126 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:23 smithi059 ceph-mon[31664]: pgmap v1029: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-04-02T17:34:24.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:23 smithi154 ceph-mon[36479]: pgmap v1029: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-04-02T17:34:24.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:23 smithi164 ceph-mon[36499]: pgmap v1029: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-04-02T17:34:24.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 348611 38.38 MB/sec execute 351 sec latency 94.230 ms 2024-04-02T17:34:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.840+0000 7fa33ca00640 1 -- 172.21.15.59:0/840286619 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32802cc80 con 0x7fa340077700 2024-04-02T17:34:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 -- 172.21.15.59:0/840286619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa340077700 msgr2=0x7fa340077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 --2- 172.21.15.59:0/840286619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa340077700 0x7fa340077b60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa328009900 tx=0x7fa32802bc30 comp rx=0 tx=0).stop 2024-04-02T17:34:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 -- 172.21.15.59:0/840286619 shutdown_connections 2024-04-02T17:34:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 --2- 172.21.15.59:0/840286619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400780a0 0x7fa340078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 --2- 172.21.15.59:0/840286619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa340077700 0x7fa340077b60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 --2- 172.21.15.59:0/840286619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa3400768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 -- 172.21.15.59:0/840286619 >> 172.21.15.59:0/840286619 conn(0x7fa3400ff110 msgr2=0x7fa340101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 -- 172.21.15.59:0/840286619 shutdown_connections 2024-04-02T17:34:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.841+0000 7fa33f200640 1 -- 172.21.15.59:0/840286619 wait complete. 2024-04-02T17:34:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.843+0000 7fa33f200640 1 Processor -- start 2024-04-02T17:34:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.843+0000 7fa33f200640 1 -- start start 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.843+0000 7fa33f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa34019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.843+0000 7fa33f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340077700 0x7fa34019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 0x7fa3401a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa340113c00 con 0x7fa3400764c0 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa340113a80 con 0x7fa340077700 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa340113d80 con 0x7fa3400780a0 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340077700 0x7fa34019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa34019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa34019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34652/0 (socket says 172.21.15.59:34652) 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33de00640 1 -- 172.21.15.59:0/3667136544 learned_addr learned my addr 172.21.15.59:0/3667136544 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.844+0000 7fa33e800640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 0x7fa3401a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 -- 172.21.15.59:0/3667136544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340077700 msgr2=0x7fa34019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340077700 0x7fa34019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 -- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 msgr2=0x7fa34019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa34019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 -- 172.21.15.59:0/3667136544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3401a5f10 con 0x7fa3400780a0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33e800640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 0x7fa3401a58a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa33000d680 tx=0x7fa33000db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa336a00640 1 -- 172.21.15.59:0/3667136544 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330017070 con 0x7fa3400780a0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa336a00640 1 -- 172.21.15.59:0/3667136544 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330009820 con 0x7fa3400780a0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.845+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3401a6200 con 0x7fa3400780a0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.846+0000 7fa336a00640 1 -- 172.21.15.59:0/3667136544 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330004e00 con 0x7fa3400780a0 2024-04-02T17:34:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.846+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3401a66b0 con 0x7fa3400780a0 2024-04-02T17:34:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.850+0000 7fa347daa740 1 Processor -- start 2024-04-02T17:34:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.851+0000 7fa336a00640 1 -- 172.21.15.59:0/3667136544 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa330010460 con 0x7fa3400780a0 2024-04-02T17:34:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.852+0000 7fa347daa740 1 -- start start 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee37e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee3860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fee2e3d70 con 0x561fee37eaf0 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561fee28c2a0 con 0x561fee37e100 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa347daa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561fee2dfd20 con 0x561fee383c60 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa33de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa33de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34654/0 (socket says 172.21.15.59:34654) 2024-04-02T17:34:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.853+0000 7fa33de00640 1 -- 172.21.15.59:0/4084120237 learned_addr learned my addr 172.21.15.59:0/4084120237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.854+0000 7fa33de00640 1 -- 172.21.15.59:0/4084120237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 msgr2=0x561fee3860f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.854+0000 7fa33de00640 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee3860f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.854+0000 7fa33de00640 1 -- 172.21.15.59:0/4084120237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 msgr2=0x561fee37e520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.854+0000 7fa33de00640 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee37e520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.854+0000 7fa33de00640 1 -- 172.21.15.59:0/4084120237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fee2a0840 con 0x561fee37eaf0 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.855+0000 7fa336a00640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 0x7fa3240a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.855+0000 7fa33de00640 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa3380115f0 tx=0x7fa338011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32020b3f005933ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.855+0000 7fa336a00640 1 -- 172.21.15.59:0/3667136544 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa3300cfb70 con 0x7fa3400780a0 2024-04-02T17:34:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.856+0000 7fa33d400640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 0x7fa3240a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.856+0000 7fa32f400640 1 -- 172.21.15.59:0/4084120237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800f8f0 con 0x561fee37eaf0 2024-04-02T17:34:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.856+0000 7fa32f400640 1 -- 172.21.15.59:0/4084120237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338019420 con 0x561fee37eaf0 2024-04-02T17:34:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.856+0000 7fa32f400640 1 -- 172.21.15.59:0/4084120237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338018630 con 0x561fee37eaf0 2024-04-02T17:34:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.858+0000 7fa33d400640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 0x7fa3240a60e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa32802c7b0 tx=0x7fa328005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.858+0000 7fa347daa740 1 -- 172.21.15.59:0/4084120237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 msgr2=0x561fee3835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.858+0000 7fa347daa740 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa3380115f0 tx=0x7fa338011ab0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 -- 172.21.15.59:0/4084120237 shutdown_connections 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee3835f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee3860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 --2- 172.21.15.59:0/4084120237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee37e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 -- 172.21.15.59:0/4084120237 >> 172.21.15.59:0/4084120237 conn(0x561fee379670 msgr2=0x561fee379a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 -- 172.21.15.59:0/4084120237 shutdown_connections 2024-04-02T17:34:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.859+0000 7fa347daa740 1 -- 172.21.15.59:0/4084120237 wait complete. 2024-04-02T17:34:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.860+0000 7fa347daa740 1 -- start start 2024-04-02T17:34:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.860+0000 7fa347daa740 1 Processor -- start 2024-04-02T17:34:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37eaf0 0x561fee388bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fee386c10 con 0x561fee37e100 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561fee29f660 con 0x561fee383c60 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561fee386d90 con 0x561fee37eaf0 2024-04-02T17:34:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa347daa740 10 client.0 _ll_register_callbacks cb 0x561fee29f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa33e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa33e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:38454/0 (socket says 172.21.15.59:38454) 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa33e800640 1 -- 172.21.15.59:0/488682736 learned_addr learned my addr 172.21.15.59:0/488682736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.862+0000 7fa33d400640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37eaf0 0x561fee388bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33e800640 1 -- 172.21.15.59:0/488682736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37eaf0 msgr2=0x561fee388bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33de00640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33e800640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37eaf0 0x561fee388bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33e800640 1 -- 172.21.15.59:0/488682736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 msgr2=0x561fee388670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33e800640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.863+0000 7fa33e800640 1 -- 172.21.15.59:0/488682736 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fee2aee80 con 0x561fee383c60 2024-04-02T17:34:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.864+0000 7fa33de00640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:34:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.864+0000 7fa33e800640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa33000d650 tx=0x7fa330004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4724713238681c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.864+0000 7fa323e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:34:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.865+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330020070 con 0x561fee383c60 2024-04-02T17:34:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.865+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa338019420 con 0x561fee383c60 2024-04-02T17:34:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa330098510 con 0x561fee383c60 2024-04-02T17:34:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa3300b5200 con 0x561fee383c60 2024-04-02T17:34:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa347daa740 10 client.27539 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561fedff8210 con 0x561fee383c60 2024-04-02T17:34:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338019420 con 0x561fee383c60 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.866+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fa33009b360 con 0x561fee383c60 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 10 client.27539 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 10 client.27539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 20 client.27539 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3944e0 0x561fee3968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 4 client.27539 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fee293cd0 con 0x561fee3944e0 2024-04-02T17:34:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.867+0000 7fa33de00640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3944e0 0x561fee3968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.868+0000 7fa33de00640 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3944e0 0x561fee3968d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf8f8f1475cf046 server_cookie=5f3037623048d236 in_seq=0 out_seq=0 2024-04-02T17:34:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.868+0000 7fa323e00640 10 client.27539 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.869+0000 7fa323e00640 1 -- 172.21.15.59:0/488682736 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fee293cd0 con 0x561fee3944e0 2024-04-02T17:34:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.869+0000 7fa323e00640 10 client.27539 handle_command_reply: tid=0 2024-04-02T17:34:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.869+0000 7fa347daa740 1 client.27539 shutdown 2024-04-02T17:34:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 msgr2=0x561fee38fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa33000d650 tx=0x7fa330004800 comp rx=0 tx=0).stop 2024-04-02T17:34:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 shutdown_connections 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3944e0 0x561fee3968d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee383c60 0x561fee38fbf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37eaf0 0x561fee388bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 --2- 172.21.15.59:0/488682736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 >> 172.21.15.59:0/488682736 conn(0x561fee379670 msgr2=0x561fee378870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa323e00640 0 client.27539 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 shutdown_connections 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.870+0000 7fa347daa740 1 -- 172.21.15.59:0/488682736 wait complete. 2024-04-02T17:34:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.871+0000 7fa347daa740 20 client.27539 trim_cache size 0 max 16384 2024-04-02T17:34:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 Processor -- start 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 -- start start 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee38fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401af750 con 0x561fee37eaf0 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401b1290 con 0x561fee37e100 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.892+0000 7fa347daa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401afe20 con 0x561fee383c60 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:37466/0 (socket says 172.21.15.59:37466) 2024-04-02T17:34:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 -- 172.21.15.59:0/3597506963 learned_addr learned my addr 172.21.15.59:0/3597506963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33de00640 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 -- 172.21.15.59:0/3597506963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 msgr2=0x561fee388670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 -- 172.21.15.59:0/3597506963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 msgr2=0x561fee38fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee38fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.893+0000 7fa33e800640 1 -- 172.21.15.59:0/3597506963 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fee0c4620 con 0x561fee383c60 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.894+0000 7fa33e800640 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa330098870 tx=0x7fa330000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223f0235245868bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.894+0000 7fa32ea00640 1 -- 172.21.15.59:0/3597506963 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330020070 con 0x561fee383c60 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.894+0000 7fa32ea00640 1 -- 172.21.15.59:0/3597506963 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330098510 con 0x561fee383c60 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.894+0000 7fa32ea00640 1 -- 172.21.15.59:0/3597506963 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330097bf0 con 0x561fee383c60 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- 172.21.15.59:0/3597506963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 msgr2=0x561fee388bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa330098870 tx=0x7fa330000bc0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- 172.21.15.59:0/3597506963 shutdown_connections 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee37eaf0 0x561fee38fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee383c60 0x561fee388bb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 --2- 172.21.15.59:0/3597506963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37e100 0x561fee388670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- 172.21.15.59:0/3597506963 >> 172.21.15.59:0/3597506963 conn(0x561fee379670 msgr2=0x561fee379a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- 172.21.15.59:0/3597506963 shutdown_connections 2024-04-02T17:34:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- 172.21.15.59:0/3597506963 wait complete. 2024-04-02T17:34:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.895+0000 7fa347daa740 1 -- start start 2024-04-02T17:34:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 Processor -- start 2024-04-02T17:34:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37eaf0 0x561fee38cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee383c60 0x561fee3f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401b0bc0 con 0x561fee383c60 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401b04f0 con 0x561fee37eaf0 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401ad730 con 0x561fee37e100 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa347daa740 10 client.0 _ll_register_callbacks cb 0x7fa33009b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa33de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa33de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:37482/0 (socket says 172.21.15.59:37482) 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa33de00640 1 -- 172.21.15.59:0/3885284579 learned_addr learned my addr 172.21.15.59:0/3885284579 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa33d400640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37eaf0 0x561fee38cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.896+0000 7fa33e800640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee383c60 0x561fee3f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 -- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37eaf0 msgr2=0x561fee38cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37eaf0 0x561fee38cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 -- 172.21.15.59:0/3885284579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee383c60 msgr2=0x561fee3f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee383c60 0x561fee3f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 -- 172.21.15.59:0/3885284579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fee38fe00 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa33de00640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa338014040 tx=0x7fa338009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb91d7cdd9c3a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.897+0000 7fa323e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338019d00 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa330098510 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa338003970 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa33803b050 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa347daa740 10 client.27604 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561fee3f9fe0 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330098510 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fa33800f750 con 0x561fee37e100 2024-04-02T17:34:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.898+0000 7fa347daa740 10 client.27604 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa347daa740 10 client.27604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa347daa740 20 client.27604 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3fa7f0 0x561fee3fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa347daa740 4 client.27604 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fee2f12b0 con 0x561fee3fa7f0 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa33d400640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3fa7f0 0x561fee3fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa33d400640 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3fa7f0 0x561fee3fcc10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5b4fdc36465ec9 server_cookie=e5dabb2edf06a4d9 in_seq=0 out_seq=0 2024-04-02T17:34:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:24.899+0000 7fa323e00640 10 client.27604 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa323e00640 1 -- 172.21.15.59:0/3885284579 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fee2f12b0 con 0x561fee3fa7f0 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa323e00640 10 client.27604 handle_command_reply: tid=0 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 client.27604 shutdown 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 msgr2=0x561fee38c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa338014040 tx=0x7fa338009ca0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 shutdown_connections 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x561fee3fa7f0 0x561fee3fcc10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fee383c60 0x561fee3f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561fee37eaf0 0x561fee38cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa323e00640 0 client.27604 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 --2- 172.21.15.59:0/3885284579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fee37e100 0x561fee38c7f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.106+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 >> 172.21.15.59:0/3885284579 conn(0x561fee379670 msgr2=0x561fee380080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.107+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 shutdown_connections 2024-04-02T17:34:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.107+0000 7fa347daa740 1 -- 172.21.15.59:0/3885284579 wait complete. 2024-04-02T17:34:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.107+0000 7fa347daa740 20 client.27604 trim_cache size 0 max 16384 2024-04-02T17:34:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:34:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:34:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:34:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:34:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 msgr2=0x7fa3240a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 0x7fa3240a60e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa32802c7b0 tx=0x7fa328005fd0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 msgr2=0x7fa3401a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 0x7fa3401a58a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa33000d680 tx=0x7fa33000db40 comp rx=0 tx=0).stop 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 shutdown_connections 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400780a0 0x7fa3401a58a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340077700 0x7fa34019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa3240a3c20 0x7fa3240a60e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 --2- 172.21.15.59:0/3667136544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3400764c0 0x7fa34019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 >> 172.21.15.59:0/3667136544 conn(0x7fa3400ff110 msgr2=0x7fa340100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 shutdown_connections 2024-04-02T17:34:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:25.108+0000 7fa33f200640 1 -- 172.21.15.59:0/3667136544 wait complete. 2024-04-02T17:34:25.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 349090 38.35 MB/sec execute 352 sec latency 170.662 ms 2024-04-02T17:34:26.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:25 smithi154 ceph-mon[36479]: pgmap v1030: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-04-02T17:34:26.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:25 smithi059 ceph-mon[31664]: pgmap v1030: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-04-02T17:34:26.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:25 smithi164 ceph-mon[36499]: pgmap v1030: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-04-02T17:34:26.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 349898 38.34 MB/sec execute 353 sec latency 65.067 ms 2024-04-02T17:34:27.645 INFO:tasks.workunit.client.0.smithi059.stdout: 10 350563 38.34 MB/sec execute 354 sec latency 231.103 ms 2024-04-02T17:34:28.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:27 smithi154 ceph-mon[36479]: pgmap v1031: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 705 op/s 2024-04-02T17:34:28.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:27 smithi059 ceph-mon[31664]: pgmap v1031: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 705 op/s 2024-04-02T17:34:28.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:27 smithi164 ceph-mon[36499]: pgmap v1031: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 705 op/s 2024-04-02T17:34:28.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 351196 38.34 MB/sec execute 355 sec latency 198.944 ms 2024-04-02T17:34:29.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 351938 38.35 MB/sec execute 356 sec latency 57.257 ms 2024-04-02T17:34:30.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:29 smithi154 ceph-mon[36479]: pgmap v1032: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-02T17:34:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:29 smithi059 ceph-mon[31664]: pgmap v1032: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-02T17:34:30.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:29 smithi164 ceph-mon[36499]: pgmap v1032: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-02T17:34:30.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 352554 38.33 MB/sec execute 357 sec latency 120.293 ms 2024-04-02T17:34:31.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 353216 38.31 MB/sec execute 358 sec latency 69.130 ms 2024-04-02T17:34:32.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:31 smithi154 ceph-mon[36479]: pgmap v1033: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-04-02T17:34:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:31 smithi059 ceph-mon[31664]: pgmap v1033: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-04-02T17:34:32.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:31 smithi164 ceph-mon[36499]: pgmap v1033: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-04-02T17:34:32.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 354031 38.34 MB/sec execute 359 sec latency 50.921 ms 2024-04-02T17:34:33.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 354876 38.35 MB/sec execute 360 sec latency 55.453 ms 2024-04-02T17:34:34.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:33 smithi154 ceph-mon[36479]: pgmap v1034: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 877 op/s 2024-04-02T17:34:34.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:33 smithi059 ceph-mon[31664]: pgmap v1034: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 877 op/s 2024-04-02T17:34:34.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:33 smithi164 ceph-mon[36499]: pgmap v1034: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 877 op/s 2024-04-02T17:34:34.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 355618 38.37 MB/sec execute 361 sec latency 41.600 ms 2024-04-02T17:34:35.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 356258 38.36 MB/sec execute 362 sec latency 63.778 ms 2024-04-02T17:34:36.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:35 smithi059 ceph-mon[31664]: pgmap v1035: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 944 op/s 2024-04-02T17:34:36.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:35 smithi164 ceph-mon[36499]: pgmap v1035: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 944 op/s 2024-04-02T17:34:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:35 smithi154 ceph-mon[36479]: pgmap v1035: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 944 op/s 2024-04-02T17:34:36.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 356964 38.33 MB/sec execute 363 sec latency 69.089 ms 2024-04-02T17:34:37.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 357745 38.36 MB/sec execute 364 sec latency 58.124 ms 2024-04-02T17:34:38.201 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:37 smithi154 ceph-mon[36479]: pgmap v1036: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-04-02T17:34:38.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:37 smithi059 ceph-mon[31664]: pgmap v1036: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-04-02T17:34:38.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:37 smithi164 ceph-mon[36499]: pgmap v1036: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.327+0000 7f9fda600640 1 -- 172.21.15.59:0/2783157491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4102b60 msgr2=0x7f9fd4102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.327+0000 7f9fda600640 1 --2- 172.21.15.59:0/2783157491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4102b60 0x7f9fd4102fe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009900 tx=0x7f9fc002bc50 comp rx=0 tx=0).stop 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 -- 172.21.15.59:0/2783157491 shutdown_connections 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 --2- 172.21.15.59:0/2783157491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4103520 0x7f9fd4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 --2- 172.21.15.59:0/2783157491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4102b60 0x7f9fd4102fe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 --2- 172.21.15.59:0/2783157491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 -- 172.21.15.59:0/2783157491 >> 172.21.15.59:0/2783157491 conn(0x7f9fd40fcee0 msgr2=0x7f9fd40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 -- 172.21.15.59:0/2783157491 shutdown_connections 2024-04-02T17:34:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.328+0000 7f9fda600640 1 -- 172.21.15.59:0/2783157491 wait complete. 2024-04-02T17:34:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.330+0000 7f9fda600640 1 Processor -- start 2024-04-02T17:34:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.330+0000 7f9fda600640 1 -- start start 2024-04-02T17:34:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 0x7f9fd419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd410d620 con 0x7f9fd4101970 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd410d4a0 con 0x7f9fd4102b60 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.331+0000 7f9fda600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd410d7a0 con 0x7f9fd4103520 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 0x7f9fd419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43334/0 (socket says 172.21.15.59:43334) 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 -- 172.21.15.59:0/3568543835 learned_addr learned my addr 172.21.15.59:0/3568543835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 0x7f9fd419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41314/0 (socket says 172.21.15.59:41314) 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd9200640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 -- 172.21.15.59:0/3568543835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 msgr2=0x7f9fd419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 0x7f9fd419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 -- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 msgr2=0x7f9fd4197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd3e00640 1 -- 172.21.15.59:0/3568543835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd4073e30 con 0x7f9fd4102b60 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.332+0000 7f9fd9200640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc00052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fd1600640 1 -- 172.21.15.59:0/3568543835 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0041020 con 0x7f9fd4102b60 2024-04-02T17:34:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fd1600640 1 -- 172.21.15.59:0/3568543835 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc002cad0 con 0x7f9fd4102b60 2024-04-02T17:34:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd40740c0 con 0x7f9fd4102b60 2024-04-02T17:34:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd4074600 con 0x7f9fd4102b60 2024-04-02T17:34:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.334+0000 7f9fd1600640 1 -- 172.21.15.59:0/3568543835 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0034c40 con 0x7f9fd4102b60 2024-04-02T17:34:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.342+0000 7f9fd1600640 1 -- 172.21.15.59:0/3568543835 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9fc0044020 con 0x7f9fd4102b60 2024-04-02T17:34:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.343+0000 7f9fde84f740 1 Processor -- start 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.343+0000 7f9fde84f740 1 -- start start 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737d100 0x55bef737d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 0x55bef73850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fde84f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bef72e2d70 con 0x55bef7382c60 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fde84f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bef728b2a0 con 0x55bef737d100 2024-04-02T17:34:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.346+0000 7f9fde84f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef72ded20 con 0x55bef737daf0 2024-04-02T17:34:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.344+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.346+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:41318/0 (socket says 172.21.15.59:41318) 2024-04-02T17:34:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.346+0000 7f9fd9200640 1 -- 172.21.15.59:0/2969189208 learned_addr learned my addr 172.21.15.59:0/2969189208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.345+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737d100 0x55bef737d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.345+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 0x55bef73850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd9200640 1 -- 172.21.15.59:0/2969189208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737d100 msgr2=0x55bef737d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737d100 0x55bef737d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd9200640 1 -- 172.21.15.59:0/2969189208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 msgr2=0x55bef73850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 0x55bef73850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd9200640 1 -- 172.21.15.59:0/2969189208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef729f840 con 0x55bef737daf0 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.347+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 0x55bef73850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.348+0000 7f9fd1600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 0x7f9fbc0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.348+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800aed0 tx=0x7f9fc800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4c7facf374a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.348+0000 7f9fd1600640 1 -- 172.21.15.59:0/3568543835 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9fc00f1d90 con 0x7f9fd4102b60 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.349+0000 7f9fc6000640 1 -- 172.21.15.59:0/2969189208 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004d20 con 0x55bef737daf0 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.349+0000 7f9fc6000640 1 -- 172.21.15.59:0/2969189208 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8011b30 con 0x55bef737daf0 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.349+0000 7f9fc6000640 1 -- 172.21.15.59:0/2969189208 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8005760 con 0x55bef737daf0 2024-04-02T17:34:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.350+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 0x7f9fbc0a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.351+0000 7f9fde84f740 1 -- 172.21.15.59:0/2969189208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 msgr2=0x55bef73825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.351+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800aed0 tx=0x7f9fc800f730 comp rx=0 tx=0).stop 2024-04-02T17:34:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.351+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 0x7f9fbc0a6130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55bef737e350 tx=0x7f9fcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 -- 172.21.15.59:0/2969189208 shutdown_connections 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef7382c60 0x55bef73850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef73825f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2969189208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737d100 0x55bef737d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 -- 172.21.15.59:0/2969189208 >> 172.21.15.59:0/2969189208 conn(0x55bef7378670 msgr2=0x55bef7378a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 -- 172.21.15.59:0/2969189208 shutdown_connections 2024-04-02T17:34:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.352+0000 7f9fde84f740 1 -- 172.21.15.59:0/2969189208 wait complete. 2024-04-02T17:34:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.353+0000 7f9fde84f740 1 -- start start 2024-04-02T17:34:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.353+0000 7f9fde84f740 1 Processor -- start 2024-04-02T17:34:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef73986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef739fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bef7396c20 con 0x55bef737d100 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bef729e660 con 0x55bef737daf0 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef7396da0 con 0x55bef7382c60 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fde84f740 10 client.0 _ll_register_callbacks cb 0x55bef729e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43352/0 (socket says 172.21.15.59:43352) 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.355+0000 7f9fd3e00640 1 -- 172.21.15.59:0/2413799435 learned_addr learned my addr 172.21.15.59:0/2413799435 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef73986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 -- 172.21.15.59:0/2413799435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 msgr2=0x55bef739fc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:34:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef739fc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 -- 172.21.15.59:0/2413799435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 msgr2=0x55bef73986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef73986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 -- 172.21.15.59:0/2413799435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef72ade80 con 0x55bef737daf0 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef73986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.356+0000 7f9fd3e00640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00c6750 tx=0x7f9fc00bb3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5234f2b0dbb3e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.357+0000 7f9fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:34:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.357+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00bdea0 con 0x55bef737daf0 2024-04-02T17:34:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.358+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9fc002c5f0 con 0x55bef737daf0 2024-04-02T17:34:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.359+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc8011b30 con 0x55bef737daf0 2024-04-02T17:34:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.359+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fc00be750 con 0x55bef737daf0 2024-04-02T17:34:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.359+0000 7f9fde84f740 10 client.27557 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.359+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bef6ff7210 con 0x55bef737daf0 2024-04-02T17:34:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.360+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8011b30 con 0x55bef737daf0 2024-04-02T17:34:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.360+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f9fc00d2bd0 con 0x55bef737daf0 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 10 client.27557 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 10 client.27557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 20 client.27557 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef73a4560 0x55bef73a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 4 client.27557 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:34:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.361+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef7292cd0 con 0x55bef73a4560 2024-04-02T17:34:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.362+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef73a4560 0x55bef73a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.363+0000 7f9fd9200640 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef73a4560 0x55bef73a69e0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ba319c3b372bb1 server_cookie=4c098094d78cbeaa in_seq=0 out_seq=0 2024-04-02T17:34:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.363+0000 7f9fbaa00640 10 client.27557 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.368+0000 7f9fbaa00640 1 -- 172.21.15.59:0/2413799435 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bef7292cd0 con 0x55bef73a4560 2024-04-02T17:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.368+0000 7f9fbaa00640 10 client.27557 handle_command_reply: tid=0 2024-04-02T17:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.368+0000 7f9fde84f740 1 client.27557 shutdown 2024-04-02T17:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 msgr2=0x55bef7398c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00c6750 tx=0x7f9fc00bb3a0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 shutdown_connections 2024-04-02T17:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef73a4560 0x55bef73a69e0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef739fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef7398c20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fbaa00640 0 client.27557 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 --2- 172.21.15.59:0/2413799435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef73986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 >> 172.21.15.59:0/2413799435 conn(0x55bef7378670 msgr2=0x55bef7377870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 shutdown_connections 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 1 -- 172.21.15.59:0/2413799435 wait complete. 2024-04-02T17:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.369+0000 7f9fde84f740 20 client.27557 trim_cache size 0 max 16384 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 Processor -- start 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 -- start start 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef739fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef737f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400e5b0 con 0x55bef737d100 2024-04-02T17:34:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb40100f0 con 0x55bef737daf0 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fde84f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400ec80 con 0x55bef7382c60 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fd9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56664/0 (socket says 172.21.15.59:56664) 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.393+0000 7f9fd9200640 1 -- 172.21.15.59:0/614510013 learned_addr learned my addr 172.21.15.59:0/614510013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef737f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9200640 1 -- 172.21.15.59:0/614510013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 msgr2=0x55bef737f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9200640 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef737f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9200640 1 -- 172.21.15.59:0/614510013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 msgr2=0x55bef739fc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9200640 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef739fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.394+0000 7f9fd9200640 1 -- 172.21.15.59:0/614510013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef70c3620 con 0x55bef737d100 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.395+0000 7f9fd9200640 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000f80 tx=0x7f9fc800adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec917984d354368 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.395+0000 7f9fc5600640 1 -- 172.21.15.59:0/614510013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004c00 con 0x55bef737d100 2024-04-02T17:34:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.395+0000 7f9fc5600640 1 -- 172.21.15.59:0/614510013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc802f070 con 0x55bef737d100 2024-04-02T17:34:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.395+0000 7f9fc5600640 1 -- 172.21.15.59:0/614510013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8029880 con 0x55bef737d100 2024-04-02T17:34:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 -- 172.21.15.59:0/614510013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 msgr2=0x55bef737f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000f80 tx=0x7f9fc800adf0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 -- 172.21.15.59:0/614510013 shutdown_connections 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef7382c60 0x55bef737f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef737daf0 0x55bef739fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 --2- 172.21.15.59:0/614510013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef737f030 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.397+0000 7f9fde84f740 1 -- 172.21.15.59:0/614510013 >> 172.21.15.59:0/614510013 conn(0x55bef7378670 msgr2=0x55bef73971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.398+0000 7f9fde84f740 1 -- 172.21.15.59:0/614510013 shutdown_connections 2024-04-02T17:34:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.398+0000 7f9fde84f740 1 -- 172.21.15.59:0/614510013 wait complete. 2024-04-02T17:34:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.398+0000 7f9fde84f740 1 -- start start 2024-04-02T17:34:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.399+0000 7f9fde84f740 1 Processor -- start 2024-04-02T17:34:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef739ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef739d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400fa20 con 0x55bef737d100 2024-04-02T17:34:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400f350 con 0x55bef7382c60 2024-04-02T17:34:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb400c590 con 0x55bef737daf0 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fde84f740 10 client.0 _ll_register_callbacks cb 0x55bef729e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.400+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef739d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.401+0000 7f9fd9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:43374/0 (socket says 172.21.15.59:43374) 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.401+0000 7f9fd9c00640 1 -- 172.21.15.59:0/1737069109 learned_addr learned my addr 172.21.15.59:0/1737069109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.401+0000 7f9fd9c00640 1 -- 172.21.15.59:0/1737069109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 msgr2=0x55bef739d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.401+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef739d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.401+0000 7f9fd9c00640 1 -- 172.21.15.59:0/1737069109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 msgr2=0x55bef739ce70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:34:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.402+0000 7f9fd9200640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef739ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.402+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef739ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.402+0000 7f9fd9c00640 1 -- 172.21.15.59:0/1737069109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef739e960 con 0x55bef7382c60 2024-04-02T17:34:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.403+0000 7f9fd9c00640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc004580 tx=0x7f9fcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8116574f3f24f9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.403+0000 7f9fbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:34:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.403+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc02e060 con 0x55bef7382c60 2024-04-02T17:34:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.403+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9fcc01e680 con 0x55bef7382c60 2024-04-02T17:34:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.405+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc802f070 con 0x55bef7382c60 2024-04-02T17:34:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.406+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc009e50 con 0x55bef7382c60 2024-04-02T17:34:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.406+0000 7f9fde84f740 10 client.27560 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.406+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bef739fce0 con 0x55bef7382c60 2024-04-02T17:34:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc009e50 con 0x55bef7382c60 2024-04-02T17:34:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f9fcc029c20 con 0x55bef7382c60 2024-04-02T17:34:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 10 client.27560 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 10 client.27560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 20 client.27560 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef740b000 0x55bef740d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 4 client.27560 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.407+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bef72f02b0 con 0x55bef740b000 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.408+0000 7f9fd9200640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef740b000 0x55bef740d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.409+0000 7f9fd9200640 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef740b000 0x55bef740d420 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=630f935b1ae9d729 server_cookie=c928d29e09971a80 in_seq=0 out_seq=0 2024-04-02T17:34:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.409+0000 7f9fbaa00640 10 client.27560 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:38.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 358453 38.36 MB/sec execute 365 sec latency 179.676 ms 2024-04-02T17:34:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.659+0000 7f9fbaa00640 1 -- 172.21.15.59:0/1737069109 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bef72f02b0 con 0x55bef740b000 2024-04-02T17:34:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.659+0000 7f9fbaa00640 10 client.27560 handle_command_reply: tid=0 2024-04-02T17:34:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.659+0000 7f9fde84f740 1 client.27560 shutdown 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 msgr2=0x55bef739e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc004580 tx=0x7f9fcc0046c0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 shutdown_connections 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55bef740b000 0x55bef740d420 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bef7382c60 0x55bef739e370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef737daf0 0x55bef739d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 --2- 172.21.15.59:0/1737069109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bef737d100 0x55bef739ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 >> 172.21.15.59:0/1737069109 conn(0x55bef7378670 msgr2=0x55bef737f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fbaa00640 0 client.27560 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.660+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 shutdown_connections 2024-04-02T17:34:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.661+0000 7f9fde84f740 1 -- 172.21.15.59:0/1737069109 wait complete. 2024-04-02T17:34:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.661+0000 7f9fde84f740 20 client.27560 trim_cache size 0 max 16384 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 msgr2=0x7f9fbc0a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 0x7f9fbc0a6130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55bef737e350 tx=0x7f9fcc005f90 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 msgr2=0x7f9fd41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc00052f0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 shutdown_connections 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd4103520 0x7f9fd419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fd4102b60 0x7f9fd41981c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9fbc0a3c70 0x7f9fbc0a6130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 --2- 172.21.15.59:0/3568543835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fd4101970 0x7f9fd4197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.662+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 >> 172.21.15.59:0/3568543835 conn(0x7f9fd40fcee0 msgr2=0x7f9fd4105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.663+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 shutdown_connections 2024-04-02T17:34:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:38.663+0000 7f9fda600640 1 -- 172.21.15.59:0/3568543835 wait complete. 2024-04-02T17:34:39.646 INFO:tasks.workunit.client.0.smithi059.stdout: 10 359144 38.35 MB/sec execute 366 sec latency 116.717 ms 2024-04-02T17:34:40.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:39 smithi059 ceph-mon[31664]: pgmap v1037: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 743 op/s 2024-04-02T17:34:40.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:39 smithi164 ceph-mon[36499]: pgmap v1037: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 743 op/s 2024-04-02T17:34:40.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:39 smithi154 ceph-mon[36479]: pgmap v1037: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 743 op/s 2024-04-02T17:34:40.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 359751 38.35 MB/sec execute 367 sec latency 232.612 ms 2024-04-02T17:34:41.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 360477 38.33 MB/sec execute 368 sec latency 46.161 ms 2024-04-02T17:34:42.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:41 smithi059 ceph-mon[31664]: pgmap v1038: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 577 op/s 2024-04-02T17:34:42.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:41 smithi164 ceph-mon[36499]: pgmap v1038: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 577 op/s 2024-04-02T17:34:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:41 smithi154 ceph-mon[36479]: pgmap v1038: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.9 MiB/s wr, 577 op/s 2024-04-02T17:34:42.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 361274 38.36 MB/sec execute 369 sec latency 48.964 ms 2024-04-02T17:34:43.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 362015 38.36 MB/sec execute 370 sec latency 49.394 ms 2024-04-02T17:34:44.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:43 smithi059 ceph-mon[31664]: pgmap v1039: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-04-02T17:34:44.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:43 smithi164 ceph-mon[36499]: pgmap v1039: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-04-02T17:34:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:43 smithi154 ceph-mon[36479]: pgmap v1039: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-04-02T17:34:44.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 362746 38.36 MB/sec execute 371 sec latency 56.266 ms 2024-04-02T17:34:45.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 363461 38.36 MB/sec execute 372 sec latency 55.479 ms 2024-04-02T17:34:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:45 smithi059 ceph-mon[31664]: pgmap v1040: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-04-02T17:34:46.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:45 smithi164 ceph-mon[36499]: pgmap v1040: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-04-02T17:34:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:45 smithi154 ceph-mon[36479]: pgmap v1040: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-04-02T17:34:46.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 364231 38.36 MB/sec execute 373 sec latency 44.392 ms 2024-04-02T17:34:47.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 364951 38.40 MB/sec execute 374 sec latency 47.206 ms 2024-04-02T17:34:48.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:47 smithi059 ceph-mon[31664]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2024-04-02T17:34:48.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:47 smithi164 ceph-mon[36499]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2024-04-02T17:34:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:47 smithi154 ceph-mon[36479]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2024-04-02T17:34:48.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 365584 38.36 MB/sec execute 375 sec latency 94.663 ms 2024-04-02T17:34:49.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 366147 38.34 MB/sec execute 376 sec latency 129.408 ms 2024-04-02T17:34:50.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:49 smithi059 ceph-mon[31664]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 622 op/s 2024-04-02T17:34:50.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:49 smithi164 ceph-mon[36499]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 622 op/s 2024-04-02T17:34:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:49 smithi154 ceph-mon[36479]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 622 op/s 2024-04-02T17:34:50.647 INFO:tasks.workunit.client.0.smithi059.stdout: 10 366855 38.34 MB/sec execute 377 sec latency 114.810 ms 2024-04-02T17:34:51.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 367528 38.32 MB/sec execute 378 sec latency 61.364 ms 2024-04-02T17:34:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.872+0000 7fe175400640 1 -- 172.21.15.59:0/2639168033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 msgr2=0x7fe170108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.872+0000 7fe175400640 1 --2- 172.21.15.59:0/2639168033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe170108580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe168009ca0 tx=0x7fe16802c100 comp rx=0 tx=0).stop 2024-04-02T17:34:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.873+0000 7fe175400640 1 -- 172.21.15.59:0/2639168033 shutdown_connections 2024-04-02T17:34:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.873+0000 7fe175400640 1 --2- 172.21.15.59:0/2639168033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109d10 0x7fe1701105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.873+0000 7fe175400640 1 --2- 172.21.15.59:0/2639168033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109350 0x7fe1701097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.873+0000 7fe175400640 1 --2- 172.21.15.59:0/2639168033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe170108580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.873+0000 7fe175400640 1 -- 172.21.15.59:0/2639168033 >> 172.21.15.59:0/2639168033 conn(0x7fe170077110 msgr2=0x7fe170079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.874+0000 7fe175400640 1 -- 172.21.15.59:0/2639168033 shutdown_connections 2024-04-02T17:34:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.874+0000 7fe175400640 1 -- 172.21.15.59:0/2639168033 wait complete. 2024-04-02T17:34:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.875+0000 7fe175400640 1 Processor -- start 2024-04-02T17:34:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.876+0000 7fe175400640 1 -- start start 2024-04-02T17:34:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe17019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 0x7fe17019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe170113e10 con 0x7fe170109350 2024-04-02T17:34:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe170113c90 con 0x7fe170109d10 2024-04-02T17:34:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe175400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe170113f90 con 0x7fe170108160 2024-04-02T17:34:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe16f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe16f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33366/0 (socket says 172.21.15.59:33366) 2024-04-02T17:34:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.877+0000 7fe16f200640 1 -- 172.21.15.59:0/3511095217 learned_addr learned my addr 172.21.15.59:0/3511095217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16e800640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe17019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16de00640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 0x7fe17019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16f200640 1 -- 172.21.15.59:0/3511095217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 msgr2=0x7fe17019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16f200640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe17019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16f200640 1 -- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 msgr2=0x7fe17019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16f200640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 0x7fe17019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16f200640 1 -- 172.21.15.59:0/3511095217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a5fe0 con 0x7fe170109d10 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.878+0000 7fe16de00640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 0x7fe17019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.879+0000 7fe16f200640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00c970 tx=0x7fe15c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.879+0000 7fe167400640 1 -- 172.21.15.59:0/3511095217 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c007bb0 con 0x7fe170109d10 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.879+0000 7fe167400640 1 -- 172.21.15.59:0/3511095217 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c007d50 con 0x7fe170109d10 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.880+0000 7fe167400640 1 -- 172.21.15.59:0/3511095217 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c018800 con 0x7fe170109d10 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.880+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a62d0 con 0x7fe170109d10 2024-04-02T17:34:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.880+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a67d0 con 0x7fe170109d10 2024-04-02T17:34:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.889+0000 7fe17822e740 1 Processor -- start 2024-04-02T17:34:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.890+0000 7fe167400640 1 -- 172.21.15.59:0/3511095217 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe15c00fa50 con 0x7fe170109d10 2024-04-02T17:34:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 -- start start 2024-04-02T17:34:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf9022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf9073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf909ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555daf867d70 con 0x555daf907a30 2024-04-02T17:34:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555daf8102a0 con 0x555daf901ed0 2024-04-02T17:34:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.891+0000 7fe17822e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555daf863d20 con 0x555daf9028c0 2024-04-02T17:34:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.892+0000 7fe16de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf909ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf909ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57436/0 (socket says 172.21.15.59:57436) 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 -- 172.21.15.59:0/706169987 learned_addr learned my addr 172.21.15.59:0/706169987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe167400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 0x7fe1540a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf9022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16e800640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf9073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe167400640 1 -- 172.21.15.59:0/3511095217 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fe15c0d0360 con 0x7fe170109d10 2024-04-02T17:34:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 -- 172.21.15.59:0/706169987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 msgr2=0x555daf9073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf9073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 -- 172.21.15.59:0/706169987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 msgr2=0x555daf909ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf909ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.893+0000 7fe16de00640 1 -- 172.21.15.59:0/706169987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555daf824840 con 0x555daf901ed0 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.894+0000 7fe16f200640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 0x7fe1540a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.894+0000 7fe16e800640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf9073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.894+0000 7fe16de00640 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf9022f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe160013810 tx=0x7fe160004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad4c5ca02baeaef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.894+0000 7fe15b400640 1 -- 172.21.15.59:0/706169987 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160013110 con 0x555daf901ed0 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.895+0000 7fe15b400640 1 -- 172.21.15.59:0/706169987 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe160013e50 con 0x555daf901ed0 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.895+0000 7fe15b400640 1 -- 172.21.15.59:0/706169987 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16000f840 con 0x555daf901ed0 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.896+0000 7fe17822e740 1 -- 172.21.15.59:0/706169987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 msgr2=0x555daf9022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.896+0000 7fe17822e740 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf9022f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe160013810 tx=0x7fe160004730 comp rx=0 tx=0).stop 2024-04-02T17:34:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.896+0000 7fe16f200640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 0x7fe1540a62d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe15c09c5f0 tx=0x7fe15c00fd40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.897+0000 7fe17822e740 1 -- 172.21.15.59:0/706169987 shutdown_connections 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.897+0000 7fe17822e740 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf909ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.897+0000 7fe17822e740 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf9073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.897+0000 7fe17822e740 1 --2- 172.21.15.59:0/706169987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf9022f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.897+0000 7fe17822e740 1 -- 172.21.15.59:0/706169987 >> 172.21.15.59:0/706169987 conn(0x555daf8fd440 msgr2=0x555daf8fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.898+0000 7fe17822e740 1 -- 172.21.15.59:0/706169987 shutdown_connections 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.898+0000 7fe17822e740 1 -- 172.21.15.59:0/706169987 wait complete. 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.898+0000 7fe17822e740 1 -- start start 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.899+0000 7fe17822e740 1 Processor -- start 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.899+0000 7fe17822e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.899+0000 7fe17822e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.899+0000 7fe17822e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.899+0000 7fe17822e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555daf90a980 con 0x555daf907a30 2024-04-02T17:34:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe17822e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555daf823660 con 0x555daf901ed0 2024-04-02T17:34:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe17822e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555daf90ab00 con 0x555daf9028c0 2024-04-02T17:34:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe17822e740 10 client.0 _ll_register_callbacks cb 0x555daf823050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe16e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe16e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54740/0 (socket says 172.21.15.59:54740) 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe16e800640 1 -- 172.21.15.59:0/1709380027 learned_addr learned my addr 172.21.15.59:0/1709380027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.900+0000 7fe16de00640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.901+0000 7fe16e800640 1 -- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 msgr2=0x555daf90c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.901+0000 7fe16e800640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.901+0000 7fe16e800640 1 -- 172.21.15.59:0/1709380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 msgr2=0x555daf913960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.901+0000 7fe16e800640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.901+0000 7fe16e800640 1 -- 172.21.15.59:0/1709380027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555daf832e80 con 0x555daf9028c0 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.902+0000 7fe16e800640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe168032420 tx=0x7fe168009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc857f9ceee79ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.902+0000 7fe14fe00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:34:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.902+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168044070 con 0x555daf9028c0 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.902+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe160013e50 con 0x555daf9028c0 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.903+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fe168032890 con 0x555daf9028c0 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.903+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe168050050 con 0x555daf9028c0 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.903+0000 7fe17822e740 10 client.27631 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.903+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555daf57c210 con 0x555daf9028c0 2024-04-02T17:34:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.903+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe160013e50 con 0x555daf9028c0 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fe168004680 con 0x555daf9028c0 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 10 client.27631 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 10 client.27631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 20 client.27631 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf9182b0 0x555daf91a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 4 client.27631 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.904+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555daf817cd0 con 0x555daf9182b0 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.905+0000 7fe16de00640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf9182b0 0x555daf91a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.905+0000 7fe16de00640 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf9182b0 0x555daf91a6a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305b9cffdbd316e7 server_cookie=9dac4dadbf90eb93 in_seq=0 out_seq=0 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.905+0000 7fe14fe00640 10 client.27631 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.906+0000 7fe14fe00640 1 -- 172.21.15.59:0/1709380027 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555daf817cd0 con 0x555daf9182b0 2024-04-02T17:34:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.906+0000 7fe14fe00640 10 client.27631 handle_command_reply: tid=0 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.906+0000 7fe17822e740 1 client.27631 shutdown 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 msgr2=0x555daf90c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe168032420 tx=0x7fe168009960 comp rx=0 tx=0).stop 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 shutdown_connections 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf9182b0 0x555daf91a6a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 --2- 172.21.15.59:0/1709380027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 >> 172.21.15.59:0/1709380027 conn(0x555daf8fd440 msgr2=0x555daf8fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe14fe00640 0 client.27631 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 shutdown_connections 2024-04-02T17:34:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 1 -- 172.21.15.59:0/1709380027 wait complete. 2024-04-02T17:34:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.907+0000 7fe17822e740 20 client.27631 trim_cache size 0 max 16384 2024-04-02T17:34:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.935+0000 7fe17822e740 1 Processor -- start 2024-04-02T17:34:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.935+0000 7fe17822e740 1 -- start start 2024-04-02T17:34:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701af720 con 0x555daf907a30 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701b1260 con 0x555daf901ed0 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe17822e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701afdf0 con 0x555daf9028c0 2024-04-02T17:34:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe16de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.936+0000 7fe16e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54754/0 (socket says 172.21.15.59:54754) 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 -- 172.21.15.59:0/1167971159 learned_addr learned my addr 172.21.15.59:0/1167971159 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33374/0 (socket says 172.21.15.59:33374) 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 -- 172.21.15.59:0/1167971159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 msgr2=0x555daf90c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 -- 172.21.15.59:0/1167971159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 msgr2=0x555daf913920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16e800640 1 -- 172.21.15.59:0/1167971159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555daf648620 con 0x555daf9028c0 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.937+0000 7fe16de00640 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.939+0000 7fe16e800640 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe168033000 tx=0x7fe16802c100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b2fd454b5adb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.939+0000 7fe15aa00640 1 -- 172.21.15.59:0/1167971159 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168059070 con 0x555daf9028c0 2024-04-02T17:34:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.939+0000 7fe15aa00640 1 -- 172.21.15.59:0/1167971159 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168009540 con 0x555daf9028c0 2024-04-02T17:34:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.939+0000 7fe15aa00640 1 -- 172.21.15.59:0/1167971159 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168044050 con 0x555daf9028c0 2024-04-02T17:34:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.940+0000 7fe17822e740 1 -- 172.21.15.59:0/1167971159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 msgr2=0x555daf90c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.940+0000 7fe17822e740 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe168033000 tx=0x7fe16802c100 comp rx=0 tx=0).stop 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.940+0000 7fe17822e740 1 -- 172.21.15.59:0/1167971159 shutdown_connections 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf907a30 0x555daf913920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf9028c0 0x555daf90c920 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 --2- 172.21.15.59:0/1167971159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf90c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 -- 172.21.15.59:0/1167971159 >> 172.21.15.59:0/1167971159 conn(0x555daf8fd440 msgr2=0x555daf903240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 -- 172.21.15.59:0/1167971159 shutdown_connections 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 -- 172.21.15.59:0/1167971159 wait complete. 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.941+0000 7fe17822e740 1 -- start start 2024-04-02T17:34:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.942+0000 7fe17822e740 1 Processor -- start 2024-04-02T17:34:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf9028c0 0x555daf910dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf907a30 0x555daf97c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701b0b90 con 0x555daf9028c0 2024-04-02T17:34:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701b04c0 con 0x555daf901ed0 2024-04-02T17:34:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.943+0000 7fe17822e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701ad700 con 0x555daf907a30 2024-04-02T17:34:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.944+0000 7fe16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.944+0000 7fe16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:33380/0 (socket says 172.21.15.59:33380) 2024-04-02T17:34:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.944+0000 7fe16de00640 1 -- 172.21.15.59:0/1486026418 learned_addr learned my addr 172.21.15.59:0/1486026418 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:34:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.944+0000 7fe17822e740 10 client.0 _ll_register_callbacks cb 0x7fe168009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf907a30 0x555daf97c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 -- 172.21.15.59:0/1486026418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf907a30 msgr2=0x555daf97c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf907a30 0x555daf97c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 -- 172.21.15.59:0/1486026418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf9028c0 msgr2=0x555daf910dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf9028c0 0x555daf910dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 -- 172.21.15.59:0/1486026418 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555daf97cdc0 con 0x555daf901ed0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.945+0000 7fe16de00640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe16001d040 tx=0x7fe1600099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52807149931ab6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.946+0000 7fe14fe00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.946+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160034020 con 0x555daf901ed0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.946+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fe16000c6a0 con 0x555daf901ed0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.946+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe168009540 con 0x555daf901ed0 2024-04-02T17:34:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.946+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe16000c8a0 con 0x555daf901ed0 2024-04-02T17:34:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.947+0000 7fe17822e740 10 client.27578 fetch_fsmap learned FSMap version 59 2024-04-02T17:34:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.947+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555daf97e050 con 0x555daf901ed0 2024-04-02T17:34:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168009540 con 0x555daf901ed0 2024-04-02T17:34:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fe160038070 con 0x555daf901ed0 2024-04-02T17:34:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe17822e740 10 client.27578 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:34:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe17822e740 10 client.27578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:34:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe17822e740 20 client.27578 populate_metadata read hostname 'smithi059' 2024-04-02T17:34:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.948+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf97e970 0x555daf980d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:34:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.949+0000 7fe17822e740 4 client.27578 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:34:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.950+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555daf8752b0 con 0x555daf97e970 2024-04-02T17:34:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.949+0000 7fe16e800640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf97e970 0x555daf980d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:34:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.950+0000 7fe16e800640 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf97e970 0x555daf980d90 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb672dc87844a978 server_cookie=7c4718f9ce56edfd in_seq=0 out_seq=0 2024-04-02T17:34:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:51.950+0000 7fe14fe00640 10 client.27578 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:51 smithi059 ceph-mon[31664]: pgmap v1043: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:34:52.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:51 smithi164 ceph-mon[36499]: pgmap v1043: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:34:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:51 smithi154 ceph-mon[36479]: pgmap v1043: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:34:52.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 367728 38.25 MB/sec execute 379 sec latency 603.752 ms 2024-04-02T17:34:53.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 367778 38.16 MB/sec execute 380 sec latency 1024.462 ms 2024-04-02T17:34:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.664+0000 7fe14fe00640 1 -- 172.21.15.59:0/1486026418 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555daf8752b0 con 0x555daf97e970 2024-04-02T17:34:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.664+0000 7fe14fe00640 10 client.27578 handle_command_reply: tid=0 2024-04-02T17:34:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.664+0000 7fe17822e740 1 client.27578 shutdown 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.664+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 msgr2=0x555daf910890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.664+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe16001d040 tx=0x7fe1600099b0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 shutdown_connections 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x555daf97e970 0x555daf980d90 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555daf9028c0 0x555daf910dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555daf907a30 0x555daf97c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 --2- 172.21.15.59:0/1486026418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555daf901ed0 0x555daf910890 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 >> 172.21.15.59:0/1486026418 conn(0x555daf8fd440 msgr2=0x555daf90c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe14fe00640 0 client.27578 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.665+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 shutdown_connections 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.666+0000 7fe17822e740 1 -- 172.21.15.59:0/1486026418 wait complete. 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.666+0000 7fe17822e740 20 client.27578 trim_cache size 0 max 16384 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:34:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:34:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 msgr2=0x7fe1540a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 0x7fe1540a62d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe15c09c5f0 tx=0x7fe15c00fd40 comp rx=0 tx=0).stop 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 msgr2=0x7fe1701a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00c970 tx=0x7fe15c00ce30 comp rx=0 tx=0).stop 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 shutdown_connections 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fe1540a3e10 0x7fe1540a62d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe170109350 0x7fe17019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170109d10 0x7fe1701a5970 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 --2- 172.21.15.59:0/3511095217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe170108160 0x7fe17019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:34:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.667+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 >> 172.21.15.59:0/3511095217 conn(0x7fe170077110 msgr2=0x7fe17010c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:34:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.668+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 shutdown_connections 2024-04-02T17:34:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:34:53.668+0000 7fe175400640 1 -- 172.21.15.59:0/3511095217 wait complete. 2024-04-02T17:34:54.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:53 smithi059 ceph-mon[31664]: pgmap v1044: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-04-02T17:34:54.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:53 smithi164 ceph-mon[36499]: pgmap v1044: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-04-02T17:34:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:53 smithi154 ceph-mon[36479]: pgmap v1044: 97 pgs: 97 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 588 op/s 2024-04-02T17:34:54.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 368142 38.12 MB/sec execute 381 sec latency 959.832 ms 2024-04-02T17:34:55.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 368515 38.08 MB/sec execute 382 sec latency 177.142 ms 2024-04-02T17:34:56.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:55 smithi059 ceph-mon[31664]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 673 op/s 2024-04-02T17:34:56.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:55 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:34:56.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:55 smithi164 ceph-mon[36499]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 673 op/s 2024-04-02T17:34:56.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:55 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:34:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:55 smithi154 ceph-mon[36479]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 673 op/s 2024-04-02T17:34:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:55 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:34:56.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 369122 38.06 MB/sec execute 383 sec latency 74.493 ms 2024-04-02T17:34:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:57 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:57 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:57 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:57 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:57 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:57 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:57 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:57 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:57 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:57 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:57 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:57 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:57.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 369574 38.03 MB/sec execute 384 sec latency 243.235 ms 2024-04-02T17:34:58.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:58 smithi059 ceph-mon[31664]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-04-02T17:34:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:34:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:34:58.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:34:58 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:58 smithi164 ceph-mon[36499]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-04-02T17:34:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:34:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:34:58.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:34:58 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:58 smithi154 ceph-mon[36479]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 746 op/s 2024-04-02T17:34:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:34:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:34:58.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:34:58 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:34:58.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 370160 38.01 MB/sec execute 385 sec latency 123.425 ms 2024-04-02T17:34:59.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 370807 37.98 MB/sec execute 386 sec latency 165.561 ms 2024-04-02T17:35:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:00 smithi059 ceph-mon[31664]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-04-02T17:35:00.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:00 smithi164 ceph-mon[36499]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-04-02T17:35:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:00 smithi154 ceph-mon[36479]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-04-02T17:35:00.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 371656 38.02 MB/sec execute 387 sec latency 53.256 ms 2024-04-02T17:35:01.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 372501 38.04 MB/sec execute 388 sec latency 67.516 ms 2024-04-02T17:35:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:02 smithi059 ceph-mon[31664]: pgmap v1048: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 826 op/s 2024-04-02T17:35:02.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:02 smithi164 ceph-mon[36499]: pgmap v1048: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 826 op/s 2024-04-02T17:35:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:02 smithi154 ceph-mon[36479]: pgmap v1048: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 826 op/s 2024-04-02T17:35:02.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 373210 38.05 MB/sec execute 389 sec latency 48.286 ms 2024-04-02T17:35:03.648 INFO:tasks.workunit.client.0.smithi059.stdout: 10 373616 38.01 MB/sec execute 390 sec latency 368.230 ms 2024-04-02T17:35:04.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:04 smithi059 ceph-mon[31664]: pgmap v1049: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 884 op/s 2024-04-02T17:35:04.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:04 smithi164 ceph-mon[36499]: pgmap v1049: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 884 op/s 2024-04-02T17:35:04.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:04 smithi154 ceph-mon[36479]: pgmap v1049: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 884 op/s 2024-04-02T17:35:04.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 374270 37.97 MB/sec execute 391 sec latency 56.011 ms 2024-04-02T17:35:05.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 375110 37.99 MB/sec execute 392 sec latency 56.550 ms 2024-04-02T17:35:06.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:06 smithi164 ceph-mon[36499]: pgmap v1050: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 918 op/s 2024-04-02T17:35:06.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:06 smithi154 ceph-mon[36479]: pgmap v1050: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 918 op/s 2024-04-02T17:35:06.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:06 smithi059 ceph-mon[31664]: pgmap v1050: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 918 op/s 2024-04-02T17:35:06.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 376020 38.04 MB/sec execute 393 sec latency 53.957 ms 2024-04-02T17:35:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2534441428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 msgr2=0x7fbcc0109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2534441428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0109dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009ca0 tx=0x7fbcb802c100 comp rx=0 tx=0).stop 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2534441428 shutdown_connections 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2534441428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 0x7fbcc0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2534441428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0109dc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2534441428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc0101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2534441428 >> 172.21.15.59:0/2534441428 conn(0x7fbcc00fcf10 msgr2=0x7fbcc00ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2534441428 shutdown_connections 2024-04-02T17:35:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.828+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2534441428 wait complete. 2024-04-02T17:35:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.829+0000 7fbcc6a00640 1 Processor -- start 2024-04-02T17:35:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 -- start start 2024-04-02T17:35:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 0x7fbcc0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010d600 con 0x7fbcc0102b70 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010d480 con 0x7fbcc0101980 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010d780 con 0x7fbcc0103530 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 0x7fbcc0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58328/0 (socket says 172.21.15.59:58328) 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc5600640 1 -- 172.21.15.59:0/2119576302 learned_addr learned my addr 172.21.15.59:0/2119576302 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.830+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 -- 172.21.15.59:0/2119576302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 msgr2=0x7fbcc0074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 -- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 msgr2=0x7fbcc0073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 0x7fbcc0073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 -- 172.21.15.59:0/2119576302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc0074cc0 con 0x7fbcc0101980 2024-04-02T17:35:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc5600640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e9c0 tx=0x7fbcb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcbe000640 1 -- 172.21.15.59:0/2119576302 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000cd80 con 0x7fbcc0101980 2024-04-02T17:35:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcbe000640 1 -- 172.21.15.59:0/2119576302 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb00035f0 con 0x7fbcc0101980 2024-04-02T17:35:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01a1d30 con 0x7fbcc0101980 2024-04-02T17:35:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcbe000640 1 -- 172.21.15.59:0/2119576302 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0010690 con 0x7fbcc0101980 2024-04-02T17:35:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.831+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a21d0 con 0x7fbcc0101980 2024-04-02T17:35:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.834+0000 7fbcbe000640 1 -- 172.21.15.59:0/2119576302 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbcb00037a0 con 0x7fbcc0101980 2024-04-02T17:35:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 Processor -- start 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 -- start start 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a18a02ed70 con 0x55a18a0c9100 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a189fd72a0 con 0x55a18a0c9af0 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a18a02ad20 con 0x55a18a0cec60 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37972/0 (socket says 172.21.15.59:37972) 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52184/0 (socket says 172.21.15.59:52184) 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.835+0000 7fbcc6000640 1 -- 172.21.15.59:0/1338470368 learned_addr learned my addr 172.21.15.59:0/1338470368 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 -- 172.21.15.59:0/1338470368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 msgr2=0x55a18a0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 -- 172.21.15.59:0/1338470368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 msgr2=0x55a18a0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 -- 172.21.15.59:0/1338470368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a189feb840 con 0x55a18a0c9af0 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc4c00640 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcbe000640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 0x7fbca40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0ce5f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00ce260 tx=0x7fbcb00d6b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7412735afe5ea89c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcbe000640 1 -- 172.21.15.59:0/2119576302 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbcb00d0880 con 0x7fbcc0101980 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 0x7fbca40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcb6600640 1 -- 172.21.15.59:0/1338470368 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0098d40 con 0x55a18a0c9af0 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcb6600640 1 -- 172.21.15.59:0/1338470368 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb009d1a0 con 0x55a18a0c9af0 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.836+0000 7fbcb6600640 1 -- 172.21.15.59:0/1338470368 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb009c400 con 0x55a18a0c9af0 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1338470368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 msgr2=0x55a18a0ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0ce5f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00ce260 tx=0x7fbcb00d6b20 comp rx=0 tx=0).stop 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 0x7fbca40a6060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1338470368 shutdown_connections 2024-04-02T17:35:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0ce5f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1338470368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1338470368 >> 172.21.15.59:0/1338470368 conn(0x55a18a0c4670 msgr2=0x55a18a0c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1338470368 shutdown_connections 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.837+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1338470368 wait complete. 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 -- start start 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 Processor -- start 2024-04-02T17:35:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a18a0d1bb0 con 0x55a18a0c9100 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a189fea660 con 0x55a18a0c9af0 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.838+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a18a0d1d30 con 0x55a18a0cec60 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcca9fd740 10 client.0 _ll_register_callbacks cb 0x55a189fea050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:52194/0 (socket says 172.21.15.59:52194) 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 -- 172.21.15.59:0/750515800 learned_addr learned my addr 172.21.15.59:0/750515800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58342/0 (socket says 172.21.15.59:58342) 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc5600640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37984/0 (socket says 172.21.15.59:37984) 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 -- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 msgr2=0x55a18a0d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 -- 172.21.15.59:0/750515800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 msgr2=0x55a18a0d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc6000640 1 -- 172.21.15.59:0/750515800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a189ff9e80 con 0x55a18a0cec60 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc4c00640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.839+0000 7fbcc5600640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcc6000640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009ca0 tx=0x7fbcb80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9478c068b93c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8051020 con 0x55a18a0cec60 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbcb8003f00 con 0x55a18a0cec60 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb009d1a0 con 0x55a18a0cec60 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbcb805c050 con 0x55a18a0cec60 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcca9fd740 10 client.27649 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.840+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a189d43210 con 0x55a18a0cec60 2024-04-02T17:35:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb009d1a0 con 0x55a18a0cec60 2024-04-02T17:35:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbcb804d9e0 con 0x55a18a0cec60 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 10 client.27649 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 10 client.27649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 20 client.27649 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a0df4c0 0x55a18a0e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 4 client.27649 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.841+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a189fdecd0 con 0x55a18a0df4c0 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.842+0000 7fbcc5600640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a0df4c0 0x55a18a0e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.842+0000 7fbcc5600640 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a0df4c0 0x55a18a0e18f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c43008994a534f2 server_cookie=4774fc3c04e4ed3d in_seq=0 out_seq=0 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.842+0000 7fbcaaa00640 10 client.27649 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcaaa00640 1 -- 172.21.15.59:0/750515800 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a189fdecd0 con 0x55a18a0df4c0 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcaaa00640 10 client.27649 handle_command_reply: tid=0 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 client.27649 shutdown 2024-04-02T17:35:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 msgr2=0x55a18a0dab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009ca0 tx=0x7fbcb80095c0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 shutdown_connections 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a0df4c0 0x55a18a0e18f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0dab70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/750515800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcaaa00640 0 client.27649 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.843+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 >> 172.21.15.59:0/750515800 conn(0x55a18a0c4670 msgr2=0x55a18a0c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.844+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 shutdown_connections 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.844+0000 7fbcca9fd740 1 -- 172.21.15.59:0/750515800 wait complete. 2024-04-02T17:35:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.844+0000 7fbcca9fd740 20 client.27649 trim_cache size 0 max 16384 2024-04-02T17:35:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.862+0000 7fbcca9fd740 1 Processor -- start 2024-04-02T17:35:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 -- start start 2024-04-02T17:35:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0c9100 0x55a18a0d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9af0 0x55a18a0dab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01ab0d0 con 0x55a18a0c9af0 2024-04-02T17:35:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.865+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01acc10 con 0x55a18a0cec60 2024-04-02T17:35:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.866+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01ab7a0 con 0x55a18a0c9100 2024-04-02T17:35:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.870+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.870+0000 7fbcc6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58352/0 (socket says 172.21.15.59:58352) 2024-04-02T17:35:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.870+0000 7fbcc6000640 1 -- 172.21.15.59:0/2776671902 learned_addr learned my addr 172.21.15.59:0/2776671902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 -- 172.21.15.59:0/2776671902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0c9100 msgr2=0x55a18a0d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0c9100 0x55a18a0d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 -- 172.21.15.59:0/2776671902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9af0 msgr2=0x55a18a0dab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9af0 0x55a18a0dab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 -- 172.21.15.59:0/2776671902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a189e0f620 con 0x55a18a0cec60 2024-04-02T17:35:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.871+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8032bc0 tx=0x7fbcb8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec8ecc7973fdf94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.872+0000 7fbcb5c00640 1 -- 172.21.15.59:0/2776671902 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8051070 con 0x55a18a0cec60 2024-04-02T17:35:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.872+0000 7fbcb5c00640 1 -- 172.21.15.59:0/2776671902 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb8034cf0 con 0x55a18a0cec60 2024-04-02T17:35:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.872+0000 7fbcb5c00640 1 -- 172.21.15.59:0/2776671902 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8039040 con 0x55a18a0cec60 2024-04-02T17:35:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.873+0000 7fbcca9fd740 1 -- 172.21.15.59:0/2776671902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 msgr2=0x55a18a0c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.873+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8032bc0 tx=0x7fbcb8009bd0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.875+0000 7fbcca9fd740 1 -- 172.21.15.59:0/2776671902 shutdown_connections 2024-04-02T17:35:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.876+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9af0 0x55a18a0dab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.877+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0cec60 0x55a18a0c4a50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.877+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/2776671902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0c9100 0x55a18a0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.877+0000 7fbcca9fd740 1 -- 172.21.15.59:0/2776671902 >> 172.21.15.59:0/2776671902 conn(0x55a18a0c4670 msgr2=0x55a18a0ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.877+0000 7fbcca9fd740 1 -- 172.21.15.59:0/2776671902 shutdown_connections 2024-04-02T17:35:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.877+0000 7fbcca9fd740 1 -- 172.21.15.59:0/2776671902 wait complete. 2024-04-02T17:35:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.878+0000 7fbcca9fd740 1 -- start start 2024-04-02T17:35:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.879+0000 7fbcca9fd740 1 Processor -- start 2024-04-02T17:35:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01ac540 con 0x55a18a0c9100 2024-04-02T17:35:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01abe70 con 0x55a18a0c9af0 2024-04-02T17:35:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01a90b0 con 0x55a18a0cec60 2024-04-02T17:35:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.883+0000 7fbcca9fd740 10 client.0 _ll_register_callbacks cb 0x7fbcb8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.886+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.886+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.886+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:58364/0 (socket says 172.21.15.59:58364) 2024-04-02T17:35:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.886+0000 7fbcc5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37992/0 (socket says 172.21.15.59:37992) 2024-04-02T17:35:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.886+0000 7fbcc5600640 1 -- 172.21.15.59:0/1780150523 learned_addr learned my addr 172.21.15.59:0/1780150523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.888+0000 7fbcc6000640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.888+0000 7fbcc5600640 1 -- 172.21.15.59:0/1780150523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 msgr2=0x55a18a0d9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc5600640 1 -- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 msgr2=0x55a18a0d8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc5600640 1 -- 172.21.15.59:0/1780150523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18a0d9810 con 0x55a18a0c9100 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc6000640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.889+0000 7fbcc4c00640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.891+0000 7fbcc5600640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbcb009ca30 tx=0x7fbcb000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc2557d5fecb37a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.891+0000 7fbcaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:35:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.891+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb009c5e0 con 0x55a18a0c9100 2024-04-02T17:35:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.891+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbcb009ada0 con 0x55a18a0c9100 2024-04-02T17:35:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.892+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb8034cf0 con 0x55a18a0c9100 2024-04-02T17:35:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.893+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbcb009c870 con 0x55a18a0c9100 2024-04-02T17:35:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.893+0000 7fbcca9fd740 10 client.17457 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.893+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a18a0dab20 con 0x55a18a0c9100 2024-04-02T17:35:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.894+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb8034cf0 con 0x55a18a0c9100 2024-04-02T17:35:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.894+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbcb00b9030 con 0x55a18a0c9100 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 10 client.17457 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 20 client.17457 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a145a70 0x55a18a147e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 4 client.17457 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a18a03c2b0 con 0x55a18a145a70 2024-04-02T17:35:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.898+0000 7fbcc4c00640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a145a70 0x55a18a147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.899+0000 7fbcc4c00640 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a145a70 0x55a18a147e90 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1904d980b6b43d9b server_cookie=31f3bf7aab71bf72 in_seq=0 out_seq=0 2024-04-02T17:35:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:06.899+0000 7fbcaaa00640 10 client.17457 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:07.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 376128 37.95 MB/sec execute 394 sec latency 844.033 ms 2024-04-02T17:35:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.241+0000 7fbcaaa00640 1 -- 172.21.15.59:0/1780150523 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a18a03c2b0 con 0x55a18a145a70 2024-04-02T17:35:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.241+0000 7fbcaaa00640 10 client.17457 handle_command_reply: tid=0 2024-04-02T17:35:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.241+0000 7fbcca9fd740 1 client.17457 shutdown 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.242+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 msgr2=0x55a18a0d7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.242+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbcb009ca30 tx=0x7fbcb000c7d0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 shutdown_connections 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55a18a145a70 0x55a18a147e90 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcaaa00640 0 client.17457 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a18a0cec60 0x55a18a0d9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a18a0c9af0 0x55a18a0d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 --2- 172.21.15.59:0/1780150523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18a0c9100 0x55a18a0d7cd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 >> 172.21.15.59:0/1780150523 conn(0x55a18a0c4670 msgr2=0x55a18a0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 shutdown_connections 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.243+0000 7fbcca9fd740 1 -- 172.21.15.59:0/1780150523 wait complete. 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.244+0000 7fbcca9fd740 20 client.17457 trim_cache size 0 max 16384 2024-04-02T17:35:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.245+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 msgr2=0x7fbca40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.245+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 0x7fbca40a6060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8032590 comp rx=0 tx=0).stop 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.245+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 msgr2=0x7fbcc00758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.245+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e9c0 tx=0x7fbcb000ee80 comp rx=0 tx=0).stop 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 shutdown_connections 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbca40a3ba0 0x7fbca40a6060 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc0102b70 0x7fbcc0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc0103530 0x7fbcc0074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 --2- 172.21.15.59:0/2119576302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcc0101980 0x7fbcc00758b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 >> 172.21.15.59:0/2119576302 conn(0x7fbcc00fcf10 msgr2=0x7fbcc0105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 shutdown_connections 2024-04-02T17:35:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:08.246+0000 7fbcc6a00640 1 -- 172.21.15.59:0/2119576302 wait complete. 2024-04-02T17:35:08.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:08 smithi164 ceph-mon[36499]: pgmap v1051: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 746 op/s 2024-04-02T17:35:08.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:08 smithi154 ceph-mon[36479]: pgmap v1051: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 746 op/s 2024-04-02T17:35:08.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:08 smithi059 ceph-mon[31664]: pgmap v1051: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 10 MiB/s wr, 746 op/s 2024-04-02T17:35:08.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 376472 37.91 MB/sec execute 395 sec latency 480.096 ms 2024-04-02T17:35:09.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 376915 37.87 MB/sec execute 396 sec latency 135.750 ms 2024-04-02T17:35:10.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:10 smithi164 ceph-mon[36499]: pgmap v1052: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-04-02T17:35:10.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:10 smithi059 ceph-mon[31664]: pgmap v1052: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-04-02T17:35:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:10 smithi154 ceph-mon[36479]: pgmap v1052: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-04-02T17:35:10.649 INFO:tasks.workunit.client.0.smithi059.stdout: 10 377601 37.88 MB/sec execute 397 sec latency 90.861 ms 2024-04-02T17:35:11.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 378343 37.86 MB/sec execute 398 sec latency 66.756 ms 2024-04-02T17:35:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:12 smithi154 ceph-mon[36479]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-04-02T17:35:12.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:12 smithi059 ceph-mon[31664]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-04-02T17:35:12.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:12 smithi164 ceph-mon[36499]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 648 op/s 2024-04-02T17:35:12.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 379096 37.88 MB/sec execute 399 sec latency 67.917 ms 2024-04-02T17:35:13.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 379815 37.88 MB/sec execute 400 sec latency 55.473 ms 2024-04-02T17:35:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:14 smithi154 ceph-mon[36479]: pgmap v1054: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:14.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:14 smithi059 ceph-mon[31664]: pgmap v1054: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:14.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:14 smithi164 ceph-mon[36499]: pgmap v1054: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 380506 37.87 MB/sec execute 401 sec latency 40.105 ms 2024-04-02T17:35:15.653 INFO:tasks.workunit.client.0.smithi059.stdout: 10 381274 37.89 MB/sec execute 402 sec latency 79.723 ms 2024-04-02T17:35:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:16 smithi154 ceph-mon[36479]: pgmap v1055: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 759 op/s 2024-04-02T17:35:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:16 smithi059 ceph-mon[31664]: pgmap v1055: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 759 op/s 2024-04-02T17:35:16.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:16 smithi164 ceph-mon[36499]: pgmap v1055: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 759 op/s 2024-04-02T17:35:16.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 381998 37.88 MB/sec execute 403 sec latency 199.117 ms 2024-04-02T17:35:17.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 382793 37.91 MB/sec execute 404 sec latency 47.469 ms 2024-04-02T17:35:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:18 smithi154 ceph-mon[36479]: pgmap v1056: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-04-02T17:35:18.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:18 smithi059 ceph-mon[31664]: pgmap v1056: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-04-02T17:35:18.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:18 smithi164 ceph-mon[36499]: pgmap v1056: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 729 op/s 2024-04-02T17:35:18.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 383569 37.90 MB/sec execute 405 sec latency 45.559 ms 2024-04-02T17:35:19.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 384195 37.90 MB/sec execute 406 sec latency 55.612 ms 2024-04-02T17:35:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:20 smithi154 ceph-mon[36479]: pgmap v1057: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 719 op/s 2024-04-02T17:35:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:20 smithi059 ceph-mon[31664]: pgmap v1057: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 719 op/s 2024-04-02T17:35:20.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:20 smithi164 ceph-mon[36499]: pgmap v1057: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 719 op/s 2024-04-02T17:35:20.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 384997 37.91 MB/sec execute 407 sec latency 82.777 ms 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 -- 172.21.15.59:0/2507593562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e81036e0 msgr2=0x7f88e8109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 --2- 172.21.15.59:0/2507593562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e81036e0 0x7f88e8109f70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009ca0 tx=0x7f88e002c0f0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 -- 172.21.15.59:0/2507593562 shutdown_connections 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 --2- 172.21.15.59:0/2507593562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e8102d20 0x7f88e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 --2- 172.21.15.59:0/2507593562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e81036e0 0x7f88e8109f70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 --2- 172.21.15.59:0/2507593562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e8101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.445+0000 7f88eec00640 1 -- 172.21.15.59:0/2507593562 >> 172.21.15.59:0/2507593562 conn(0x7f88e80fd0c0 msgr2=0x7f88e80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.446+0000 7f88eec00640 1 -- 172.21.15.59:0/2507593562 shutdown_connections 2024-04-02T17:35:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.446+0000 7f88eec00640 1 -- 172.21.15.59:0/2507593562 wait complete. 2024-04-02T17:35:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.447+0000 7f88eec00640 1 Processor -- start 2024-04-02T17:35:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.447+0000 7f88eec00640 1 -- start start 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e80758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 0x7f88e8073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e81036e0 0x7f88e8074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d790 con 0x7f88e81036e0 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d610 con 0x7f88e8102d20 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88eec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d910 con 0x7f88e8101b30 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e80758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e80758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58354/0 (socket says 172.21.15.59:58354) 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88e7400640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 0x7f88e8073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88e7e00640 1 -- 172.21.15.59:0/2687809234 learned_addr learned my addr 172.21.15.59:0/2687809234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.448+0000 7f88ecc00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e81036e0 0x7f88e8074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 -- 172.21.15.59:0/2687809234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 msgr2=0x7f88e80758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e80758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 -- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e81036e0 msgr2=0x7f88e8074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e81036e0 0x7f88e8074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 -- 172.21.15.59:0/2687809234 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8074cc0 con 0x7f88e8102d20 2024-04-02T17:35:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.449+0000 7f88e7400640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 0x7f88e8073ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9f0 tx=0x7f88d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.450+0000 7f88e4c00640 1 -- 172.21.15.59:0/2687809234 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d400ccc0 con 0x7f88e8102d20 2024-04-02T17:35:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.450+0000 7f88e4c00640 1 -- 172.21.15.59:0/2687809234 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d4010910 con 0x7f88e8102d20 2024-04-02T17:35:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.450+0000 7f88e4c00640 1 -- 172.21.15.59:0/2687809234 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4018960 con 0x7f88e8102d20 2024-04-02T17:35:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.451+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81a1e00 con 0x7f88e8102d20 2024-04-02T17:35:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.451+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81a2210 con 0x7f88e8102d20 2024-04-02T17:35:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.454+0000 7f88f19f8740 1 Processor -- start 2024-04-02T17:35:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 -- start start 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd07ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd07fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd0826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558dfcfd7d70 con 0x558dfd07b0d0 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558dfcf802a0 con 0x558dfd07a6e0 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88f19f8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558dfcfd3d20 con 0x558dfd080240 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd0826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.455+0000 7f88e4c00640 1 -- 172.21.15.59:0/2687809234 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f88d4010ac0 con 0x7f88e8102d20 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.456+0000 7f88e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd0826d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:58366/0 (socket says 172.21.15.59:58366) 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.456+0000 7f88e7400640 1 -- 172.21.15.59:0/4186094620 learned_addr learned my addr 172.21.15.59:0/4186094620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.456+0000 7f88e7e00640 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd07fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.456+0000 7f88ecc00640 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd07ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 -- 172.21.15.59:0/4186094620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 msgr2=0x558dfd0826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd0826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 -- 172.21.15.59:0/4186094620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 msgr2=0x558dfd07ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd07ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 -- 172.21.15.59:0/4186094620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfcf94840 con 0x558dfd07b0d0 2024-04-02T17:35:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.457+0000 7f88e7e00640 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd07fbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f88d80058c0 tx=0x7f88d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6c1af9550f9081 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88e4c00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 0x7f88cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88dd200640 1 -- 172.21.15.59:0/4186094620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8004220 con 0x558dfd07b0d0 2024-04-02T17:35:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88dd200640 1 -- 172.21.15.59:0/4186094620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d80043c0 con 0x558dfd07b0d0 2024-04-02T17:35:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88dd200640 1 -- 172.21.15.59:0/4186094620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8004db0 con 0x558dfd07b0d0 2024-04-02T17:35:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88ecc00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 0x7f88cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.458+0000 7f88e4c00640 1 -- 172.21.15.59:0/2687809234 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f88d40d0970 con 0x7f88e8102d20 2024-04-02T17:35:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.460+0000 7f88f19f8740 1 -- 172.21.15.59:0/4186094620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 msgr2=0x558dfd07fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.460+0000 7f88f19f8740 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd07fbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f88d80058c0 tx=0x7f88d8011810 comp rx=0 tx=0).stop 2024-04-02T17:35:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.460+0000 7f88f19f8740 1 -- 172.21.15.59:0/4186094620 shutdown_connections 2024-04-02T17:35:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.460+0000 7f88f19f8740 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd07fbd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.461+0000 7f88f19f8740 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd0826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.461+0000 7f88f19f8740 1 --2- 172.21.15.59:0/4186094620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd07ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.461+0000 7f88f19f8740 1 -- 172.21.15.59:0/4186094620 >> 172.21.15.59:0/4186094620 conn(0x558dfd06d440 msgr2=0x558dfd06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.461+0000 7f88f19f8740 1 -- 172.21.15.59:0/4186094620 shutdown_connections 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.461+0000 7f88f19f8740 1 -- 172.21.15.59:0/4186094620 wait complete. 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.462+0000 7f88ecc00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 0x7f88cc0a6190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f88e002c630 tx=0x7f88e0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.462+0000 7f88f19f8740 1 -- start start 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.462+0000 7f88f19f8740 1 Processor -- start 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd084bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558dfd0831f0 con 0x558dfd07b0d0 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558dfcf93660 con 0x558dfd07a6e0 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558dfd083370 con 0x558dfd080240 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88f19f8740 10 client.0 _ll_register_callbacks cb 0x558dfcf93050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50178/0 (socket says 172.21.15.59:50178) 2024-04-02T17:35:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88e7400640 1 -- 172.21.15.59:0/84079589 learned_addr learned my addr 172.21.15.59:0/84079589 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88e7e00640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd084bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.463+0000 7f88ecc00640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.464+0000 7f88e7400640 1 -- 172.21.15.59:0/84079589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 msgr2=0x558dfd08c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.464+0000 7f88e7400640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.464+0000 7f88e7400640 1 -- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 msgr2=0x558dfd084bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.464+0000 7f88e7400640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd084bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.464+0000 7f88e7400640 1 -- 172.21.15.59:0/84079589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfcfa2e80 con 0x558dfd07b0d0 2024-04-02T17:35:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.466+0000 7f88e7400640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d40ce3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912fb78d62bad7df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.466+0000 7f88d1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.467+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d409c6d0 con 0x558dfd07b0d0 2024-04-02T17:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.467+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f88d409bfe0 con 0x558dfd07b0d0 2024-04-02T17:35:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.468+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d80043c0 con 0x558dfd07b0d0 2024-04-02T17:35:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.477+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f88d409c4f0 con 0x558dfd07b0d0 2024-04-02T17:35:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.481+0000 7f88f19f8740 10 client.17469 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.481+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558dfccec210 con 0x558dfd07b0d0 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.482+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d80043c0 con 0x558dfd07b0d0 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.483+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f88d4018b00 con 0x558dfd07b0d0 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.483+0000 7f88f19f8740 10 client.17469 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.484+0000 7f88f19f8740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.484+0000 7f88f19f8740 20 client.17469 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.484+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd090f70 0x558dfd093410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.484+0000 7f88f19f8740 4 client.17469 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.484+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dfcf87cd0 con 0x558dfd090f70 2024-04-02T17:35:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.485+0000 7f88e7e00640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd090f70 0x558dfd093410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.487+0000 7f88d1600640 10 client.17469 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.487+0000 7f88e7e00640 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd090f70 0x558dfd093410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd93f02e00adc6f2 server_cookie=734b4738d9367ffc in_seq=0 out_seq=0 2024-04-02T17:35:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.490+0000 7f88d1600640 1 -- 172.21.15.59:0/84079589 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dfcf87cd0 con 0x558dfd090f70 2024-04-02T17:35:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.490+0000 7f88d1600640 10 client.17469 handle_command_reply: tid=0 2024-04-02T17:35:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.490+0000 7f88f19f8740 1 client.17469 shutdown 2024-04-02T17:35:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.492+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 msgr2=0x558dfd085130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.492+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d40ce3c0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 shutdown_connections 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd090f70 0x558dfd093410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07b0d0 0x558dfd085130 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 --2- 172.21.15.59:0/84079589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd07a6e0 0x558dfd084bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 >> 172.21.15.59:0/84079589 conn(0x558dfd06d440 msgr2=0x558dfd06c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88d1600640 0 client.17469 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.493+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 shutdown_connections 2024-04-02T17:35:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.494+0000 7f88f19f8740 1 -- 172.21.15.59:0/84079589 wait complete. 2024-04-02T17:35:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.495+0000 7f88f19f8740 20 client.17469 trim_cache size 0 max 16384 2024-04-02T17:35:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.540+0000 7f88f19f8740 1 Processor -- start 2024-04-02T17:35:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 -- start start 2024-04-02T17:35:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 0x558dfd070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd082860 0x558dfd097d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a7090 con 0x558dfd07a6e0 2024-04-02T17:35:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a8bd0 con 0x558dfd082860 2024-04-02T17:35:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.541+0000 7f88f19f8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a7760 con 0x558dfd080240 2024-04-02T17:35:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 0x558dfd070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 0x558dfd070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50180/0 (socket says 172.21.15.59:50180) 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7e00640 1 -- 172.21.15.59:0/2439376236 learned_addr learned my addr 172.21.15.59:0/2439376236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7400640 1 -- 172.21.15.59:0/2439376236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd082860 msgr2=0x558dfd097d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7400640 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd082860 0x558dfd097d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7400640 1 -- 172.21.15.59:0/2439376236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 msgr2=0x558dfd070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.542+0000 7f88e7400640 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 0x558dfd070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.543+0000 7f88e7400640 1 -- 172.21.15.59:0/2439376236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dfcdb8620 con 0x558dfd080240 2024-04-02T17:35:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.543+0000 7f88e7400640 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c130 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88d40d0000 tx=0x7f88d40b6240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c919d412594e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.544+0000 7f88d3e00640 1 -- 172.21.15.59:0/2439376236 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d409c4f0 con 0x558dfd080240 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.544+0000 7f88d3e00640 1 -- 172.21.15.59:0/2439376236 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d409d290 con 0x558dfd080240 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.544+0000 7f88d3e00640 1 -- 172.21.15.59:0/2439376236 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4098af0 con 0x558dfd080240 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.545+0000 7f88f19f8740 1 -- 172.21.15.59:0/2439376236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 msgr2=0x558dfd08c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.545+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c130 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88d40d0000 tx=0x7f88d40b6240 comp rx=0 tx=0).stop 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 -- 172.21.15.59:0/2439376236 shutdown_connections 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd082860 0x558dfd097d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd080240 0x558dfd08c130 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2439376236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd07a6e0 0x558dfd070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 -- 172.21.15.59:0/2439376236 >> 172.21.15.59:0/2439376236 conn(0x558dfd06d440 msgr2=0x558dfd0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 -- 172.21.15.59:0/2439376236 shutdown_connections 2024-04-02T17:35:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 -- 172.21.15.59:0/2439376236 wait complete. 2024-04-02T17:35:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.546+0000 7f88f19f8740 1 -- start start 2024-04-02T17:35:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 Processor -- start 2024-04-02T17:35:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd07a6e0 0x558dfd08bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd082860 0x558dfd08a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a8500 con 0x558dfd082860 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a7e30 con 0x558dfd080240 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81a5030 con 0x558dfd07a6e0 2024-04-02T17:35:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.547+0000 7f88f19f8740 10 client.0 _ll_register_callbacks cb 0x7f88d400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.549+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd07a6e0 0x558dfd08bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.549+0000 7f88e7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.549+0000 7f88e7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:39014/0 (socket says 172.21.15.59:39014) 2024-04-02T17:35:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.549+0000 7f88e7400640 1 -- 172.21.15.59:0/2729681870 learned_addr learned my addr 172.21.15.59:0/2729681870 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.549+0000 7f88ecc00640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd082860 0x558dfd08a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.551+0000 7f88e7400640 1 -- 172.21.15.59:0/2729681870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd07a6e0 msgr2=0x558dfd08bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.551+0000 7f88e7400640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd07a6e0 0x558dfd08bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.551+0000 7f88e7400640 1 -- 172.21.15.59:0/2729681870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd082860 msgr2=0x558dfd08a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.551+0000 7f88e7400640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd082860 0x558dfd08a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.551+0000 7f88e7400640 1 -- 172.21.15.59:0/2729681870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dfd08a8d0 con 0x558dfd080240 2024-04-02T17:35:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.552+0000 7f88e7400640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d4098950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8374737897843b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.552+0000 7f88d1600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:35:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.553+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d409c4f0 con 0x558dfd080240 2024-04-02T17:35:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.554+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f88d40b6040 con 0x558dfd080240 2024-04-02T17:35:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.554+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d409d290 con 0x558dfd080240 2024-04-02T17:35:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.555+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f88d409c4f0 con 0x558dfd080240 2024-04-02T17:35:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.555+0000 7f88f19f8740 10 client.27611 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.555+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558dfd072680 con 0x558dfd080240 2024-04-02T17:35:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.556+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d4080070 con 0x558dfd080240 2024-04-02T17:35:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.557+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7f88d409bc50 con 0x558dfd080240 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 10 client.27611 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 10 client.27611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 20 client.27611 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd072e50 0x558dfd075270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 4 client.27611 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.558+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f88d409d290 con 0x558dfd072e50 2024-04-02T17:35:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.559+0000 7f88e7e00640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd072e50 0x558dfd075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.559+0000 7f88e7e00640 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd072e50 0x558dfd075270 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a6b26f4f8270d8 server_cookie=e151c22677ff4a3d in_seq=0 out_seq=0 2024-04-02T17:35:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:21.560+0000 7f88d1600640 10 client.27611 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:21.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 385667 37.92 MB/sec execute 408 sec latency 167.240 ms 2024-04-02T17:35:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:22 smithi154 ceph-mon[36479]: pgmap v1058: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.4 MiB/s wr, 686 op/s 2024-04-02T17:35:22.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:22 smithi059 ceph-mon[31664]: pgmap v1058: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.4 MiB/s wr, 686 op/s 2024-04-02T17:35:22.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:22 smithi164 ceph-mon[36499]: pgmap v1058: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.4 MiB/s wr, 686 op/s 2024-04-02T17:35:22.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 385725 37.84 MB/sec execute 409 sec latency 770.666 ms 2024-04-02T17:35:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88d1600640 1 -- 172.21.15.59:0/2729681870 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f88d409d290 con 0x558dfd072e50 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88d1600640 10 client.27611 handle_command_reply: tid=0 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 client.27611 shutdown 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 msgr2=0x558dfd08c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d4098950 comp rx=0 tx=0).stop 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 shutdown_connections 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558dfd072e50 0x558dfd075270 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558dfd082860 0x558dfd08a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558dfd080240 0x558dfd08c000 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 --2- 172.21.15.59:0/2729681870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558dfd07a6e0 0x558dfd08bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 >> 172.21.15.59:0/2729681870 conn(0x558dfd06d440 msgr2=0x558dfd070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.760+0000 7f88d1600640 0 client.27611 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.761+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 shutdown_connections 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.761+0000 7f88f19f8740 1 -- 172.21.15.59:0/2729681870 wait complete. 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.761+0000 7f88f19f8740 20 client.27611 trim_cache size 0 max 16384 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:35:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.762+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 msgr2=0x7f88cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.762+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 0x7f88cc0a6190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f88e002c630 tx=0x7f88e0032590 comp rx=0 tx=0).stop 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.762+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 msgr2=0x7f88e8073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.762+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 0x7f88e8073ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9f0 tx=0x7f88d400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 shutdown_connections 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f88cc0a3cd0 0x7f88cc0a6190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e81036e0 0x7f88e8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e8102d20 0x7f88e8073ef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 --2- 172.21.15.59:0/2687809234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e8101b30 0x7f88e80758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 >> 172.21.15.59:0/2687809234 conn(0x7f88e80fd0c0 msgr2=0x7f88e8105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 shutdown_connections 2024-04-02T17:35:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:22.763+0000 7f88eec00640 1 -- 172.21.15.59:0/2687809234 wait complete. 2024-04-02T17:35:23.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 386064 37.78 MB/sec execute 410 sec latency 209.982 ms 2024-04-02T17:35:24.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:24 smithi154 ceph-mon[36479]: pgmap v1059: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-04-02T17:35:24.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:24 smithi059 ceph-mon[31664]: pgmap v1059: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-04-02T17:35:24.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:24 smithi164 ceph-mon[36499]: pgmap v1059: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 632 op/s 2024-04-02T17:35:24.650 INFO:tasks.workunit.client.0.smithi059.stdout: 10 386616 37.77 MB/sec execute 411 sec latency 176.826 ms 2024-04-02T17:35:25.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 387218 37.74 MB/sec execute 412 sec latency 154.119 ms 2024-04-02T17:35:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:26 smithi154 ceph-mon[36479]: pgmap v1060: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 697 op/s 2024-04-02T17:35:26.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:26 smithi059 ceph-mon[31664]: pgmap v1060: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 697 op/s 2024-04-02T17:35:26.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:26 smithi164 ceph-mon[36499]: pgmap v1060: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 697 op/s 2024-04-02T17:35:26.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 387911 37.73 MB/sec execute 413 sec latency 93.006 ms 2024-04-02T17:35:27.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 388741 37.75 MB/sec execute 414 sec latency 81.808 ms 2024-04-02T17:35:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:28 smithi154 ceph-mon[36479]: pgmap v1061: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-04-02T17:35:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:28 smithi059 ceph-mon[31664]: pgmap v1061: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-04-02T17:35:28.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:28 smithi164 ceph-mon[36499]: pgmap v1061: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-04-02T17:35:28.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 389538 37.78 MB/sec execute 415 sec latency 56.222 ms 2024-04-02T17:35:29.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 390278 37.78 MB/sec execute 416 sec latency 53.415 ms 2024-04-02T17:35:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:30 smithi154 ceph-mon[36479]: pgmap v1062: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-02T17:35:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:30 smithi059 ceph-mon[31664]: pgmap v1062: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-02T17:35:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:30 smithi164 ceph-mon[36499]: pgmap v1062: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-02T17:35:30.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 390932 37.77 MB/sec execute 417 sec latency 127.526 ms 2024-04-02T17:35:31.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 391657 37.77 MB/sec execute 418 sec latency 56.404 ms 2024-04-02T17:35:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:32 smithi154 ceph-mon[36479]: pgmap v1063: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-04-02T17:35:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:32 smithi059 ceph-mon[31664]: pgmap v1063: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-04-02T17:35:32.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:32 smithi164 ceph-mon[36499]: pgmap v1063: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 745 op/s 2024-04-02T17:35:32.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 392515 37.78 MB/sec execute 419 sec latency 55.058 ms 2024-04-02T17:35:33.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 393283 37.81 MB/sec execute 420 sec latency 69.713 ms 2024-04-02T17:35:34.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:34 smithi154 ceph-mon[36479]: pgmap v1064: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-04-02T17:35:34.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:34 smithi059 ceph-mon[31664]: pgmap v1064: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-04-02T17:35:34.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:34 smithi164 ceph-mon[36499]: pgmap v1064: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-04-02T17:35:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 393942 37.80 MB/sec execute 421 sec latency 41.351 ms 2024-04-02T17:35:35.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 394622 37.79 MB/sec execute 422 sec latency 72.375 ms 2024-04-02T17:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.946+0000 7fbde1c00640 1 -- 172.21.15.59:0/3089823350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0780a0 msgr2=0x7fbddc078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.946+0000 7fbde1c00640 1 --2- 172.21.15.59:0/3089823350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0780a0 0x7fbddc078940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009ca0 tx=0x7fbdd402c100 comp rx=0 tx=0).stop 2024-04-02T17:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.948+0000 7fbde1c00640 1 -- 172.21.15.59:0/3089823350 shutdown_connections 2024-04-02T17:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.948+0000 7fbde1c00640 1 --2- 172.21.15.59:0/3089823350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc077700 0x7fbddc077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.948+0000 7fbde1c00640 1 --2- 172.21.15.59:0/3089823350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc0780a0 0x7fbddc078940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.948+0000 7fbde1c00640 1 --2- 172.21.15.59:0/3089823350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0764c0 0x7fbddc0768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.948+0000 7fbde1c00640 1 -- 172.21.15.59:0/3089823350 >> 172.21.15.59:0/3089823350 conn(0x7fbddc0ff2f0 msgr2=0x7fbddc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.949+0000 7fbde1c00640 1 -- 172.21.15.59:0/3089823350 shutdown_connections 2024-04-02T17:35:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.950+0000 7fbde1c00640 1 -- 172.21.15.59:0/3089823350 wait complete. 2024-04-02T17:35:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.952+0000 7fbde1c00640 1 Processor -- start 2024-04-02T17:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.952+0000 7fbde1c00640 1 -- start start 2024-04-02T17:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.952+0000 7fbde1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc0764c0 0x7fbddc19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbde1c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc077700 0x7fbddc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbde1c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbde1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113e30 con 0x7fbddc0764c0 2024-04-02T17:35:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbde1c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113cb0 con 0x7fbddc0780a0 2024-04-02T17:35:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbde1c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113fb0 con 0x7fbddc077700 2024-04-02T17:35:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbddb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbddb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:47110/0 (socket says 172.21.15.59:47110) 2024-04-02T17:35:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbddb400640 1 -- 172.21.15.59:0/695811363 learned_addr learned my addr 172.21.15.59:0/695811363 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.953+0000 7fbdda000640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc077700 0x7fbddc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.954+0000 7fbddb400640 1 -- 172.21.15.59:0/695811363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc077700 msgr2=0x7fbddc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.954+0000 7fbddb400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc077700 0x7fbddc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.954+0000 7fbddb400640 1 -- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc0764c0 msgr2=0x7fbddc19e3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:35:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.957+0000 7fbddaa00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc0764c0 0x7fbddc19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.957+0000 7fbddb400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc0764c0 0x7fbddc19e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.957+0000 7fbddb400640 1 -- 172.21.15.59:0/695811363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc1a5fd0 con 0x7fbddc0780a0 2024-04-02T17:35:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.958+0000 7fbddb400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbdd402c640 tx=0x7fbdd4035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.959+0000 7fbdd3400640 1 -- 172.21.15.59:0/695811363 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4039040 con 0x7fbddc0780a0 2024-04-02T17:35:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.959+0000 7fbdd3400640 1 -- 172.21.15.59:0/695811363 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd4044070 con 0x7fbddc0780a0 2024-04-02T17:35:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.959+0000 7fbdd3400640 1 -- 172.21.15.59:0/695811363 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40044a0 con 0x7fbddc0780a0 2024-04-02T17:35:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.959+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbddc1a6260 con 0x7fbddc0780a0 2024-04-02T17:35:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.959+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc1a6780 con 0x7fbddc0780a0 2024-04-02T17:35:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.967+0000 7fbde486d740 1 Processor -- start 2024-04-02T17:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.970+0000 7fbdd3400640 1 -- 172.21.15.59:0/695811363 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbdd4004640 con 0x7fbddc0780a0 2024-04-02T17:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.970+0000 7fbde486d740 1 -- start start 2024-04-02T17:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e1fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d7dd70 con 0x55cdf7e17ed0 2024-04-02T17:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d262a0 con 0x55cdf7e188c0 2024-04-02T17:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbde486d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d79d20 con 0x55cdf7e1da30 2024-04-02T17:35:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbddaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbddaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:47126/0 (socket says 172.21.15.59:47126) 2024-04-02T17:35:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbddaa00640 1 -- 172.21.15.59:0/2117085757 learned_addr learned my addr 172.21.15.59:0/2117085757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.971+0000 7fbdda000640 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e1fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.972+0000 7fbdd3400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 0x7fbdc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.972+0000 7fbdda000640 1 -- 172.21.15.59:0/2117085757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 msgr2=0x55cdf7e1d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbdd3400640 1 -- 172.21.15.59:0/695811363 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbdd40f2870 con 0x7fbddc0780a0 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbdda000640 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbdda000640 1 -- 172.21.15.59:0/2117085757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 msgr2=0x55cdf7e182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbdda000640 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbddb400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 0x7fbdc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbdda000640 1 -- 172.21.15.59:0/2117085757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf7d3a840 con 0x55cdf7e1da30 2024-04-02T17:35:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.973+0000 7fbddaa00640 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:35:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.974+0000 7fbddb400640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 0x7fbdc00a6190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40be7f0 tx=0x7fbdd40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.975+0000 7fbdda000640 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e1fec0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbddc19f8c0 tx=0x7fbdc800acc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83819c02564e7103 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.976+0000 7fbdc7e00640 1 -- 172.21.15.59:0/2117085757 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800f9c0 con 0x55cdf7e1da30 2024-04-02T17:35:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.976+0000 7fbdc7e00640 1 -- 172.21.15.59:0/2117085757 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8018070 con 0x55cdf7e1da30 2024-04-02T17:35:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.976+0000 7fbdc7e00640 1 -- 172.21.15.59:0/2117085757 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8012360 con 0x55cdf7e1da30 2024-04-02T17:35:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.978+0000 7fbde486d740 1 -- 172.21.15.59:0/2117085757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 msgr2=0x55cdf7e1fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.978+0000 7fbde486d740 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e1fec0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbddc19f8c0 tx=0x7fbdc800acc0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.979+0000 7fbde486d740 1 -- 172.21.15.59:0/2117085757 shutdown_connections 2024-04-02T17:35:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.979+0000 7fbde486d740 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e1fec0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.979+0000 7fbde486d740 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e1d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.979+0000 7fbde486d740 1 --2- 172.21.15.59:0/2117085757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.979+0000 7fbde486d740 1 -- 172.21.15.59:0/2117085757 >> 172.21.15.59:0/2117085757 conn(0x55cdf7e13440 msgr2=0x55cdf7e13840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.980+0000 7fbde486d740 1 -- 172.21.15.59:0/2117085757 shutdown_connections 2024-04-02T17:35:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.980+0000 7fbde486d740 1 -- 172.21.15.59:0/2117085757 wait complete. 2024-04-02T17:35:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.980+0000 7fbde486d740 1 -- start start 2024-04-02T17:35:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.981+0000 7fbde486d740 1 Processor -- start 2024-04-02T17:35:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 0x55cdf7e229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7e209e0 con 0x55cdf7e1da30 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d39660 con 0x55cdf7e17ed0 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7e20b60 con 0x55cdf7e188c0 2024-04-02T17:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.984+0000 7fbde486d740 10 client.0 _ll_register_callbacks cb 0x55cdf7d39050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.986+0000 7fbdda000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.986+0000 7fbdda000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51554/0 (socket says 172.21.15.59:51554) 2024-04-02T17:35:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.986+0000 7fbdda000640 1 -- 172.21.15.59:0/2621738841 learned_addr learned my addr 172.21.15.59:0/2621738841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.987+0000 7fbdda000640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.987+0000 7fbddaa00640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 0x55cdf7e229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.988+0000 7fbdda000640 1 -- 172.21.15.59:0/2621738841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 msgr2=0x55cdf7e229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.989+0000 7fbdda000640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 0x55cdf7e229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.989+0000 7fbdda000640 1 -- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e17ed0 msgr2=0x55cdf7e224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.989+0000 7fbdda000640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.989+0000 7fbdda000640 1 -- 172.21.15.59:0/2621738841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf7d48e80 con 0x55cdf7e1da30 2024-04-02T17:35:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.989+0000 7fbddaa00640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 0x55cdf7e229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.990+0000 7fbdda000640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006cd0 tx=0x7fbdc800f0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0af8452e381474 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.990+0000 7fbdc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:35:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.991+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800fbd0 con 0x55cdf7e1da30 2024-04-02T17:35:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.991+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbdc800fd70 con 0x55cdf7e1da30 2024-04-02T17:35:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.992+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc8018070 con 0x55cdf7e1da30 2024-04-02T17:35:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.993+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8012520 con 0x55cdf7e1da30 2024-04-02T17:35:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbde486d740 10 client.17478 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cdf7a92210 con 0x55cdf7e1da30 2024-04-02T17:35:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8018070 con 0x55cdf7e1da30 2024-04-02T17:35:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbdc801f5c0 con 0x55cdf7e1da30 2024-04-02T17:35:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbde486d740 10 client.17478 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.994+0000 7fbde486d740 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.995+0000 7fbde486d740 20 client.17478 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.995+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e2e020 0x55cdf7e30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.995+0000 7fbde486d740 4 client.17478 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.995+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdf7d2dcd0 con 0x55cdf7e2e020 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.995+0000 7fbddaa00640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e2e020 0x55cdf7e30410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.996+0000 7fbdc4c00640 10 client.17478 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.996+0000 7fbddaa00640 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e2e020 0x55cdf7e30410 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61ad083dc769a24 server_cookie=341bf5a75eca71ab in_seq=0 out_seq=0 2024-04-02T17:35:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbdc4c00640 1 -- 172.21.15.59:0/2621738841 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdf7d2dcd0 con 0x55cdf7e2e020 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbdc4c00640 10 client.17478 handle_command_reply: tid=0 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 client.17478 shutdown 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 msgr2=0x55cdf7e29a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006cd0 tx=0x7fbdc800f0b0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 shutdown_connections 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e2e020 0x55cdf7e30410 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e1da30 0x55cdf7e29a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e188c0 0x55cdf7e229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbde486d740 1 --2- 172.21.15.59:0/2621738841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:35.999+0000 7fbdc4c00640 0 client.17478 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.000+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 >> 172.21.15.59:0/2621738841 conn(0x55cdf7e13440 msgr2=0x55cdf7e12640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.000+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 shutdown_connections 2024-04-02T17:35:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.000+0000 7fbde486d740 1 -- 172.21.15.59:0/2621738841 wait complete. 2024-04-02T17:35:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.000+0000 7fbde486d740 20 client.17478 trim_cache size 0 max 16384 2024-04-02T17:35:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.033+0000 7fbde486d740 1 Processor -- start 2024-04-02T17:35:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 -- start start 2024-04-02T17:35:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e16db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1af8c0 con 0x55cdf7e17ed0 2024-04-02T17:35:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b1400 con 0x55cdf7e188c0 2024-04-02T17:35:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.034+0000 7fbde486d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1aff90 con 0x55cdf7e1da30 2024-04-02T17:35:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:47138/0 (socket says 172.21.15.59:47138) 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 -- 172.21.15.59:0/1168754685 learned_addr learned my addr 172.21.15.59:0/1168754685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbdda000640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e16db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbdda000640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 -- 172.21.15.59:0/1168754685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 msgr2=0x55cdf7e34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e34c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 -- 172.21.15.59:0/1168754685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 msgr2=0x55cdf7e16db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e16db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbddaa00640 1 -- 172.21.15.59:0/1168754685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf7b5e620 con 0x55cdf7e188c0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.035+0000 7fbdda000640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e16db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.036+0000 7fbddaa00640 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4596585f8cafb815 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.036+0000 7fbdc7400640 1 -- 172.21.15.59:0/1168754685 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc004be0 con 0x55cdf7e188c0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.037+0000 7fbdc7400640 1 -- 172.21.15.59:0/1168754685 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc004d80 con 0x55cdf7e188c0 2024-04-02T17:35:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.037+0000 7fbdc7400640 1 -- 172.21.15.59:0/1168754685 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc013cb0 con 0x55cdf7e188c0 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 -- 172.21.15.59:0/1168754685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 msgr2=0x55cdf7e29940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc009460 comp rx=0 tx=0).stop 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 -- 172.21.15.59:0/1168754685 shutdown_connections 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e29940 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 --2- 172.21.15.59:0/1168754685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e16db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.038+0000 7fbde486d740 1 -- 172.21.15.59:0/1168754685 >> 172.21.15.59:0/1168754685 conn(0x55cdf7e13440 msgr2=0x55cdf7e166a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.039+0000 7fbde486d740 1 -- 172.21.15.59:0/1168754685 shutdown_connections 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.039+0000 7fbde486d740 1 -- 172.21.15.59:0/1168754685 wait complete. 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.039+0000 7fbde486d740 1 -- start start 2024-04-02T17:35:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.039+0000 7fbde486d740 1 Processor -- start 2024-04-02T17:35:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b0d30 con 0x55cdf7e17ed0 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b0660 con 0x55cdf7e188c0 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1ad8a0 con 0x55cdf7e1da30 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.040+0000 7fbde486d740 10 client.0 _ll_register_callbacks cb 0x7fbdc8013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.041+0000 7fbdda000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.041+0000 7fbdda000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51586/0 (socket says 172.21.15.59:51586) 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.041+0000 7fbdda000640 1 -- 172.21.15.59:0/1750561947 learned_addr learned my addr 172.21.15.59:0/1750561947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.041+0000 7fbdda000640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.043+0000 7fbdda000640 1 -- 172.21.15.59:0/1750561947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 msgr2=0x55cdf7e286f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.043+0000 7fbdda000640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e286f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.043+0000 7fbdda000640 1 -- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 msgr2=0x55cdf7e276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.043+0000 7fbdda000640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.043+0000 7fbdda000640 1 -- 172.21.15.59:0/1750561947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf7e28ce0 con 0x55cdf7e17ed0 2024-04-02T17:35:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.044+0000 7fbdda000640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009040 tx=0x7fbdc8014fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4867e7781bddef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.044+0000 7fbdc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:35:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.045+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800fde0 con 0x55cdf7e17ed0 2024-04-02T17:35:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.045+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbdc8013750 con 0x55cdf7e17ed0 2024-04-02T17:35:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.046+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdcc004d80 con 0x55cdf7e17ed0 2024-04-02T17:35:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.046+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8013950 con 0x55cdf7e17ed0 2024-04-02T17:35:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.046+0000 7fbde486d740 10 client.17484 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.046+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cdf7e945f0 con 0x55cdf7e17ed0 2024-04-02T17:35:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc004d80 con 0x55cdf7e17ed0 2024-04-02T17:35:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbdc801c440 con 0x55cdf7e17ed0 2024-04-02T17:35:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 10 client.17484 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 20 client.17484 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e94d80 0x55cdf7e971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 4 client.17484 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:35:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.048+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdf7d8b2b0 con 0x55cdf7e94d80 2024-04-02T17:35:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.049+0000 7fbddaa00640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e94d80 0x55cdf7e971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.050+0000 7fbdc4c00640 10 client.17484 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.050+0000 7fbddaa00640 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e94d80 0x55cdf7e971a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721989d60cec1724 server_cookie=6793b06703f76543 in_seq=0 out_seq=0 2024-04-02T17:35:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbdc4c00640 1 -- 172.21.15.59:0/1750561947 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdf7d8b2b0 con 0x55cdf7e94d80 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbdc4c00640 10 client.17484 handle_command_reply: tid=0 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 client.17484 shutdown 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 msgr2=0x55cdf7e271a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009040 tx=0x7fbdc8014fa0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 shutdown_connections 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55cdf7e94d80 0x55cdf7e971a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdf7e1da30 0x55cdf7e286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cdf7e188c0 0x55cdf7e276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 --2- 172.21.15.59:0/1750561947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdf7e17ed0 0x55cdf7e271a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 >> 172.21.15.59:0/1750561947 conn(0x55cdf7e13440 msgr2=0x55cdf7e16f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.389+0000 7fbdc4c00640 0 client.17484 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.390+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 shutdown_connections 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.390+0000 7fbde486d740 1 -- 172.21.15.59:0/1750561947 wait complete. 2024-04-02T17:35:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.390+0000 7fbde486d740 20 client.17484 trim_cache size 0 max 16384 2024-04-02T17:35:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:35:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:35:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:35:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 msgr2=0x7fbdc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 0x7fbdc00a6190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40be7f0 tx=0x7fbdd40095c0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 msgr2=0x7fbddc1a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbdd402c640 tx=0x7fbdd4035d10 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 shutdown_connections 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbddc0780a0 0x7fbddc1a5960 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbddc077700 0x7fbddc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbdc00a3cd0 0x7fbdc00a6190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 --2- 172.21.15.59:0/695811363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbddc0764c0 0x7fbddc19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 >> 172.21.15.59:0/695811363 conn(0x7fbddc0ff2f0 msgr2=0x7fbddc100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 shutdown_connections 2024-04-02T17:35:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:36.391+0000 7fbde1c00640 1 -- 172.21.15.59:0/695811363 wait complete. 2024-04-02T17:35:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:36 smithi154 ceph-mon[36479]: pgmap v1065: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-04-02T17:35:36.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:36 smithi059 ceph-mon[31664]: pgmap v1065: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-04-02T17:35:36.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:36 smithi164 ceph-mon[36499]: pgmap v1065: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 753 op/s 2024-04-02T17:35:36.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 395026 37.76 MB/sec execute 423 sec latency 225.423 ms 2024-04-02T17:35:37.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:37 smithi059 ceph-mon[31664]: pgmap v1066: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 695 op/s 2024-04-02T17:35:37.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:37 smithi164 ceph-mon[36499]: pgmap v1066: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 695 op/s 2024-04-02T17:35:37.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:37 smithi154 ceph-mon[36479]: pgmap v1066: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.7 MiB/s wr, 695 op/s 2024-04-02T17:35:37.651 INFO:tasks.workunit.client.0.smithi059.stdout: 10 395839 37.76 MB/sec execute 424 sec latency 60.344 ms 2024-04-02T17:35:38.653 INFO:tasks.workunit.client.0.smithi059.stdout: 10 396635 37.79 MB/sec execute 425 sec latency 42.689 ms 2024-04-02T17:35:39.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:39 smithi154 ceph-mon[36479]: pgmap v1067: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-04-02T17:35:39.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:39 smithi059 ceph-mon[31664]: pgmap v1067: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-04-02T17:35:39.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:39 smithi164 ceph-mon[36499]: pgmap v1067: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-04-02T17:35:39.653 INFO:tasks.workunit.client.0.smithi059.stdout: 10 397349 37.78 MB/sec execute 426 sec latency 40.065 ms 2024-04-02T17:35:40.653 INFO:tasks.workunit.client.0.smithi059.stdout: 10 398028 37.78 MB/sec execute 427 sec latency 57.910 ms 2024-04-02T17:35:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:41 smithi059 ceph-mon[31664]: pgmap v1068: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 608 op/s 2024-04-02T17:35:41.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:41 smithi164 ceph-mon[36499]: pgmap v1068: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 608 op/s 2024-04-02T17:35:41.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:41 smithi154 ceph-mon[36479]: pgmap v1068: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 608 op/s 2024-04-02T17:35:41.652 INFO:tasks.workunit.client.0.smithi059.stdout: 10 398805 37.78 MB/sec execute 428 sec latency 58.001 ms 2024-04-02T17:35:42.654 INFO:tasks.workunit.client.0.smithi059.stdout: 10 399502 37.79 MB/sec execute 429 sec latency 78.443 ms 2024-04-02T17:35:43.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:43 smithi059 ceph-mon[31664]: pgmap v1069: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-04-02T17:35:43.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:43 smithi164 ceph-mon[36499]: pgmap v1069: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-04-02T17:35:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:43 smithi154 ceph-mon[36479]: pgmap v1069: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-04-02T17:35:43.654 INFO:tasks.workunit.client.0.smithi059.stdout: 10 400025 37.79 MB/sec execute 430 sec latency 265.988 ms 2024-04-02T17:35:44.654 INFO:tasks.workunit.client.0.smithi059.stdout: 10 400727 37.75 MB/sec execute 431 sec latency 63.538 ms 2024-04-02T17:35:45.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:45 smithi059 ceph-mon[31664]: pgmap v1070: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-04-02T17:35:45.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:45 smithi164 ceph-mon[36499]: pgmap v1070: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-04-02T17:35:45.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:45 smithi154 ceph-mon[36479]: pgmap v1070: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-04-02T17:35:45.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 401427 37.77 MB/sec execute 432 sec latency 57.798 ms 2024-04-02T17:35:46.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 402213 37.79 MB/sec execute 433 sec latency 44.885 ms 2024-04-02T17:35:47.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:47 smithi059 ceph-mon[31664]: pgmap v1071: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-04-02T17:35:47.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:47 smithi164 ceph-mon[36499]: pgmap v1071: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-04-02T17:35:47.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:47 smithi154 ceph-mon[36479]: pgmap v1071: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-04-02T17:35:47.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 402929 37.79 MB/sec execute 434 sec latency 66.546 ms 2024-04-02T17:35:48.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 402996 37.71 MB/sec execute 435 sec latency 603.693 ms 2024-04-02T17:35:49.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:49 smithi059 ceph-mon[31664]: pgmap v1072: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-04-02T17:35:49.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:49 smithi164 ceph-mon[36499]: pgmap v1072: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-04-02T17:35:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.583+0000 7fbc56e00640 1 -- 172.21.15.59:0/1162045756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc480990f0 msgr2=0x7fbc48099570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.583+0000 7fbc56e00640 1 --2- 172.21.15.59:0/1162045756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc480990f0 0x7fbc48099570 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009900 tx=0x7fbc3c02bc80 comp rx=0 tx=0).stop 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 -- 172.21.15.59:0/1162045756 shutdown_connections 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 --2- 172.21.15.59:0/1162045756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc48099ab0 0x7fbc480a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 --2- 172.21.15.59:0/1162045756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc480990f0 0x7fbc48099570 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 --2- 172.21.15.59:0/1162045756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc48097f20 0x7fbc48098360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 -- 172.21.15.59:0/1162045756 >> 172.21.15.59:0/1162045756 conn(0x7fbc480934e0 msgr2=0x7fbc48095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.585+0000 7fbc56e00640 1 -- 172.21.15.59:0/1162045756 shutdown_connections 2024-04-02T17:35:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.586+0000 7fbc56e00640 1 -- 172.21.15.59:0/1162045756 wait complete. 2024-04-02T17:35:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.587+0000 7fbc56e00640 1 Processor -- start 2024-04-02T17:35:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.587+0000 7fbc56e00640 1 -- start start 2024-04-02T17:35:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc48097f20 0x7fbc480a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc48099ab0 0x7fbc480aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48009f30 con 0x7fbc48097f20 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48009db0 con 0x7fbc480990f0 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc56e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4800a0b0 con 0x7fbc48099ab0 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc55000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc55000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40906/0 (socket says 172.21.15.59:40906) 2024-04-02T17:35:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.590+0000 7fbc55000640 1 -- 172.21.15.59:0/2816849465 learned_addr learned my addr 172.21.15.59:0/2816849465 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.591+0000 7fbc55000640 1 -- 172.21.15.59:0/2816849465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc48099ab0 msgr2=0x7fbc480aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.591+0000 7fbc55000640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc48099ab0 0x7fbc480aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.591+0000 7fbc55000640 1 -- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc48097f20 msgr2=0x7fbc480a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.591+0000 7fbc55000640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc48097f20 0x7fbc480a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.591+0000 7fbc55000640 1 -- 172.21.15.59:0/2816849465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc480aad80 con 0x7fbc480990f0 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.592+0000 7fbc55000640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c02ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.592+0000 7fbc4e000640 1 -- 172.21.15.59:0/2816849465 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c041020 con 0x7fbc480990f0 2024-04-02T17:35:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.592+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc480ab010 con 0x7fbc480990f0 2024-04-02T17:35:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.592+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc48143350 con 0x7fbc480990f0 2024-04-02T17:35:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.593+0000 7fbc4e000640 1 -- 172.21.15.59:0/2816849465 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c009520 con 0x7fbc480990f0 2024-04-02T17:35:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.593+0000 7fbc4e000640 1 -- 172.21.15.59:0/2816849465 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c02ce60 con 0x7fbc480990f0 2024-04-02T17:35:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.598+0000 7fbc4e000640 1 -- 172.21.15.59:0/2816849465 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc3c044020 con 0x7fbc480990f0 2024-04-02T17:35:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.602+0000 7fbc4e000640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 0x7fbc380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.602+0000 7fbc4e000640 1 -- 172.21.15.59:0/2816849465 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbc3c0f1a20 con 0x7fbc480990f0 2024-04-02T17:35:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.603+0000 7fbc55a00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 0x7fbc380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.605+0000 7fbc55a00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 0x7fbc380a60f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbc48097c40 tx=0x7fbc44006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.608+0000 7fbc5b07f740 1 Processor -- start 2024-04-02T17:35:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.608+0000 7fbc5b07f740 1 -- start start 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f9930520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 0x5608f99355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 0x5608f99380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5608f9895d70 con 0x5608f9935c60 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5608f983e2a0 con 0x5608f9930af0 2024-04-02T17:35:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608f9891d20 con 0x5608f9930100 2024-04-02T17:35:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.609+0000 7fbc56400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 0x5608f99380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 0x5608f99355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc56400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 0x5608f99380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51874/0 (socket says 172.21.15.59:51874) 2024-04-02T17:35:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc56400640 1 -- 172.21.15.59:0/743853008 learned_addr learned my addr 172.21.15.59:0/743853008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 -- 172.21.15.59:0/743853008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 msgr2=0x5608f9930520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f9930520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 -- 172.21.15.59:0/743853008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 msgr2=0x5608f99380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 0x5608f99380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.610+0000 7fbc55000640 1 -- 172.21.15.59:0/743853008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f9852840 con 0x5608f9930af0 2024-04-02T17:35:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.611+0000 7fbc55000640 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 0x5608f99355f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c02bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c72cef679185596 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.611+0000 7fbc42a00640 1 -- 172.21.15.59:0/743853008 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0be340 con 0x5608f9930af0 2024-04-02T17:35:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.611+0000 7fbc42a00640 1 -- 172.21.15.59:0/743853008 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c0bdc40 con 0x5608f9930af0 2024-04-02T17:35:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.611+0000 7fbc42a00640 1 -- 172.21.15.59:0/743853008 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0ba2b0 con 0x5608f9930af0 2024-04-02T17:35:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.613+0000 7fbc5b07f740 1 -- 172.21.15.59:0/743853008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 msgr2=0x5608f99355f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.613+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 0x5608f99355f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006010 tx=0x7fbc3c02bc80 comp rx=0 tx=0).stop 2024-04-02T17:35:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 -- 172.21.15.59:0/743853008 shutdown_connections 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9935c60 0x5608f99380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9930af0 0x5608f99355f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/743853008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f9930520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 -- 172.21.15.59:0/743853008 >> 172.21.15.59:0/743853008 conn(0x5608f992b670 msgr2=0x5608f992ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 -- 172.21.15.59:0/743853008 shutdown_connections 2024-04-02T17:35:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.614+0000 7fbc5b07f740 1 -- 172.21.15.59:0/743853008 wait complete. 2024-04-02T17:35:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.615+0000 7fbc5b07f740 1 -- start start 2024-04-02T17:35:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.616+0000 7fbc5b07f740 1 Processor -- start 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.616+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.616+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f993a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9941880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5608f99389d0 con 0x5608f9930100 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5608f9851660 con 0x5608f9935c60 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608f9938b50 con 0x5608f9930af0 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc5b07f740 10 client.0 _ll_register_callbacks cb 0x5608f9851050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51876/0 (socket says 172.21.15.59:51876) 2024-04-02T17:35:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc55a00640 1 -- 172.21.15.59:0/2667073115 learned_addr learned my addr 172.21.15.59:0/2667073115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc56400640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9941880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.617+0000 7fbc55000640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f993a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 -- 172.21.15.59:0/2667073115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 msgr2=0x5608f993a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f993a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 -- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 msgr2=0x5608f9941880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9941880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 -- 172.21.15.59:0/2667073115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f9860e80 con 0x5608f9930100 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc55a00640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006eb0 tx=0x7fbc4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317c5c65713b5be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.618+0000 7fbc37400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.619+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44023060 con 0x5608f9930100 2024-04-02T17:35:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.619+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbc44013420 con 0x5608f9930100 2024-04-02T17:35:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.620+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc3c0bdc40 con 0x5608f9930100 2024-04-02T17:35:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.623+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbc4400be80 con 0x5608f9930100 2024-04-02T17:35:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.623+0000 7fbc5b07f740 10 client.17490 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.623+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5608f95aa210 con 0x5608f9930100 2024-04-02T17:35:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.624+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c0bdc40 con 0x5608f9930100 2024-04-02T17:35:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.624+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbc44012870 con 0x5608f9930100 2024-04-02T17:35:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 10 client.17490 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 20 client.17490 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f9946130 0x5608f9948580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 4 client.17490 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:35:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.626+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f9845cd0 con 0x5608f9946130 2024-04-02T17:35:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.627+0000 7fbc55000640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f9946130 0x5608f9948580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.628+0000 7fbc55000640 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f9946130 0x5608f9948580 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286f5125a6b4521d server_cookie=406550bf759c5cdc in_seq=0 out_seq=0 2024-04-02T17:35:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.628+0000 7fbc37400640 10 client.17490 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.629+0000 7fbc37400640 1 -- 172.21.15.59:0/2667073115 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608f9845cd0 con 0x5608f9946130 2024-04-02T17:35:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.629+0000 7fbc37400640 10 client.17490 handle_command_reply: tid=0 2024-04-02T17:35:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.629+0000 7fbc5b07f740 1 client.17490 shutdown 2024-04-02T17:35:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.630+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 msgr2=0x5608f993a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.630+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006eb0 tx=0x7fbc4400b840 comp rx=0 tx=0).stop 2024-04-02T17:35:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 shutdown_connections 2024-04-02T17:35:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f9946130 0x5608f9948580 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9941880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f993a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/2667073115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 >> 172.21.15.59:0/2667073115 conn(0x5608f992b670 msgr2=0x5608f992a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc37400640 0 client.17490 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.631+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 shutdown_connections 2024-04-02T17:35:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.632+0000 7fbc5b07f740 1 -- 172.21.15.59:0/2667073115 wait complete. 2024-04-02T17:35:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.632+0000 7fbc5b07f740 20 client.17490 trim_cache size 0 max 16384 2024-04-02T17:35:49.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:49 smithi154 ceph-mon[36479]: pgmap v1072: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-04-02T17:35:49.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 403273 37.65 MB/sec execute 436 sec latency 585.443 ms 2024-04-02T17:35:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 Processor -- start 2024-04-02T17:35:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 -- start start 2024-04-02T17:35:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f9941840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9926360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000e630 con 0x5608f9930100 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30010170 con 0x5608f9935c60 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.661+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000ed00 con 0x5608f9930af0 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51878/0 (socket says 172.21.15.59:51878) 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 -- 172.21.15.59:0/140933689 learned_addr learned my addr 172.21.15.59:0/140933689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55000640 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f9941840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc56400640 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9926360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 -- 172.21.15.59:0/140933689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 msgr2=0x5608f9941840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f9941840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 -- 172.21.15.59:0/140933689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 msgr2=0x5608f9926360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9926360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.662+0000 7fbc55a00640 1 -- 172.21.15.59:0/140933689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f9676620 con 0x5608f9930100 2024-04-02T17:35:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.663+0000 7fbc55a00640 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc440152e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1453b27ab548ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.664+0000 7fbc42000640 1 -- 172.21.15.59:0/140933689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44023040 con 0x5608f9930100 2024-04-02T17:35:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.665+0000 7fbc42000640 1 -- 172.21.15.59:0/140933689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44012890 con 0x5608f9930100 2024-04-02T17:35:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.665+0000 7fbc42000640 1 -- 172.21.15.59:0/140933689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4402d080 con 0x5608f9930100 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.665+0000 7fbc5b07f740 1 -- 172.21.15.59:0/140933689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 msgr2=0x5608f993a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.665+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc440152e0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 -- 172.21.15.59:0/140933689 shutdown_connections 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9926360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930af0 0x5608f9941840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/140933689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930100 0x5608f993a7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 -- 172.21.15.59:0/140933689 >> 172.21.15.59:0/140933689 conn(0x5608f992b670 msgr2=0x5608f992ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 -- 172.21.15.59:0/140933689 shutdown_connections 2024-04-02T17:35:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.666+0000 7fbc5b07f740 1 -- 172.21.15.59:0/140933689 wait complete. 2024-04-02T17:35:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.667+0000 7fbc5b07f740 1 -- start start 2024-04-02T17:35:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.667+0000 7fbc5b07f740 1 Processor -- start 2024-04-02T17:35:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 0x5608f993f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9940280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000faa0 con 0x5608f9930af0 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000f3d0 con 0x5608f9935c60 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000c610 con 0x5608f9930100 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc5b07f740 10 client.0 _ll_register_callbacks cb 0x7fbc44015cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:49596/0 (socket says 172.21.15.59:49596) 2024-04-02T17:35:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.668+0000 7fbc55a00640 1 -- 172.21.15.59:0/3466185901 learned_addr learned my addr 172.21.15.59:0/3466185901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:35:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.669+0000 7fbc56400640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9940280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.669+0000 7fbc55a00640 1 -- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 msgr2=0x5608f9940280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.669+0000 7fbc55000640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 0x5608f993f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.669+0000 7fbc55a00640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9940280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.669+0000 7fbc55a00640 1 -- 172.21.15.59:0/3466185901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 msgr2=0x5608f993f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc55a00640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 0x5608f993f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc55a00640 1 -- 172.21.15.59:0/3466185901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f9940870 con 0x5608f9930100 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc55000640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 0x5608f993f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:35:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc55a00640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b880 tx=0x7fbc44027620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c4cdb758286fc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:35:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc37400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:35:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.670+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44035020 con 0x5608f9930100 2024-04-02T17:35:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.671+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fbc44012a40 con 0x5608f9930100 2024-04-02T17:35:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.671+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc44012890 con 0x5608f9930100 2024-04-02T17:35:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.673+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fbc44023070 con 0x5608f9930100 2024-04-02T17:35:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.673+0000 7fbc5b07f740 10 client.27697 fetch_fsmap learned FSMap version 59 2024-04-02T17:35:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.673+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5608f9941b80 con 0x5608f9930100 2024-04-02T17:35:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44012890 con 0x5608f9930100 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7fbc44038020 con 0x5608f9930100 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 10 client.27697 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 10 client.27697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 20 client.27697 populate_metadata read hostname 'smithi059' 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f99ace10 0x5608f99af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 4 client.27697 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:35:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.674+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608f98a32b0 con 0x5608f99ace10 2024-04-02T17:35:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.675+0000 7fbc55000640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f99ace10 0x5608f99af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:35:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.677+0000 7fbc55000640 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f99ace10 0x5608f99af230 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4aa3e83d7d7b2 server_cookie=f2c283f135c28375 in_seq=0 out_seq=0 2024-04-02T17:35:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.678+0000 7fbc37400640 10 client.27697 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.943+0000 7fbc37400640 1 -- 172.21.15.59:0/3466185901 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608f98a32b0 con 0x5608f99ace10 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.943+0000 7fbc37400640 10 client.27697 handle_command_reply: tid=0 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.943+0000 7fbc5b07f740 1 client.27697 shutdown 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.944+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 msgr2=0x5608f993ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.944+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b880 tx=0x7fbc44027620 comp rx=0 tx=0).stop 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 shutdown_connections 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5608f99ace10 0x5608f99af230 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5608f9930af0 0x5608f993f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5608f9935c60 0x5608f9940280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 --2- 172.21.15.59:0/3466185901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608f9930100 0x5608f993ed30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 >> 172.21.15.59:0/3466185901 conn(0x5608f992b670 msgr2=0x5608f9926360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc37400640 0 client.27697 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 shutdown_connections 2024-04-02T17:35:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.945+0000 7fbc5b07f740 1 -- 172.21.15.59:0/3466185901 wait complete. 2024-04-02T17:35:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.946+0000 7fbc5b07f740 20 client.27697 trim_cache size 0 max 16384 2024-04-02T17:35:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:35:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:35:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:35:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.947+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 msgr2=0x7fbc380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.947+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 0x7fbc380a60f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbc48097c40 tx=0x7fbc44006c90 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.947+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 msgr2=0x7fbc480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.947+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c02ec20 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 shutdown_connections 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc48099ab0 0x7fbc480aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc480990f0 0x7fbc480a5700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fbc380a3c30 0x7fbc380a60f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 --2- 172.21.15.59:0/2816849465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc48097f20 0x7fbc480a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 >> 172.21.15.59:0/2816849465 conn(0x7fbc480934e0 msgr2=0x7fbc4800c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:35:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 shutdown_connections 2024-04-02T17:35:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:35:49.948+0000 7fbc56e00640 1 -- 172.21.15.59:0/2816849465 wait complete. 2024-04-02T17:35:50.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 403564 37.60 MB/sec execute 437 sec latency 259.204 ms 2024-04-02T17:35:51.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:51 smithi059 ceph-mon[31664]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:51.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:51 smithi164 ceph-mon[36499]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:51.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:51 smithi154 ceph-mon[36479]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 674 op/s 2024-04-02T17:35:51.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 404277 37.59 MB/sec execute 438 sec latency 79.937 ms 2024-04-02T17:35:52.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 404797 37.58 MB/sec execute 439 sec latency 219.007 ms 2024-04-02T17:35:53.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:53 smithi059 ceph-mon[31664]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-02T17:35:53.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:53 smithi164 ceph-mon[36499]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-02T17:35:53.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:53 smithi154 ceph-mon[36479]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-02T17:35:53.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 405584 37.58 MB/sec execute 440 sec latency 75.429 ms 2024-04-02T17:35:54.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 406326 37.62 MB/sec execute 441 sec latency 44.527 ms 2024-04-02T17:35:55.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:55 smithi059 ceph-mon[31664]: pgmap v1075: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-02T17:35:55.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:55 smithi164 ceph-mon[36499]: pgmap v1075: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-02T17:35:55.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:55 smithi154 ceph-mon[36479]: pgmap v1075: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-02T17:35:55.655 INFO:tasks.workunit.client.0.smithi059.stdout: 10 406926 37.59 MB/sec execute 442 sec latency 72.844 ms 2024-04-02T17:35:56.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 407607 37.58 MB/sec execute 443 sec latency 52.814 ms 2024-04-02T17:35:57.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:57 smithi059 ceph-mon[31664]: pgmap v1076: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 828 op/s 2024-04-02T17:35:57.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:57 smithi164 ceph-mon[36499]: pgmap v1076: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 828 op/s 2024-04-02T17:35:57.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:57 smithi154 ceph-mon[36479]: pgmap v1076: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 828 op/s 2024-04-02T17:35:57.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 408341 37.57 MB/sec execute 444 sec latency 107.220 ms 2024-04-02T17:35:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:58 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:35:58.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:58 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:35:58.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:58 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:35:58.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 408967 37.57 MB/sec execute 445 sec latency 476.400 ms 2024-04-02T17:35:59.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:35:59 smithi059 ceph-mon[31664]: pgmap v1077: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-04-02T17:35:59.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:35:59 smithi164 ceph-mon[36499]: pgmap v1077: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-04-02T17:35:59.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:35:59 smithi154 ceph-mon[36479]: pgmap v1077: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 707 op/s 2024-04-02T17:35:59.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 409548 37.57 MB/sec execute 446 sec latency 60.489 ms 2024-04-02T17:36:00.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:36:00.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:00 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:36:00.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:00 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:36:00.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:36:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:00 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:36:00.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:00 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:36:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:36:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:00 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:36:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:00 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:36:00.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 410045 37.55 MB/sec execute 447 sec latency 169.086 ms 2024-04-02T17:36:01.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 410705 37.54 MB/sec execute 448 sec latency 57.734 ms 2024-04-02T17:36:01.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:01 smithi164 ceph-mon[36499]: pgmap v1078: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-04-02T17:36:01.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:01 smithi154 ceph-mon[36479]: pgmap v1078: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-04-02T17:36:02.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:01 smithi059 ceph-mon[31664]: pgmap v1078: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 732 op/s 2024-04-02T17:36:02.657 INFO:tasks.workunit.client.0.smithi059.stdout: 10 411379 37.53 MB/sec execute 449 sec latency 61.376 ms 2024-04-02T17:36:03.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.257+0000 7ff218a00640 1 -- 172.21.15.59:0/3863764744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09c370 msgr2=0x7ff20c09c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.257+0000 7ff218a00640 1 --2- 172.21.15.59:0/3863764744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09c370 0x7ff20c09c790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff208009900 tx=0x7ff20802bc50 comp rx=0 tx=0).stop 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 -- 172.21.15.59:0/3863764744 shutdown_connections 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 --2- 172.21.15.59:0/3863764744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff20c09d560 0x7ff20c09d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 --2- 172.21.15.59:0/3863764744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09df20 0x7ff20c0a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 --2- 172.21.15.59:0/3863764744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09c370 0x7ff20c09c790 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 -- 172.21.15.59:0/3863764744 >> 172.21.15.59:0/3863764744 conn(0x7ff20c00b390 msgr2=0x7ff20c00b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 -- 172.21.15.59:0/3863764744 shutdown_connections 2024-04-02T17:36:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.258+0000 7ff218a00640 1 -- 172.21.15.59:0/3863764744 wait complete. 2024-04-02T17:36:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.260+0000 7ff218a00640 1 Processor -- start 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.260+0000 7ff218a00640 1 -- start start 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 0x7ff20c132660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff20c09d560 0x7ff20c132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 0x7ff20c139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c0a8020 con 0x7ff20c09d560 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c0a7ea0 con 0x7ff20c09df20 2024-04-02T17:36:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff218a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c0a81a0 con 0x7ff20c09c370 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff213400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 0x7ff20c132660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff213400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 0x7ff20c132660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36840/0 (socket says 172.21.15.59:36840) 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff213400640 1 -- 172.21.15.59:0/4266390346 learned_addr learned my addr 172.21.15.59:0/4266390346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.261+0000 7ff213e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 0x7ff20c139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.262+0000 7ff213e00640 1 -- 172.21.15.59:0/4266390346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 msgr2=0x7ff20c132660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.262+0000 7ff213e00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 0x7ff20c132660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.262+0000 7ff213e00640 1 -- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff20c09d560 msgr2=0x7ff20c132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.262+0000 7ff213e00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff20c09d560 0x7ff20c132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.262+0000 7ff213e00640 1 -- 172.21.15.59:0/4266390346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c13a280 con 0x7ff20c09df20 2024-04-02T17:36:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.263+0000 7ff213e00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 0x7ff20c139c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff21404fb30 tx=0x7ff21407b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.263+0000 7ff207e00640 1 -- 172.21.15.59:0/4266390346 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214068e50 con 0x7ff20c09df20 2024-04-02T17:36:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.264+0000 7ff207e00640 1 -- 172.21.15.59:0/4266390346 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21407d900 con 0x7ff20c09df20 2024-04-02T17:36:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.264+0000 7ff207e00640 1 -- 172.21.15.59:0/4266390346 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214085890 con 0x7ff20c09df20 2024-04-02T17:36:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.264+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c13a570 con 0x7ff20c09df20 2024-04-02T17:36:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.264+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c13aa40 con 0x7ff20c09df20 2024-04-02T17:36:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.269+0000 7ff21cdf1740 1 Processor -- start 2024-04-02T17:36:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.270+0000 7ff21cdf1740 1 -- start start 2024-04-02T17:36:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.270+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.270+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff207e00640 1 -- 172.21.15.59:0/4266390346 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff21407a040 con 0x7ff20c09df20 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff213400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff213400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44038/0 (socket says 172.21.15.59:44038) 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff213400640 1 -- 172.21.15.59:0/1567151117 learned_addr learned my addr 172.21.15.59:0/1567151117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654d7983d70 con 0x5654d7a1eaf0 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654d792c2a0 con 0x5654d7a1e100 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654d797fd20 con 0x5654d7a23c60 2024-04-02T17:36:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.271+0000 7ff212a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 -- 172.21.15.59:0/1567151117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 msgr2=0x5654d7a260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 -- 172.21.15.59:0/1567151117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 msgr2=0x5654d7a235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 -- 172.21.15.59:0/1567151117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7940840 con 0x5654d7a1e100 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.272+0000 7ff212a00640 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a1e520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff200009ec0 tx=0x7ff200009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e288fd8b9576961 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.273+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1567151117 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000ee20 con 0x5654d7a1e100 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.273+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1567151117 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200005020 con 0x5654d7a1e100 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.273+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1567151117 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004b60 con 0x5654d7a1e100 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.273+0000 7ff207e00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 0x7ff1fc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.273+0000 7ff207e00640 1 -- 172.21.15.59:0/4266390346 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff21413c480 con 0x7ff20c09df20 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.274+0000 7ff213400640 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:36:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.274+0000 7ff213400640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 0x7ff1fc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.274+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 msgr2=0x5654d7a1e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.274+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a1e520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff200009ec0 tx=0x7ff200009840 comp rx=0 tx=0).stop 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 shutdown_connections 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/1567151117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a1e520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 >> 172.21.15.59:0/1567151117 conn(0x5654d7a19670 msgr2=0x5654d7a19a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.275+0000 7ff213400640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 0x7ff1fc0a63e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff20802cfd0 tx=0x7ff20802c8d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.276+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 shutdown_connections 2024-04-02T17:36:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.276+0000 7ff21cdf1740 1 -- 172.21.15.59:0/1567151117 wait complete. 2024-04-02T17:36:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.278+0000 7ff21cdf1740 1 -- start start 2024-04-02T17:36:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.278+0000 7ff21cdf1740 1 Processor -- start 2024-04-02T17:36:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.279+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a26c10 con 0x5654d7a1eaf0 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654d793f660 con 0x5654d7a1e100 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a26d90 con 0x5654d7a23c60 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff21cdf1740 10 client.0 _ll_register_callbacks cb 0x5654d793f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff213e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff213e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36862/0 (socket says 172.21.15.59:36862) 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff213e00640 1 -- 172.21.15.59:0/381995623 learned_addr learned my addr 172.21.15.59:0/381995623 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.280+0000 7ff213400640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 -- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 msgr2=0x5654d7a286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a286d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 -- 172.21.15.59:0/381995623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 msgr2=0x5654d7a28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 -- 172.21.15.59:0/381995623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d794ee80 con 0x5654d7a23c60 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff213e00640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff214064c40 tx=0x7ff2141081e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93878806ffcb49e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff1f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214108da0 con 0x5654d7a23c60 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff21407dab0 con 0x5654d7a23c60 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.281+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff200005020 con 0x5654d7a23c60 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.283+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2141086a0 con 0x5654d7a23c60 2024-04-02T17:36:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.283+0000 7ff21cdf1740 10 client.27709 fetch_fsmap learned FSMap version 59 2024-04-02T17:36:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.283+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5654d7698210 con 0x5654d7a23c60 2024-04-02T17:36:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.283+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200005020 con 0x5654d7a23c60 2024-04-02T17:36:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.284+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7ff21408d030 con 0x5654d7a23c60 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 10 client.27709 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 10 client.27709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 20 client.27709 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a34570 0x5654d7a36a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 4 client.27709 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:36:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.285+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d7933cd0 con 0x5654d7a34570 2024-04-02T17:36:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.286+0000 7ff213400640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a34570 0x5654d7a36a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.289+0000 7ff213400640 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a34570 0x5654d7a36a00 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e928ff1d641093 server_cookie=b2b93503efad23ac in_seq=0 out_seq=0 2024-04-02T17:36:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.289+0000 7ff1f8c00640 10 client.27709 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff1f8c00640 1 -- 172.21.15.59:0/381995623 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654d7933cd0 con 0x5654d7a34570 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff1f8c00640 10 client.27709 handle_command_reply: tid=0 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 client.27709 shutdown 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 msgr2=0x5654d7a2fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff214064c40 tx=0x7ff2141081e0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 shutdown_connections 2024-04-02T17:36:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a34570 0x5654d7a36a00 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1eaf0 0x5654d7a28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2fc50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/381995623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1e100 0x5654d7a286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.291+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 >> 172.21.15.59:0/381995623 conn(0x5654d7a19670 msgr2=0x5654d7a18870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.292+0000 7ff1f8c00640 0 client.27709 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.293+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 shutdown_connections 2024-04-02T17:36:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.293+0000 7ff21cdf1740 1 -- 172.21.15.59:0/381995623 wait complete. 2024-04-02T17:36:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.293+0000 7ff21cdf1740 20 client.27709 trim_cache size 0 max 16384 2024-04-02T17:36:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.326+0000 7ff21cdf1740 1 Processor -- start 2024-04-02T17:36:03.326 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.327+0000 7ff21cdf1740 1 -- start start 2024-04-02T17:36:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.327+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 0x5654d7a1cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.327+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a17fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.327+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a23c60 0x5654d7a18520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff213400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 0x5654d7a1cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff213400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 0x5654d7a1cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36868/0 (socket says 172.21.15.59:36868) 2024-04-02T17:36:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c143960 con 0x5654d7a23c60 2024-04-02T17:36:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1454a0 con 0x5654d7a1eaf0 2024-04-02T17:36:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff213400640 1 -- 172.21.15.59:0/4281136538 learned_addr learned my addr 172.21.15.59:0/4281136538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c144030 con 0x5654d7a1e100 2024-04-02T17:36:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a17fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff213e00640 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a23c60 0x5654d7a18520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 -- 172.21.15.59:0/4281136538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 msgr2=0x5654d7a1cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 0x5654d7a1cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 -- 172.21.15.59:0/4281136538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a23c60 msgr2=0x5654d7a18520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a23c60 0x5654d7a18520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.328+0000 7ff212a00640 1 -- 172.21.15.59:0/4281136538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7764620 con 0x5654d7a1eaf0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.329+0000 7ff212a00640 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a17fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5654d7a1a5b0 tx=0x7ff200009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db4bf254c0f8fd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.329+0000 7ff1fb400640 1 -- 172.21.15.59:0/4281136538 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004040 con 0x5654d7a1eaf0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.330+0000 7ff1fb400640 1 -- 172.21.15.59:0/4281136538 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200022030 con 0x5654d7a1eaf0 2024-04-02T17:36:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.330+0000 7ff1fb400640 1 -- 172.21.15.59:0/4281136538 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004a20 con 0x5654d7a1eaf0 2024-04-02T17:36:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.330+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 msgr2=0x5654d7a17fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.330+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a17fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5654d7a1a5b0 tx=0x7ff200009b50 comp rx=0 tx=0).stop 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 shutdown_connections 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a23c60 0x5654d7a18520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a17fe0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/4281136538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a1e100 0x5654d7a1cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 >> 172.21.15.59:0/4281136538 conn(0x5654d7a19670 msgr2=0x5654d7a1c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 shutdown_connections 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.331+0000 7ff21cdf1740 1 -- 172.21.15.59:0/4281136538 wait complete. 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.332+0000 7ff21cdf1740 1 -- start start 2024-04-02T17:36:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.332+0000 7ff21cdf1740 1 Processor -- start 2024-04-02T17:36:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.332+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 0x5654d7a2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.332+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff21cdf1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c144dd0 con 0x5654d7a1e100 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c144700 con 0x5654d7a1eaf0 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff21cdf1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c141940 con 0x5654d7a23c60 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff21cdf1740 10 client.0 _ll_register_callbacks cb 0x7ff214104a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff212a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff213e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff213400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 0x5654d7a2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff213e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36884/0 (socket says 172.21.15.59:36884) 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff212a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51200/0 (socket says 172.21.15.59:51200) 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff212a00640 1 -- 172.21.15.59:0/3913443807 learned_addr learned my addr 172.21.15.59:0/3913443807 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.333+0000 7ff213400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 0x5654d7a2c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44068/0 (socket says 172.21.15.59:44068) 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.334+0000 7ff212a00640 1 -- 172.21.15.59:0/3913443807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 msgr2=0x5654d7a2d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.334+0000 7ff212a00640 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.334+0000 7ff212a00640 1 -- 172.21.15.59:0/3913443807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 msgr2=0x5654d7a2c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.334+0000 7ff212a00640 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 0x5654d7a2c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.334+0000 7ff212a00640 1 -- 172.21.15.59:0/3913443807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d7a2df50 con 0x5654d7a1eaf0 2024-04-02T17:36:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.336+0000 7ff212a00640 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff200009d70 tx=0x7ff200004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c410d1ef4850698 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.336+0000 7ff1f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:36:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.337+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200005570 con 0x5654d7a1eaf0 2024-04-02T17:36:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.337+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff200009530 con 0x5654d7a1eaf0 2024-04-02T17:36:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.337+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff200022030 con 0x5654d7a1eaf0 2024-04-02T17:36:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.338+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff200036050 con 0x5654d7a1eaf0 2024-04-02T17:36:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.338+0000 7ff21cdf1740 10 client.27656 fetch_fsmap learned FSMap version 59 2024-04-02T17:36:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.338+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5654d7a2f260 con 0x5654d7a1eaf0 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200022030 con 0x5654d7a1eaf0 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 59) ==== 8354+0+0 (secure 0 0 0) 0x7ff200003bd0 con 0x5654d7a1eaf0 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 10 client.27656 fetch_fsmap finished waiting for FSMap version 59 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 10 client.27656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 20 client.27656 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a9ae20 0x5654d7a9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 4 client.27656 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:36:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.339+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654d79912b0 con 0x5654d7a9ae20 2024-04-02T17:36:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.340+0000 7ff213400640 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a9ae20 0x5654d7a9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.341+0000 7ff213400640 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a9ae20 0x5654d7a9d210 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba48eb45a97ba082 server_cookie=9e62b1abc974cdba in_seq=0 out_seq=0 2024-04-02T17:36:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.341+0000 7ff1f8c00640 10 client.27656 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:03.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 411911 37.51 MB/sec execute 450 sec latency 223.267 ms 2024-04-02T17:36:03.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.661+0000 7ff1f8c00640 1 -- 172.21.15.59:0/3913443807 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654d79912b0 con 0x5654d7a9ae20 2024-04-02T17:36:03.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.661+0000 7ff1f8c00640 10 client.27656 handle_command_reply: tid=0 2024-04-02T17:36:03.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 client.27656 shutdown 2024-04-02T17:36:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 msgr2=0x5654d7a2ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff200009d70 tx=0x7ff200004ee0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 shutdown_connections 2024-04-02T17:36:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5654d7a9ae20 0x5654d7a9d210 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5654d7a23c60 0x5654d7a2d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5654d7a1eaf0 0x5654d7a2ca10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 --2- 172.21.15.59:0/3913443807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654d7a1e100 0x5654d7a2c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 >> 172.21.15.59:0/3913443807 conn(0x5654d7a19670 msgr2=0x5654d7a1d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.662+0000 7ff1f8c00640 0 client.27656 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.663+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 shutdown_connections 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.663+0000 7ff21cdf1740 1 -- 172.21.15.59:0/3913443807 wait complete. 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.663+0000 7ff21cdf1740 20 client.27656 trim_cache size 0 max 16384 2024-04-02T17:36:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:36:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:36:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:36:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:36:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 msgr2=0x7ff1fc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 0x7ff1fc0a63e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff20802cfd0 tx=0x7ff20802c8d0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 msgr2=0x7ff20c139c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 0x7ff20c139c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff21404fb30 tx=0x7ff21407b710 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 shutdown_connections 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff1fc0a3f20 0x7ff1fc0a63e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff20c09d560 0x7ff20c132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff20c09df20 0x7ff20c139c10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 --2- 172.21.15.59:0/4266390346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff20c09c370 0x7ff20c132660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 >> 172.21.15.59:0/4266390346 conn(0x7ff20c00b390 msgr2=0x7ff20c0a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 shutdown_connections 2024-04-02T17:36:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:03.668+0000 7ff218a00640 1 -- 172.21.15.59:0/4266390346 wait complete. 2024-04-02T17:36:03.821 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:03 smithi164 ceph-mon[36499]: pgmap v1079: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-04-02T17:36:03.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:03 smithi154 ceph-mon[36479]: pgmap v1079: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-04-02T17:36:04.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:03 smithi059 ceph-mon[31664]: pgmap v1079: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-04-02T17:36:04.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 412321 37.46 MB/sec execute 451 sec latency 362.138 ms 2024-04-02T17:36:05.656 INFO:tasks.workunit.client.0.smithi059.stdout: 10 413078 37.49 MB/sec execute 452 sec latency 89.223 ms 2024-04-02T17:36:05.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:05 smithi154 ceph-mon[36479]: pgmap v1080: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-04-02T17:36:06.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:05 smithi059 ceph-mon[31664]: pgmap v1080: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-04-02T17:36:06.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:05 smithi164 ceph-mon[36499]: pgmap v1080: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-04-02T17:36:06.659 INFO:tasks.workunit.client.0.smithi059.stdout: 10 413630 37.48 MB/sec execute 453 sec latency 75.470 ms 2024-04-02T17:36:07.658 INFO:tasks.workunit.client.0.smithi059.stdout: 10 414236 37.45 MB/sec execute 454 sec latency 81.782 ms 2024-04-02T17:36:07.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:07 smithi154 ceph-mon[36479]: pgmap v1081: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-04-02T17:36:07.934 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:07 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:08.050 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:07 smithi059 ceph-mon[31664]: pgmap v1081: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-04-02T17:36:08.050 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:07 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:07 smithi164 ceph-mon[36499]: pgmap v1081: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 557 op/s 2024-04-02T17:36:08.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:07 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:08.658 INFO:tasks.workunit.client.0.smithi059.stdout: 10 414810 37.44 MB/sec execute 455 sec latency 86.247 ms 2024-04-02T17:36:09.659 INFO:tasks.workunit.client.0.smithi059.stdout: 10 414894 37.38 MB/sec execute 456 sec latency 743.255 ms 2024-04-02T17:36:09.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:09 smithi154 ceph-mon[36479]: pgmap v1082: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-04-02T17:36:10.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:09 smithi059 ceph-mon[31664]: pgmap v1082: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-04-02T17:36:10.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:09 smithi164 ceph-mon[36499]: pgmap v1082: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-04-02T17:36:10.659 INFO:tasks.workunit.client.0.smithi059.stdout: 10 415251 37.33 MB/sec execute 457 sec latency 894.355 ms 2024-04-02T17:36:11.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 415670 37.30 MB/sec execute 458 sec latency 139.835 ms 2024-04-02T17:36:11.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:11 smithi154 ceph-mon[36479]: pgmap v1083: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 624 op/s 2024-04-02T17:36:12.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:11 smithi059 ceph-mon[31664]: pgmap v1083: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 624 op/s 2024-04-02T17:36:12.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:11 smithi164 ceph-mon[36499]: pgmap v1083: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 624 op/s 2024-04-02T17:36:12.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 416260 37.28 MB/sec execute 459 sec latency 65.198 ms 2024-04-02T17:36:12.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:12 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:13.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:12 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:13.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:12 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:13.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 416906 37.27 MB/sec execute 460 sec latency 101.221 ms 2024-04-02T17:36:13.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:13 smithi154 ceph-mon[36479]: pgmap v1084: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-04-02T17:36:13.972 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:13 smithi164 ceph-mon[36499]: pgmap v1084: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-04-02T17:36:14.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:13 smithi059 ceph-mon[31664]: pgmap v1084: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.3 MiB/s wr, 634 op/s 2024-04-02T17:36:14.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 417559 37.28 MB/sec execute 461 sec latency 92.979 ms 2024-04-02T17:36:15.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 418018 37.27 MB/sec execute 462 sec latency 238.327 ms 2024-04-02T17:36:15.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:15 smithi154 ceph-mon[36479]: pgmap v1085: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-02T17:36:16.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:15 smithi059 ceph-mon[31664]: pgmap v1085: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-02T17:36:16.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:15 smithi164 ceph-mon[36499]: pgmap v1085: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-02T17:36:16.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 418667 37.24 MB/sec execute 463 sec latency 71.595 ms 2024-04-02T17:36:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.937+0000 7f8f10400640 1 -- 172.21.15.59:0/1842965673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f081036e0 msgr2=0x7f8f08109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.937+0000 7f8f10400640 1 --2- 172.21.15.59:0/1842965673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f081036e0 0x7f8f08109f70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009ca0 tx=0x7f8f0002c100 comp rx=0 tx=0).stop 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 -- 172.21.15.59:0/1842965673 shutdown_connections 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 --2- 172.21.15.59:0/1842965673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08102d20 0x7f8f081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 --2- 172.21.15.59:0/1842965673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f081036e0 0x7f8f08109f70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 --2- 172.21.15.59:0/1842965673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f08101b30 0x7f8f08101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 -- 172.21.15.59:0/1842965673 >> 172.21.15.59:0/1842965673 conn(0x7f8f080fd0c0 msgr2=0x7f8f080ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.938+0000 7f8f10400640 1 -- 172.21.15.59:0/1842965673 shutdown_connections 2024-04-02T17:36:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.939+0000 7f8f10400640 1 -- 172.21.15.59:0/1842965673 wait complete. 2024-04-02T17:36:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.940+0000 7f8f10400640 1 Processor -- start 2024-04-02T17:36:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.941+0000 7f8f10400640 1 -- start start 2024-04-02T17:36:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08101b30 0x7f8f08070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 0x7f8f081a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810d790 con 0x7f8f08101b30 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810d610 con 0x7f8f081036e0 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f10400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810d910 con 0x7f8f08102d20 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36494/0 (socket says 172.21.15.59:36494) 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.942+0000 7f8f0d000640 1 -- 172.21.15.59:0/3501129794 learned_addr learned my addr 172.21.15.59:0/3501129794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 0x7f8f081a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0da00640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08101b30 0x7f8f08070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 -- 172.21.15.59:0/3501129794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 msgr2=0x7f8f080708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 -- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08101b30 msgr2=0x7f8f08070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08101b30 0x7f8f08070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.943+0000 7f8f0e400640 1 -- 172.21.15.59:0/3501129794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080739f0 con 0x7f8f081036e0 2024-04-02T17:36:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f0d000640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f0e400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 0x7f8f081a7ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f00037040 tx=0x7f8f00035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f06000640 1 -- 172.21.15.59:0/3501129794 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000328a0 con 0x7f8f081036e0 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f06000640 1 -- 172.21.15.59:0/3501129794 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00032a40 con 0x7f8f081036e0 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f06000640 1 -- 172.21.15.59:0/3501129794 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0003c7e0 con 0x7f8f081036e0 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f08073c80 con 0x7f8f081036e0 2024-04-02T17:36:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.944+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f08074130 con 0x7f8f081036e0 2024-04-02T17:36:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.948+0000 7f8f06000640 1 -- 172.21.15.59:0/3501129794 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f0004d060 con 0x7f8f081036e0 2024-04-02T17:36:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.952+0000 7f8f06000640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 0x7f8ef00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.952+0000 7f8f0da00640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 0x7f8ef00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.953+0000 7f8f131a8740 1 Processor -- start 2024-04-02T17:36:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.954+0000 7f8f06000640 1 -- 172.21.15.59:0/3501129794 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f8f000f7380 con 0x7f8f081036e0 2024-04-02T17:36:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.954+0000 7f8f0da00640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 0x7f8ef00a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f08102b80 tx=0x7f8efc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.956+0000 7f8f131a8740 1 -- start start 2024-04-02T17:36:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.956+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.956+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 0x5564748673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.956+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474869ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.957+0000 7f8f0da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.957+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 0x5564748673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.957+0000 7f8f0da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36506/0 (socket says 172.21.15.59:36506) 2024-04-02T17:36:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.957+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 0x5564748673c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48584/0 (socket says 172.21.15.59:48584) 2024-04-02T17:36:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f0da00640 1 -- 172.21.15.59:0/2379620999 learned_addr learned my addr 172.21.15.59:0/2379620999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5564747c7d70 con 0x5564748628c0 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5564747702a0 con 0x556474867a30 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5564747c3d20 con 0x556474861ed0 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f0e400640 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474869ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.958+0000 7f8f0da00640 1 -- 172.21.15.59:0/2379620999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 msgr2=0x556474869ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0da00640 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474869ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0da00640 1 -- 172.21.15.59:0/2379620999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 msgr2=0x5564748673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0da00640 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 0x5564748673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0da00640 1 -- 172.21.15.59:0/2379620999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556474784840 con 0x556474861ed0 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0e400640 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474869ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.959+0000 7f8f0da00640 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0058a0 tx=0x7f8efc013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b4fa91568af7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.960+0000 7f8efaa00640 1 -- 172.21.15.59:0/2379620999 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc004220 con 0x556474861ed0 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.960+0000 7f8efaa00640 1 -- 172.21.15.59:0/2379620999 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8efc0043c0 con 0x556474861ed0 2024-04-02T17:36:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.960+0000 7f8efaa00640 1 -- 172.21.15.59:0/2379620999 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc01e990 con 0x556474861ed0 2024-04-02T17:36:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 msgr2=0x5564748622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0058a0 tx=0x7f8efc013000 comp rx=0 tx=0).stop 2024-04-02T17:36:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 shutdown_connections 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564748628c0 0x5564748673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474869ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 --2- 172.21.15.59:0/2379620999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x5564748622f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 >> 172.21.15.59:0/2379620999 conn(0x55647485d440 msgr2=0x55647485d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 shutdown_connections 2024-04-02T17:36:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.962+0000 7f8f131a8740 1 -- 172.21.15.59:0/2379620999 wait complete. 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.963+0000 7f8f131a8740 1 -- start start 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.963+0000 7f8f131a8740 1 Processor -- start 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x55647486c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x5564748739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55647486a9e0 con 0x556474861ed0 2024-04-02T17:36:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556474783660 con 0x556474867a30 2024-04-02T17:36:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55647486ab60 con 0x5564748628c0 2024-04-02T17:36:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f131a8740 10 client.0 _ll_register_callbacks cb 0x556474783050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36522/0 (socket says 172.21.15.59:36522) 2024-04-02T17:36:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.964+0000 7f8f0d000640 1 -- 172.21.15.59:0/301187161 learned_addr learned my addr 172.21.15.59:0/301187161 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0e400640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x5564748739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0d000640 1 -- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 msgr2=0x5564748739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0d000640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x5564748739f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0d000640 1 -- 172.21.15.59:0/301187161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 msgr2=0x55647486c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0da00640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x55647486c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0d000640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x55647486c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0d000640 1 -- 172.21.15.59:0/301187161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556474792e80 con 0x5564748628c0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.965+0000 7f8f0da00640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x55647486c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.966+0000 7f8f0d000640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4010f10 tx=0x7f8ef400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c194286fc58815c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.966+0000 7f8eef400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.966+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4005020 con 0x5564748628c0 2024-04-02T17:36:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.966+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f8ef40051c0 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.967+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8efc0043c0 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.967+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8ef40053c0 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8f131a8740 10 client.27727 fetch_fsmap learned FSMap version 61 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5564744dc210 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8efc0043c0 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 61) ==== 8370+0+0 (secure 0 0 0) 0x7f8ef4019680 con 0x5564748628c0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8f131a8740 10 client.27727 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.968+0000 7f8f131a8740 10 client.27727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.969+0000 7f8f131a8740 20 client.27727 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.969+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474880bc0 0x556474883060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.969+0000 7f8f131a8740 4 client.27727 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.969+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556474777cd0 con 0x556474880bc0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.970+0000 7f8f0da00640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474880bc0 0x556474883060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.971+0000 7f8eef400640 10 client.27727 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.971+0000 7f8f0da00640 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474880bc0 0x556474883060 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577625cd3d807637 server_cookie=1d2c54021dcdefcd in_seq=0 out_seq=0 2024-04-02T17:36:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.976+0000 7f8eef400640 1 -- 172.21.15.59:0/301187161 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556474777cd0 con 0x556474880bc0 2024-04-02T17:36:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.976+0000 7f8eef400640 10 client.27727 handle_command_reply: tid=0 2024-04-02T17:36:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.976+0000 7f8f131a8740 1 client.27727 shutdown 2024-04-02T17:36:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.977+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 msgr2=0x55647486c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.977+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4010f10 tx=0x7f8ef400f7f0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 shutdown_connections 2024-04-02T17:36:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474880bc0 0x556474883060 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x5564748739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x55647486c9b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 --2- 172.21.15.59:0/301187161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x55647486c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 >> 172.21.15.59:0/301187161 conn(0x55647485d440 msgr2=0x55647485c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8eef400640 0 client.27727 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 shutdown_connections 2024-04-02T17:36:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.978+0000 7f8f131a8740 1 -- 172.21.15.59:0/301187161 wait complete. 2024-04-02T17:36:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:16.979+0000 7f8f131a8740 20 client.27727 trim_cache size 0 max 16384 2024-04-02T17:36:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.007+0000 7f8f131a8740 1 Processor -- start 2024-04-02T17:36:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.007+0000 7f8f131a8740 1 -- start start 2024-04-02T17:36:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x556474882bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474860db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081af6e0 con 0x556474861ed0 2024-04-02T17:36:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081b12e0 con 0x556474867a30 2024-04-02T17:36:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f131a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081afe70 con 0x5564748628c0 2024-04-02T17:36:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36526/0 (socket says 172.21.15.59:36526) 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.008+0000 7f8f0d000640 1 -- 172.21.15.59:0/821008473 learned_addr learned my addr 172.21.15.59:0/821008473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0da00640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x556474882bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0e400640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474860db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0d000640 1 -- 172.21.15.59:0/821008473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 msgr2=0x556474860db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0d000640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474860db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0d000640 1 -- 172.21.15.59:0/821008473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 msgr2=0x556474882bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0d000640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x556474882bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0d000640 1 -- 172.21.15.59:0/821008473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564745a8620 con 0x5564748628c0 2024-04-02T17:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0e400640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474860db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:36:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.009+0000 7f8f0da00640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x556474882bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:36:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.010+0000 7f8f0d000640 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400a670 tx=0x7f8ef400afc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=415096ac1e66db50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.011+0000 7f8efa000640 1 -- 172.21.15.59:0/821008473 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40193e0 con 0x5564748628c0 2024-04-02T17:36:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.011+0000 7f8efa000640 1 -- 172.21.15.59:0/821008473 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef40224a0 con 0x5564748628c0 2024-04-02T17:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.011+0000 7f8efa000640 1 -- 172.21.15.59:0/821008473 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4023420 con 0x5564748628c0 2024-04-02T17:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 -- 172.21.15.59:0/821008473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 msgr2=0x5564748739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400a670 tx=0x7f8ef400afc0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 -- 172.21.15.59:0/821008473 shutdown_connections 2024-04-02T17:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556474867a30 0x556474860db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564748628c0 0x5564748739b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 --2- 172.21.15.59:0/821008473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474861ed0 0x556474882bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 -- 172.21.15.59:0/821008473 >> 172.21.15.59:0/821008473 conn(0x55647485d440 msgr2=0x5564748606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 -- 172.21.15.59:0/821008473 shutdown_connections 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.012+0000 7f8f131a8740 1 -- 172.21.15.59:0/821008473 wait complete. 2024-04-02T17:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.013+0000 7f8f131a8740 1 -- start start 2024-04-02T17:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.013+0000 7f8f131a8740 1 Processor -- start 2024-04-02T17:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x55647486e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 0x55647486edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 0x556474875f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081b0c10 con 0x556474867a30 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081b0540 con 0x5564748628c0 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f081ad6c0 con 0x556474861ed0 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f131a8740 10 client.0 _ll_register_callbacks cb 0x7f8ef4010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 0x55647486edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f0d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 0x55647486edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:51534/0 (socket says 172.21.15.59:51534) 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f0d000640 1 -- 172.21.15.59:0/4269177370 learned_addr learned my addr 172.21.15.59:0/4269177370 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.015+0000 7f8f0da00640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x55647486e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0e400640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 0x556474875f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0da00640 1 -- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 msgr2=0x55647486edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0da00640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 0x55647486edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0da00640 1 -- 172.21.15.59:0/4269177370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 msgr2=0x556474875f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0da00640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 0x556474875f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0da00640 1 -- 172.21.15.59:0/4269177370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556474876460 con 0x556474861ed0 2024-04-02T17:36:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.016+0000 7f8f0e400640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 0x556474875f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:36:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.017+0000 7f8f0da00640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x55647486e870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8efc01e460 tx=0x7f8efc010be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18b30f50c4d4fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.017+0000 7f8eef400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.017+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0043e0 con 0x556474861ed0 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.017+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef40224a0 con 0x556474861ed0 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.017+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f8efc035420 con 0x556474861ed0 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.018+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8efc035620 con 0x556474861ed0 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.018+0000 7f8f131a8740 10 client.27733 fetch_fsmap learned FSMap version 61 2024-04-02T17:36:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.018+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556474877740 con 0x556474861ed0 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.018+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef40224a0 con 0x556474861ed0 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 61) ==== 8370+0+0 (secure 0 0 0) 0x7f8efc01e490 con 0x556474861ed0 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 10 client.27733 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 10 client.27733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 20 client.27733 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474877fb0 0x55647487a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 4 client.27733 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:36:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564747d52b0 con 0x556474877fb0 2024-04-02T17:36:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.019+0000 7f8f0d000640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474877fb0 0x55647487a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.020+0000 7f8f0d000640 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474877fb0 0x55647487a3d0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3ea3256198af2c server_cookie=c735564c609732cb in_seq=0 out_seq=0 2024-04-02T17:36:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.020+0000 7f8eef400640 10 client.27733 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.291+0000 7f8eef400640 1 -- 172.21.15.59:0/4269177370 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564747d52b0 con 0x556474877fb0 2024-04-02T17:36:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.291+0000 7f8eef400640 10 client.27733 handle_command_reply: tid=0 2024-04-02T17:36:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.291+0000 7f8f131a8740 1 client.27733 shutdown 2024-04-02T17:36:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 msgr2=0x55647486e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x55647486e870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8efc01e460 tx=0x7f8efc010be0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 shutdown_connections 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x556474877fb0 0x55647487a3d0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556474867a30 0x556474875f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564748628c0 0x55647486edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 --2- 172.21.15.59:0/4269177370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556474861ed0 0x55647486e870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 >> 172.21.15.59:0/4269177370 conn(0x55647485d440 msgr2=0x556474860f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8eef400640 0 client.27733 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 shutdown_connections 2024-04-02T17:36:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.292+0000 7f8f131a8740 1 -- 172.21.15.59:0/4269177370 wait complete. 2024-04-02T17:36:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.293+0000 7f8f131a8740 20 client.27733 trim_cache size 0 max 16384 2024-04-02T17:36:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:36:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:36:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:36:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:36:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 msgr2=0x7f8ef00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 0x7f8ef00a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f08102b80 tx=0x7f8efc00b390 comp rx=0 tx=0).stop 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 msgr2=0x7f8f081a7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 0x7f8f081a7ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8f00037040 tx=0x7f8f00035d10 comp rx=0 tx=0).stop 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 shutdown_connections 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f081036e0 0x7f8f081a7ea0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f08102d20 0x7f8f080708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f8ef00a3d20 0x7f8ef00a61e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 --2- 172.21.15.59:0/3501129794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f08101b30 0x7f8f08070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 >> 172.21.15.59:0/3501129794 conn(0x7f8f080fd0c0 msgr2=0x7f8f08105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 shutdown_connections 2024-04-02T17:36:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:17.294+0000 7f8f10400640 1 -- 172.21.15.59:0/3501129794 wait complete. 2024-04-02T17:36:17.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 419110 37.22 MB/sec execute 464 sec latency 205.681 ms 2024-04-02T17:36:17.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:17 smithi154 ceph-mon[36479]: pgmap v1086: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 811 op/s 2024-04-02T17:36:18.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:17 smithi059 ceph-mon[31664]: pgmap v1086: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 811 op/s 2024-04-02T17:36:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:17 smithi164 ceph-mon[36499]: pgmap v1086: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 811 op/s 2024-04-02T17:36:18.660 INFO:tasks.workunit.client.0.smithi059.stdout: 10 419922 37.22 MB/sec execute 465 sec latency 61.352 ms 2024-04-02T17:36:19.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 420710 37.25 MB/sec execute 466 sec latency 54.315 ms 2024-04-02T17:36:19.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:19 smithi154 ceph-mon[36479]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-04-02T17:36:20.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:19 smithi059 ceph-mon[31664]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-04-02T17:36:20.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:19 smithi164 ceph-mon[36499]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-04-02T17:36:20.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 421294 37.23 MB/sec execute 467 sec latency 82.210 ms 2024-04-02T17:36:21.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 421912 37.22 MB/sec execute 468 sec latency 59.351 ms 2024-04-02T17:36:21.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:21 smithi154 ceph-mon[36479]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 749 op/s 2024-04-02T17:36:22.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:21 smithi059 ceph-mon[31664]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 749 op/s 2024-04-02T17:36:22.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:21 smithi164 ceph-mon[36499]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 749 op/s 2024-04-02T17:36:22.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 422700 37.22 MB/sec execute 469 sec latency 61.734 ms 2024-04-02T17:36:23.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 423363 37.23 MB/sec execute 470 sec latency 279.418 ms 2024-04-02T17:36:23.899 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:23 smithi154 ceph-mon[36479]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-04-02T17:36:24.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:23 smithi059 ceph-mon[31664]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-04-02T17:36:24.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:23 smithi164 ceph-mon[36499]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-04-02T17:36:24.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 424019 37.24 MB/sec execute 471 sec latency 60.593 ms 2024-04-02T17:36:25.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 424569 37.22 MB/sec execute 472 sec latency 68.836 ms 2024-04-02T17:36:26.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:25 smithi059 ceph-mon[31664]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.5 MiB/s wr, 663 op/s 2024-04-02T17:36:26.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:25 smithi164 ceph-mon[36499]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.5 MiB/s wr, 663 op/s 2024-04-02T17:36:26.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:25 smithi154 ceph-mon[36479]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.5 MiB/s wr, 663 op/s 2024-04-02T17:36:26.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 425190 37.20 MB/sec execute 473 sec latency 64.115 ms 2024-04-02T17:36:27.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 425999 37.21 MB/sec execute 474 sec latency 49.429 ms 2024-04-02T17:36:28.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:27 smithi059 ceph-mon[31664]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.2 MiB/s wr, 646 op/s 2024-04-02T17:36:28.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:27 smithi164 ceph-mon[36499]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.2 MiB/s wr, 646 op/s 2024-04-02T17:36:28.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:27 smithi154 ceph-mon[36479]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.2 MiB/s wr, 646 op/s 2024-04-02T17:36:28.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 426796 37.23 MB/sec execute 475 sec latency 64.749 ms 2024-04-02T17:36:29.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 427472 37.23 MB/sec execute 476 sec latency 59.363 ms 2024-04-02T17:36:30.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:29 smithi059 ceph-mon[31664]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 485 op/s 2024-04-02T17:36:30.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:29 smithi164 ceph-mon[36499]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 485 op/s 2024-04-02T17:36:30.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:29 smithi154 ceph-mon[36479]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 485 op/s 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 -- 172.21.15.59:0/136170906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 msgr2=0x7f319009e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 --2- 172.21.15.59:0/136170906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f319009e1c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3198067240 tx=0x7f3198097f80 comp rx=0 tx=0).stop 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 -- 172.21.15.59:0/136170906 shutdown_connections 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 --2- 172.21.15.59:0/136170906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f319009e1c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 --2- 172.21.15.59:0/136170906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f31900973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 --2- 172.21.15.59:0/136170906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 0x7f31900961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 -- 172.21.15.59:0/136170906 >> 172.21.15.59:0/136170906 conn(0x7f31900912d0 msgr2=0x7f3190093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 -- 172.21.15.59:0/136170906 shutdown_connections 2024-04-02T17:36:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.514+0000 7f319d400640 1 -- 172.21.15.59:0/136170906 wait complete. 2024-04-02T17:36:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.516+0000 7f319d400640 1 Processor -- start 2024-04-02T17:36:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.516+0000 7f319d400640 1 -- start start 2024-04-02T17:36:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 0x7f319012bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f319012c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f3190133560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31900a1910 con 0x7f3190095d80 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f31900a1790 con 0x7f3190096f70 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319d400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31900a1a90 con 0x7f3190097930 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f3190133560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f3190133560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35254/0 (socket says 172.21.15.59:35254) 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.517+0000 7f319ca00640 1 -- 172.21.15.59:0/3796939318 learned_addr learned my addr 172.21.15.59:0/3796939318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.518+0000 7f3197400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f319012c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.519+0000 7f3197400640 1 -- 172.21.15.59:0/3796939318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 msgr2=0x7f3190133560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.519+0000 7f3197400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f3190133560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.519+0000 7f3197400640 1 -- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 msgr2=0x7f319012bfb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:36:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.519+0000 7f3197e00640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 0x7f319012bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.520+0000 7f3197400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 0x7f319012bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.520+0000 7f3197400640 1 -- 172.21.15.59:0/3796939318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3190144be0 con 0x7f3190096f70 2024-04-02T17:36:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.520+0000 7f3197400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f319012c4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.522+0000 7f3194c00640 1 -- 172.21.15.59:0/3796939318 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400cd80 con 0x7f3190096f70 2024-04-02T17:36:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.522+0000 7f3194c00640 1 -- 172.21.15.59:0/3796939318 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840035f0 con 0x7f3190096f70 2024-04-02T17:36:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.522+0000 7f3194c00640 1 -- 172.21.15.59:0/3796939318 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184010690 con 0x7f3190096f70 2024-04-02T17:36:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.522+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3190144ed0 con 0x7f3190096f70 2024-04-02T17:36:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.523+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3190145340 con 0x7f3190096f70 2024-04-02T17:36:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.530+0000 7f3194c00640 1 -- 172.21.15.59:0/3796939318 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3184004020 con 0x7f3190096f70 2024-04-02T17:36:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.531+0000 7f31a17d2740 1 Processor -- start 2024-04-02T17:36:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 -- start start 2024-04-02T17:36:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac00375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac003a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562abff97d70 con 0x562ac0032100 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562abff402a0 con 0x562ac0037c60 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.532+0000 7f31a17d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562abff93d20 con 0x562ac0032af0 2024-04-02T17:36:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.533+0000 7f319ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.533+0000 7f3197e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac00375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.533+0000 7f319ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57158/0 (socket says 172.21.15.59:57158) 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.533+0000 7f319ca00640 1 -- 172.21.15.59:0/3153249245 learned_addr learned my addr 172.21.15.59:0/3153249245 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.533+0000 7f3197400640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac003a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 -- 172.21.15.59:0/3153249245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 msgr2=0x562ac00375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac00375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 -- 172.21.15.59:0/3153249245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 msgr2=0x562ac003a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac003a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 -- 172.21.15.59:0/3153249245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abff54840 con 0x562ac0032100 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f3197e00640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac00375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f319ca00640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3198067240 tx=0x7f3198067c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bbc91cee3352cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.534+0000 7f3197400640 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac003a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.535+0000 7f3194c00640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 0x7f31800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.535+0000 7f3189200640 1 -- 172.21.15.59:0/3153249245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198098920 con 0x562ac0032100 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.536+0000 7f3189200640 1 -- 172.21.15.59:0/3153249245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31980684a0 con 0x562ac0032100 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.536+0000 7f3197e00640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 0x7f31800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.536+0000 7f3189200640 1 -- 172.21.15.59:0/3153249245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198073690 con 0x562ac0032100 2024-04-02T17:36:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.536+0000 7f3194c00640 1 -- 172.21.15.59:0/3796939318 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f31840d06a0 con 0x7f3190096f70 2024-04-02T17:36:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.537+0000 7f31a17d2740 1 -- 172.21.15.59:0/3153249245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 msgr2=0x562ac0032520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.537+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3198067240 tx=0x7f3198067c20 comp rx=0 tx=0).stop 2024-04-02T17:36:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.538+0000 7f31a17d2740 1 -- 172.21.15.59:0/3153249245 shutdown_connections 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.538+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac003a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.538+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac00375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.538+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3153249245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0032520 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.538+0000 7f31a17d2740 1 -- 172.21.15.59:0/3153249245 >> 172.21.15.59:0/3153249245 conn(0x562ac002d670 msgr2=0x562ac002da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.539+0000 7f31a17d2740 1 -- 172.21.15.59:0/3153249245 shutdown_connections 2024-04-02T17:36:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.539+0000 7f31a17d2740 1 -- 172.21.15.59:0/3153249245 wait complete. 2024-04-02T17:36:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.541+0000 7f31a17d2740 1 -- start start 2024-04-02T17:36:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.542+0000 7f31a17d2740 1 Processor -- start 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.542+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 0x562ac0035360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.542+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 0x562ac00358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.542+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0037c60 0x562ac00368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.542+0000 7f31a17d2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ac003aa30 con 0x562ac0037c60 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.543+0000 7f31a17d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562abff53660 con 0x562ac0032af0 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.543+0000 7f31a17d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ac003abb0 con 0x562ac0032100 2024-04-02T17:36:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.543+0000 7f31a17d2740 10 client.0 _ll_register_callbacks cb 0x562abff53050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.543+0000 7f3197e00640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 0x7f31800a6270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x562ac002e5e0 tx=0x7f318c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.544+0000 7f3197e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 0x562ac0035360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.544+0000 7f3197e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 0x562ac0035360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35276/0 (socket says 172.21.15.59:35276) 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.544+0000 7f3197e00640 1 -- 172.21.15.59:0/849094504 learned_addr learned my addr 172.21.15.59:0/849094504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.544+0000 7f3197400640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 0x562ac00358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.544+0000 7f319ca00640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0037c60 0x562ac00368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.545+0000 7f3197400640 1 -- 172.21.15.59:0/849094504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 msgr2=0x562ac0035360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.545+0000 7f3197400640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 0x562ac0035360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.545+0000 7f3197400640 1 -- 172.21.15.59:0/849094504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0037c60 msgr2=0x562ac00368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.545+0000 7f3197400640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0037c60 0x562ac00368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.546+0000 7f3197400640 1 -- 172.21.15.59:0/849094504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abff62e80 con 0x562ac0032af0 2024-04-02T17:36:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.547+0000 7f3197400640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 0x562ac00358a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f318409c850 tx=0x7f318409c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7a6ecde219dd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.547+0000 7f317d600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:36:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.547+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318409c8c0 con 0x562ac0032af0 2024-04-02T17:36:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.547+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31980684a0 con 0x562ac0032af0 2024-04-02T17:36:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.547+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f31840037a0 con 0x562ac0032af0 2024-04-02T17:36:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.549+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f31840a0c70 con 0x562ac0032af0 2024-04-02T17:36:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.549+0000 7f31a17d2740 10 client.27683 fetch_fsmap learned FSMap version 61 2024-04-02T17:36:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.549+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562abfcac210 con 0x562ac0032af0 2024-04-02T17:36:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.551+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840a5530 con 0x562ac0032af0 2024-04-02T17:36:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.552+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 61) ==== 8370+0+0 (secure 0 0 0) 0x7f318409bd10 con 0x562ac0032af0 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.552+0000 7f31a17d2740 10 client.27683 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.552+0000 7f31a17d2740 10 client.27683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.552+0000 7f31a17d2740 20 client.27683 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.553+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac0048430 0x562ac004a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.553+0000 7f31a17d2740 4 client.27683 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:36:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.553+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31980684a0 con 0x562ac0048430 2024-04-02T17:36:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.553+0000 7f3197e00640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac0048430 0x562ac004a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.556+0000 7f3197e00640 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac0048430 0x562ac004a890 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363714ef5ba356a4 server_cookie=dca172a2e457feeb in_seq=0 out_seq=0 2024-04-02T17:36:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.556+0000 7f317d600640 10 client.27683 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.558+0000 7f317d600640 1 -- 172.21.15.59:0/849094504 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f31980684a0 con 0x562ac0048430 2024-04-02T17:36:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.558+0000 7f317d600640 10 client.27683 handle_command_reply: tid=0 2024-04-02T17:36:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.558+0000 7f31a17d2740 1 client.27683 shutdown 2024-04-02T17:36:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 msgr2=0x562ac00358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 0x562ac00358a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f318409c850 tx=0x7f318409c790 comp rx=0 tx=0).stop 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 shutdown_connections 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac0048430 0x562ac004a890 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0037c60 0x562ac00368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032af0 0x562ac00358a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 --2- 172.21.15.59:0/849094504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032100 0x562ac0035360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f317d600640 0 client.27683 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 >> 172.21.15.59:0/849094504 conn(0x562ac002d670 msgr2=0x562ac002c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 shutdown_connections 2024-04-02T17:36:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 1 -- 172.21.15.59:0/849094504 wait complete. 2024-04-02T17:36:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.559+0000 7f31a17d2740 20 client.27683 trim_cache size 0 max 16384 2024-04-02T17:36:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.592+0000 7f31a17d2740 1 Processor -- start 2024-04-02T17:36:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 -- start start 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0034d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac002bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac002c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f319014bef0 con 0x562ac0032100 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319014daf0 con 0x562ac0037c60 2024-04-02T17:36:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.593+0000 7f31a17d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319014c680 con 0x562ac0032af0 2024-04-02T17:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.594+0000 7f3197400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac002bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.594+0000 7f3197400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac002bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35284/0 (socket says 172.21.15.59:35284) 2024-04-02T17:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.594+0000 7f3197400640 1 -- 172.21.15.59:0/3630272615 learned_addr learned my addr 172.21.15.59:0/3630272615 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.594+0000 7f3197e00640 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0034d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.594+0000 7f319ca00640 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac002c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.595+0000 7f3197e00640 1 -- 172.21.15.59:0/3630272615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 msgr2=0x562ac002bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.595+0000 7f3197e00640 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac002bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.595+0000 7f3197e00640 1 -- 172.21.15.59:0/3630272615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 msgr2=0x562ac002c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.595+0000 7f3197e00640 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac002c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.595+0000 7f3197e00640 1 -- 172.21.15.59:0/3630272615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abfd78620 con 0x562ac0032100 2024-04-02T17:36:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.596+0000 7f3197e00640 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0034d60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x562ac004ae90 tx=0x7f318c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5f1cb14b30904b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.596+0000 7f317fe00640 1 -- 172.21.15.59:0/3630272615 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c028670 con 0x562ac0032100 2024-04-02T17:36:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.596+0000 7f317fe00640 1 -- 172.21.15.59:0/3630272615 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c02d070 con 0x562ac0032100 2024-04-02T17:36:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.596+0000 7f317fe00640 1 -- 172.21.15.59:0/3630272615 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c010660 con 0x562ac0032100 2024-04-02T17:36:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.598+0000 7f31a17d2740 1 -- 172.21.15.59:0/3630272615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 msgr2=0x562ac0034d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.598+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0034d60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x562ac004ae90 tx=0x7f318c015000 comp rx=0 tx=0).stop 2024-04-02T17:36:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 -- 172.21.15.59:0/3630272615 shutdown_connections 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0037c60 0x562ac002c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0032af0 0x562ac002bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 --2- 172.21.15.59:0/3630272615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032100 0x562ac0034d60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 -- 172.21.15.59:0/3630272615 >> 172.21.15.59:0/3630272615 conn(0x562ac002d670 msgr2=0x562ac0034650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 -- 172.21.15.59:0/3630272615 shutdown_connections 2024-04-02T17:36:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.599+0000 7f31a17d2740 1 -- 172.21.15.59:0/3630272615 wait complete. 2024-04-02T17:36:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.600+0000 7f31a17d2740 1 -- start start 2024-04-02T17:36:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.600+0000 7f31a17d2740 1 Processor -- start 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.601+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 0x562ac00420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.601+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 0x562ac0042600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f31a17d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0037c60 0x562ac0043610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f31a17d2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f319014d420 con 0x562ac0032af0 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f31a17d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319014cd50 con 0x562ac0032100 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f31a17d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3190149f80 con 0x562ac0037c60 2024-04-02T17:36:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f31a17d2740 10 client.0 _ll_register_callbacks cb 0x7f31840d3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f3197e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 0x562ac00420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f3197e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 0x562ac00420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:57640/0 (socket says 172.21.15.59:57640) 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f3197e00640 1 -- 172.21.15.59:0/1387413697 learned_addr learned my addr 172.21.15.59:0/1387413697 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.602+0000 7f319ca00640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0037c60 0x562ac0043610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.603+0000 7f3197400640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 0x562ac0042600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 -- 172.21.15.59:0/1387413697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0037c60 msgr2=0x562ac0043610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0037c60 0x562ac0043610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 -- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 msgr2=0x562ac00420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 0x562ac00420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 -- 172.21.15.59:0/1387413697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac0043c00 con 0x562ac0032af0 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f3197400640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 0x562ac0042600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3184098d20 tx=0x7f318409d080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f601ae5f78bb1495 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.604+0000 7f317d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:36:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.606+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318409c8c0 con 0x562ac0032af0 2024-04-02T17:36:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.606+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c02d070 con 0x562ac0032af0 2024-04-02T17:36:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.608+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f3184098f30 con 0x562ac0032af0 2024-04-02T17:36:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.609+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f31840d0110 con 0x562ac0032af0 2024-04-02T17:36:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.609+0000 7f31a17d2740 10 client.17526 fetch_fsmap learned FSMap version 61 2024-04-02T17:36:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.609+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562ac00af190 con 0x562ac0032af0 2024-04-02T17:36:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.610+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c02d070 con 0x562ac0032af0 2024-04-02T17:36:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.611+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 61) ==== 8370+0+0 (secure 0 0 0) 0x7f31840a1e70 con 0x562ac0032af0 2024-04-02T17:36:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.611+0000 7f31a17d2740 10 client.17526 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T17:36:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.611+0000 7f31a17d2740 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.611+0000 7f31a17d2740 20 client.17526 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.611+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac00afa00 0x562ac00b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.612+0000 7f31a17d2740 4 client.17526 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:36:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.612+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562abff47cd0 con 0x562ac00afa00 2024-04-02T17:36:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.612+0000 7f3197e00640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac00afa00 0x562ac00b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.614+0000 7f3197e00640 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac00afa00 0x562ac00b1e20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad9578d9aa26d3f server_cookie=3aa5c855497068ac in_seq=0 out_seq=0 2024-04-02T17:36:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.614+0000 7f317d600640 10 client.17526 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:30.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 428188 37.23 MB/sec execute 477 sec latency 65.843 ms 2024-04-02T17:36:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.961+0000 7f317d600640 1 -- 172.21.15.59:0/1387413697 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562abff47cd0 con 0x562ac00afa00 2024-04-02T17:36:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.961+0000 7f317d600640 10 client.17526 handle_command_reply: tid=0 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.961+0000 7f31a17d2740 1 client.17526 shutdown 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 msgr2=0x562ac0042600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 0x562ac0042600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3184098d20 tx=0x7f318409d080 comp rx=0 tx=0).stop 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 shutdown_connections 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x562ac00afa00 0x562ac00b1e20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ac0032af0 0x562ac0042600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ac0037c60 0x562ac0043610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 --2- 172.21.15.59:0/1387413697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ac0032100 0x562ac00420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 >> 172.21.15.59:0/1387413697 conn(0x562ac002d670 msgr2=0x562ac00308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f317d600640 0 client.17526 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 shutdown_connections 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.962+0000 7f31a17d2740 1 -- 172.21.15.59:0/1387413697 wait complete. 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.963+0000 7f31a17d2740 20 client.17526 trim_cache size 0 max 16384 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:36:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.963+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 msgr2=0x7f31800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.963+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 0x7f31800a6270 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x562ac002e5e0 tx=0x7f318c005f90 comp rx=0 tx=0).stop 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.963+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 msgr2=0x7f319012c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.963+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f319012c4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 shutdown_connections 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3190097930 0x7f3190133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3190096f70 0x7f319012c4f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31800a3db0 0x7f31800a6270 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 --2- 172.21.15.59:0/3796939318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3190095d80 0x7f319012bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 >> 172.21.15.59:0/3796939318 conn(0x7f31900912d0 msgr2=0x7f319009a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 shutdown_connections 2024-04-02T17:36:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:30.964+0000 7f319d400640 1 -- 172.21.15.59:0/3796939318 wait complete. 2024-04-02T17:36:31.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 428624 37.22 MB/sec execute 478 sec latency 220.258 ms 2024-04-02T17:36:32.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:31 smithi059 ceph-mon[31664]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-04-02T17:36:32.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:31 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:32.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:31 smithi164 ceph-mon[36499]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-04-02T17:36:32.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:31 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:32.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:31 smithi154 ceph-mon[36479]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-04-02T17:36:32.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:31 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:32.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 429278 37.19 MB/sec execute 479 sec latency 70.057 ms 2024-04-02T17:36:33.661 INFO:tasks.workunit.client.0.smithi059.stdout: 10 430012 37.21 MB/sec execute 480 sec latency 69.443 ms 2024-04-02T17:36:34.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:33 smithi059 ceph-mon[31664]: pgmap v1094: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.0 MiB/s wr, 543 op/s 2024-04-02T17:36:34.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:33 smithi164 ceph-mon[36499]: pgmap v1094: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.0 MiB/s wr, 543 op/s 2024-04-02T17:36:34.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:33 smithi154 ceph-mon[36479]: pgmap v1094: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.0 MiB/s wr, 543 op/s 2024-04-02T17:36:34.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 430719 37.19 MB/sec execute 481 sec latency 63.426 ms 2024-04-02T17:36:35.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 431374 37.19 MB/sec execute 482 sec latency 122.264 ms 2024-04-02T17:36:36.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:35 smithi059 ceph-mon[31664]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:36:36.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:35 smithi164 ceph-mon[36499]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:36:36.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:35 smithi154 ceph-mon[36479]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.0 MiB/s wr, 561 op/s 2024-04-02T17:36:36.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 432229 37.21 MB/sec execute 483 sec latency 52.088 ms 2024-04-02T17:36:37.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:36 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:37.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:36 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:37.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:36 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:36:37.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 432974 37.23 MB/sec execute 484 sec latency 55.843 ms 2024-04-02T17:36:38.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:37 smithi059 ceph-mon[31664]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.5 MiB/s wr, 605 op/s 2024-04-02T17:36:38.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:37 smithi164 ceph-mon[36499]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.5 MiB/s wr, 605 op/s 2024-04-02T17:36:38.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:37 smithi154 ceph-mon[36479]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.5 MiB/s wr, 605 op/s 2024-04-02T17:36:38.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 433521 37.22 MB/sec execute 485 sec latency 220.496 ms 2024-04-02T17:36:39.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 434012 37.22 MB/sec execute 486 sec latency 62.495 ms 2024-04-02T17:36:39.951 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:39 smithi154 ceph-mon[36479]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 525 op/s 2024-04-02T17:36:40.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:39 smithi059 ceph-mon[31664]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 525 op/s 2024-04-02T17:36:40.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:39 smithi164 ceph-mon[36499]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 525 op/s 2024-04-02T17:36:40.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 434180 37.17 MB/sec execute 487 sec latency 653.412 ms 2024-04-02T17:36:41.663 INFO:tasks.workunit.client.0.smithi059.stdout: 10 434317 37.10 MB/sec execute 488 sec latency 1068.977 ms 2024-04-02T17:36:42.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:41 smithi059 ceph-mon[31664]: pgmap v1098: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-04-02T17:36:42.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:41 smithi164 ceph-mon[36499]: pgmap v1098: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-04-02T17:36:42.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:41 smithi154 ceph-mon[36479]: pgmap v1098: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-04-02T17:36:42.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 434634 37.06 MB/sec execute 489 sec latency 137.160 ms 2024-04-02T17:36:43.662 INFO:tasks.workunit.client.0.smithi059.stdout: 10 435433 37.07 MB/sec execute 490 sec latency 126.782 ms 2024-04-02T17:36:44.048 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:43 smithi059 ceph-mon[31664]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-04-02T17:36:44.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:43 smithi164 ceph-mon[36499]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.145+0000 7fa7aa800640 1 -- 172.21.15.59:0/2642475511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 msgr2=0x7fa7a4104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.145+0000 7fa7aa800640 1 --2- 172.21.15.59:0/2642475511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a4104180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa794009900 tx=0x7fa79402bc30 comp rx=0 tx=0).stop 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 -- 172.21.15.59:0/2642475511 shutdown_connections 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 --2- 172.21.15.59:0/2642475511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 0x7fa7a410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 --2- 172.21.15.59:0/2642475511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 0x7fa7a41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 --2- 172.21.15.59:0/2642475511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a4104180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 -- 172.21.15.59:0/2642475511 >> 172.21.15.59:0/2642475511 conn(0x7fa7a40ff2f0 msgr2=0x7fa7a4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 -- 172.21.15.59:0/2642475511 shutdown_connections 2024-04-02T17:36:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.146+0000 7fa7aa800640 1 -- 172.21.15.59:0/2642475511 wait complete. 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 Processor -- start 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 -- start start 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 0x7fa7a419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 0x7fa7a41a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410fa00 con 0x7fa7a4105910 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410f880 con 0x7fa7a4104f50 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.147+0000 7fa7aa800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410fb80 con 0x7fa7a4103d60 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 0x7fa7a419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35246/0 (socket says 172.21.15.59:35246) 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 -- 172.21.15.59:0/1320324711 learned_addr learned my addr 172.21.15.59:0/1320324711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 0x7fa7a41a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 -- 172.21.15.59:0/1320324711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 msgr2=0x7fa7a419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 0x7fa7a419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 -- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 msgr2=0x7fa7a41a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 0x7fa7a41a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.148+0000 7fa7a3400640 1 -- 172.21.15.59:0/1320324711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a41a1c60 con 0x7fa7a4103d60 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.149+0000 7fa7a3400640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa79402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.149+0000 7fa79be00640 1 -- 172.21.15.59:0/1320324711 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794043020 con 0x7fa7a4103d60 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.149+0000 7fa79be00640 1 -- 172.21.15.59:0/1320324711 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7940054e0 con 0x7fa7a4103d60 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.149+0000 7fa79be00640 1 -- 172.21.15.59:0/1320324711 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794003c40 con 0x7fa7a4103d60 2024-04-02T17:36:44.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:43 smithi154 ceph-mon[36479]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-04-02T17:36:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.150+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a41a1ef0 con 0x7fa7a4103d60 2024-04-02T17:36:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.150+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a41a22a0 con 0x7fa7a4103d60 2024-04-02T17:36:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.153+0000 7fa79be00640 1 -- 172.21.15.59:0/1320324711 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa794046020 con 0x7fa7a4103d60 2024-04-02T17:36:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.154+0000 7fa7ad570740 1 Processor -- start 2024-04-02T17:36:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.154+0000 7fa7ad570740 1 -- start start 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a430d0 0x5626e5a47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626e599fd70 con 0x5626e5a430d0 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626e59482a0 con 0x5626e5a426e0 2024-04-02T17:36:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa7ad570740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626e599bd20 con 0x5626e5a48240 2024-04-02T17:36:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.155+0000 7fa79be00640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 0x7fa7880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40368/0 (socket says 172.21.15.59:40368) 2024-04-02T17:36:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa79be00640 1 -- 172.21.15.59:0/1320324711 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa7940f2490 con 0x7fa7a4103d60 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a430d0 0x5626e5a47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 -- 172.21.15.59:0/4021690376 learned_addr learned my addr 172.21.15.59:0/4021690376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a4a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:35254/0 (socket says 172.21.15.59:35254) 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 0x7fa7880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 -- 172.21.15.59:0/4021690376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 msgr2=0x5626e5a4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 -- 172.21.15.59:0/4021690376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a430d0 msgr2=0x5626e5a47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a430d0 0x5626e5a47bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.156+0000 7fa7a3e00640 1 -- 172.21.15.59:0/4021690376 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e595c840 con 0x5626e5a426e0 2024-04-02T17:36:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.157+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0099e0 tx=0x7fa79c011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb60f2c9667ed4e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.157+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 0x7fa7880a6190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa79c01d000 tx=0x7fa79c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.157+0000 7fa798c00640 1 -- 172.21.15.59:0/4021690376 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c003970 con 0x5626e5a426e0 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.157+0000 7fa798c00640 1 -- 172.21.15.59:0/4021690376 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c022020 con 0x5626e5a426e0 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.157+0000 7fa798c00640 1 -- 172.21.15.59:0/4021690376 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004050 con 0x5626e5a426e0 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.158+0000 7fa7ad570740 1 -- 172.21.15.59:0/4021690376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 msgr2=0x5626e5a42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.158+0000 7fa7ad570740 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0099e0 tx=0x7fa79c011d80 comp rx=0 tx=0).stop 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 -- 172.21.15.59:0/4021690376 shutdown_connections 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a430d0 0x5626e5a47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 --2- 172.21.15.59:0/4021690376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a426e0 0x5626e5a42b00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 -- 172.21.15.59:0/4021690376 >> 172.21.15.59:0/4021690376 conn(0x5626e5a3dc50 msgr2=0x5626e5a3e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 -- 172.21.15.59:0/4021690376 shutdown_connections 2024-04-02T17:36:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.159+0000 7fa7ad570740 1 -- 172.21.15.59:0/4021690376 wait complete. 2024-04-02T17:36:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.160+0000 7fa7ad570740 1 -- start start 2024-04-02T17:36:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.160+0000 7fa7ad570740 1 Processor -- start 2024-04-02T17:36:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.160+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5626e5a4b190 con 0x5626e5a426e0 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5626e595b660 con 0x5626e5a430d0 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626e5a4b310 con 0x5626e5a48240 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7ad570740 10 client.0 _ll_register_callbacks cb 0x5626e595b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49116/0 (socket says 172.21.15.59:49116) 2024-04-02T17:36:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7a3400640 1 -- 172.21.15.59:0/31107986 learned_addr learned my addr 172.21.15.59:0/31107986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.161+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 -- 172.21.15.59:0/31107986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 msgr2=0x5626e5a541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 -- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 msgr2=0x5626e5a4d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 -- 172.21.15.59:0/31107986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e596ae80 con 0x5626e5a426e0 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa7a3400640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7940bb080 tx=0x7fa7940be010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b195bc17f48b6a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.162+0000 7fa78d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940d7020 con 0x5626e5a426e0 2024-04-02T17:36:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa7940bc9b0 con 0x5626e5a426e0 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79c022020 con 0x5626e5a426e0 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa794005690 con 0x5626e5a426e0 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa7ad570740 10 client.17538 fetch_fsmap learned FSMap version 63 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.163+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5626e56b4210 con 0x5626e5a426e0 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c022020 con 0x5626e5a426e0 2024-04-02T17:36:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7fa7940e8b10 con 0x5626e5a426e0 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 10 client.17538 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 20 client.17538 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5a58b40 0x5626e5a5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 4 client.17538 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.164+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e594fcd0 con 0x5626e5a58b40 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.165+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5a58b40 0x5626e5a5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.165+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5a58b40 0x5626e5a5afe0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6822d747ee40981 server_cookie=eb46edc34e2f46d6 in_seq=0 out_seq=0 2024-04-02T17:36:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.165+0000 7fa78d600640 10 client.17538 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.167+0000 7fa78d600640 1 -- 172.21.15.59:0/31107986 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626e594fcd0 con 0x5626e5a58b40 2024-04-02T17:36:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.167+0000 7fa78d600640 10 client.17538 handle_command_reply: tid=0 2024-04-02T17:36:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.167+0000 7fa7ad570740 1 client.17538 shutdown 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.167+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 msgr2=0x5626e5a4cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.167+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7940bb080 tx=0x7fa7940be010 comp rx=0 tx=0).stop 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 shutdown_connections 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5a58b40 0x5626e5a5afe0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 --2- 172.21.15.59:0/31107986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a426e0 0x5626e5a4cc60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 >> 172.21.15.59:0/31107986 conn(0x5626e5a3dc50 msgr2=0x5626e5a34640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa78d600640 0 client.17538 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 shutdown_connections 2024-04-02T17:36:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 1 -- 172.21.15.59:0/31107986 wait complete. 2024-04-02T17:36:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.168+0000 7fa7ad570740 20 client.17538 trim_cache size 0 max 16384 2024-04-02T17:36:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.191+0000 7fa7ad570740 1 Processor -- start 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 -- start start 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a7080 con 0x5626e5a3e490 2024-04-02T17:36:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a8d40 con 0x5626e5a430d0 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7ad570740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a7810 con 0x5626e5a48240 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.192+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49118/0 (socket says 172.21.15.59:49118) 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 -- 172.21.15.59:0/2112953580 learned_addr learned my addr 172.21.15.59:0/2112953580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3400640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 -- 172.21.15.59:0/2112953580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 msgr2=0x5626e5a54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 -- 172.21.15.59:0/2112953580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 msgr2=0x5626e5a4cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.193+0000 7fa7a3e00640 1 -- 172.21.15.59:0/2112953580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e5780620 con 0x5626e5a3e490 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.194+0000 7fa7a3400640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.194+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.194+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0099e0 tx=0x7fa79c00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b23352ddc70d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.195+0000 7fa78fe00640 1 -- 172.21.15.59:0/2112953580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c01eb10 con 0x5626e5a3e490 2024-04-02T17:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.195+0000 7fa78fe00640 1 -- 172.21.15.59:0/2112953580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c02b040 con 0x5626e5a3e490 2024-04-02T17:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.195+0000 7fa78fe00640 1 -- 172.21.15.59:0/2112953580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0057d0 con 0x5626e5a3e490 2024-04-02T17:36:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 -- 172.21.15.59:0/2112953580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 msgr2=0x5626e5a33db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0099e0 tx=0x7fa79c00ffd0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 -- 172.21.15.59:0/2112953580 shutdown_connections 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a3e490 0x5626e5a33db0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a48240 0x5626e5a54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 --2- 172.21.15.59:0/2112953580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a430d0 0x5626e5a4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.196+0000 7fa7ad570740 1 -- 172.21.15.59:0/2112953580 >> 172.21.15.59:0/2112953580 conn(0x5626e5a3dc50 msgr2=0x5626e5a4c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.197+0000 7fa7ad570740 1 -- 172.21.15.59:0/2112953580 shutdown_connections 2024-04-02T17:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.197+0000 7fa7ad570740 1 -- 172.21.15.59:0/2112953580 wait complete. 2024-04-02T17:36:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.198+0000 7fa7ad570740 1 -- start start 2024-04-02T17:36:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.198+0000 7fa7ad570740 1 Processor -- start 2024-04-02T17:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 0x5626e5a51ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7ad570740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:40386/0 (socket says 172.21.15.59:40386) 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.199+0000 7fa7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49124/0 (socket says 172.21.15.59:49124) 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7a3400640 1 -- 172.21.15.59:0/713464826 learned_addr learned my addr 172.21.15.59:0/713464826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a8670 con 0x5626e5a48240 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a7ee0 con 0x5626e5a3e490 2024-04-02T17:36:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a5060 con 0x5626e5a430d0 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7ad570740 10 client.0 _ll_register_callbacks cb 0x7fa7940be550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3e00640 1 -- 172.21.15.59:0/713464826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 msgr2=0x5626e5a51ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.200+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 0x5626e5a51ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 0x5626e5a51ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3e00640 1 -- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 msgr2=0x5626e5a51580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3e00640 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e5a53070 con 0x5626e5a48240 2024-04-02T17:36:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.201+0000 7fa7a3400640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.202+0000 7fa7a2a00640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 0x5626e5a51ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:36:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.202+0000 7fa7a3e00640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00f790 tx=0x7fa79c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c387b3cbf63eb606 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.202+0000 7fa78d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:36:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.203+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c03c070 con 0x5626e5a48240 2024-04-02T17:36:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.203+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa79c01ed20 con 0x5626e5a48240 2024-04-02T17:36:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.203+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79c02b040 con 0x5626e5a48240 2024-04-02T17:36:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.203+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa79c048050 con 0x5626e5a48240 2024-04-02T17:36:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.206+0000 7fa7ad570740 10 client.17544 fetch_fsmap learned FSMap version 63 2024-04-02T17:36:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.206+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5626e5a54380 con 0x5626e5a48240 2024-04-02T17:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.207+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c02b040 con 0x5626e5a48240 2024-04-02T17:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7fa79c02f670 con 0x5626e5a48240 2024-04-02T17:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 10 client.17544 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 10 client.17544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 20 client.17544 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5abf4e0 0x5626e5ac1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 4 client.17544 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626e59ad2b0 con 0x5626e5abf4e0 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.208+0000 7fa7a3400640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5abf4e0 0x5626e5ac1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.209+0000 7fa7a3400640 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5abf4e0 0x5626e5ac1900 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ef5d87be101817 server_cookie=b0b8a4149158ade3 in_seq=0 out_seq=0 2024-04-02T17:36:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.209+0000 7fa78d600640 10 client.17544 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa78d600640 1 -- 172.21.15.59:0/713464826 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626e59ad2b0 con 0x5626e5abf4e0 2024-04-02T17:36:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa78d600640 10 client.17544 handle_command_reply: tid=0 2024-04-02T17:36:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 client.17544 shutdown 2024-04-02T17:36:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 msgr2=0x5626e5a52a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00f790 tx=0x7fa79c005800 comp rx=0 tx=0).stop 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 shutdown_connections 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5626e5abf4e0 0x5626e5ac1900 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5626e5a48240 0x5626e5a52a80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626e5a430d0 0x5626e5a51ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 --2- 172.21.15.59:0/713464826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626e5a3e490 0x5626e5a51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa78d600640 0 client.17544 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.513+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 >> 172.21.15.59:0/713464826 conn(0x5626e5a3dc50 msgr2=0x5626e5a40eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.514+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 shutdown_connections 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.514+0000 7fa7ad570740 1 -- 172.21.15.59:0/713464826 wait complete. 2024-04-02T17:36:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.514+0000 7fa7ad570740 20 client.17544 trim_cache size 0 max 16384 2024-04-02T17:36:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:36:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:36:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:36:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:36:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 msgr2=0x7fa7880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 0x7fa7880a6190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa79c01d000 tx=0x7fa79c01f000 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 msgr2=0x7fa7a419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa79402c650 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 shutdown_connections 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa7880a3cd0 0x7fa7880a6190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a4105910 0x7fa7a41a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7a4104f50 0x7fa7a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 --2- 172.21.15.59:0/1320324711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7a4103d60 0x7fa7a419a040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.515+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 >> 172.21.15.59:0/1320324711 conn(0x7fa7a40ff2f0 msgr2=0x7fa7a4108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.516+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 shutdown_connections 2024-04-02T17:36:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:44.516+0000 7fa7aa800640 1 -- 172.21.15.59:0/1320324711 wait complete. 2024-04-02T17:36:44.663 INFO:tasks.workunit.client.0.smithi059.stdout: 10 436205 37.07 MB/sec execute 491 sec latency 160.863 ms 2024-04-02T17:36:45.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 436937 37.11 MB/sec execute 492 sec latency 92.653 ms 2024-04-02T17:36:46.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:45 smithi154 ceph-mon[36479]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-04-02T17:36:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:45 smithi059 ceph-mon[31664]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-04-02T17:36:46.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:45 smithi164 ceph-mon[36499]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 690 op/s 2024-04-02T17:36:46.663 INFO:tasks.workunit.client.0.smithi059.stdout: 10 436971 37.04 MB/sec execute 493 sec latency 580.750 ms 2024-04-02T17:36:47.663 INFO:tasks.workunit.client.0.smithi059.stdout: 10 437254 36.99 MB/sec execute 494 sec latency 304.734 ms 2024-04-02T17:36:48.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:47 smithi059 ceph-mon[31664]: pgmap v1101: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-04-02T17:36:48.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:47 smithi164 ceph-mon[36499]: pgmap v1101: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-04-02T17:36:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:47 smithi154 ceph-mon[36479]: pgmap v1101: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 729 op/s 2024-04-02T17:36:48.663 INFO:tasks.workunit.client.0.smithi059.stdout: 10 437515 36.93 MB/sec execute 495 sec latency 160.262 ms 2024-04-02T17:36:49.665 INFO:tasks.workunit.client.0.smithi059.stdout: 10 437875 36.90 MB/sec execute 496 sec latency 429.738 ms 2024-04-02T17:36:50.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:49 smithi059 ceph-mon[31664]: pgmap v1102: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-04-02T17:36:50.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:49 smithi164 ceph-mon[36499]: pgmap v1102: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-04-02T17:36:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:49 smithi154 ceph-mon[36479]: pgmap v1102: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-04-02T17:36:50.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 438412 36.87 MB/sec execute 497 sec latency 270.208 ms 2024-04-02T17:36:51.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 439134 36.89 MB/sec execute 498 sec latency 114.468 ms 2024-04-02T17:36:52.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:51 smithi059 ceph-mon[31664]: pgmap v1103: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 900 op/s 2024-04-02T17:36:52.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:51 smithi164 ceph-mon[36499]: pgmap v1103: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 900 op/s 2024-04-02T17:36:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:51 smithi154 ceph-mon[36479]: pgmap v1103: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 900 op/s 2024-04-02T17:36:52.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 439871 36.92 MB/sec execute 499 sec latency 51.873 ms 2024-04-02T17:36:53.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 440475 36.92 MB/sec execute 500 sec latency 68.234 ms 2024-04-02T17:36:54.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:53 smithi059 ceph-mon[31664]: pgmap v1104: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 973 op/s 2024-04-02T17:36:54.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:53 smithi164 ceph-mon[36499]: pgmap v1104: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 973 op/s 2024-04-02T17:36:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:53 smithi154 ceph-mon[36479]: pgmap v1104: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 20 MiB/s wr, 973 op/s 2024-04-02T17:36:54.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 440987 36.90 MB/sec execute 501 sec latency 86.913 ms 2024-04-02T17:36:55.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 441540 36.91 MB/sec execute 502 sec latency 77.663 ms 2024-04-02T17:36:56.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:55 smithi059 ceph-mon[31664]: pgmap v1105: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 910 op/s 2024-04-02T17:36:56.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:55 smithi164 ceph-mon[36499]: pgmap v1105: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 910 op/s 2024-04-02T17:36:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:55 smithi154 ceph-mon[36479]: pgmap v1105: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 18 MiB/s wr, 910 op/s 2024-04-02T17:36:56.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 442230 36.93 MB/sec execute 503 sec latency 84.806 ms 2024-04-02T17:36:57.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 442932 36.95 MB/sec execute 504 sec latency 69.358 ms 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 -- 172.21.15.59:0/3422115621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b8077700 msgr2=0x7f57b8077b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 --2- 172.21.15.59:0/3422115621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b8077700 0x7f57b8077b60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a402bc30 comp rx=0 tx=0).stop 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 -- 172.21.15.59:0/3422115621 shutdown_connections 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 --2- 172.21.15.59:0/3422115621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80780a0 0x7f57b8078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 --2- 172.21.15.59:0/3422115621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b8077700 0x7f57b8077b60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 --2- 172.21.15.59:0/3422115621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b80768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 -- 172.21.15.59:0/3422115621 >> 172.21.15.59:0/3422115621 conn(0x7f57b80ff2c0 msgr2=0x7f57b8101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 -- 172.21.15.59:0/3422115621 shutdown_connections 2024-04-02T17:36:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.713+0000 7f57c0600640 1 -- 172.21.15.59:0/3422115621 wait complete. 2024-04-02T17:36:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.715+0000 7f57c0600640 1 Processor -- start 2024-04-02T17:36:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.715+0000 7f57c0600640 1 -- start start 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b8077700 0x7f57b80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b80780a0 0x7f57b81ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113e60 con 0x7f57b8077700 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113ce0 con 0x7f57b80764c0 2024-04-02T17:36:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.716+0000 7f57c0600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113fe0 con 0x7f57b80780a0 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42504/0 (socket says 172.21.15.59:42504) 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 -- 172.21.15.59:0/273901350 learned_addr learned my addr 172.21.15.59:0/273901350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 -- 172.21.15.59:0/273901350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b80780a0 msgr2=0x7f57b81ae570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b80780a0 0x7f57b81ae570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 -- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b8077700 msgr2=0x7f57b80709d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b8077700 0x7f57b80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.717+0000 7f57bdc00640 1 -- 172.21.15.59:0/273901350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8071070 con 0x7f57b80764c0 2024-04-02T17:36:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57bdc00640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f57b000cce0 tx=0x7f57b000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57b6000640 1 -- 172.21.15.59:0/273901350 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004b20 con 0x7f57b80764c0 2024-04-02T17:36:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57b6000640 1 -- 172.21.15.59:0/273901350 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0004cc0 con 0x7f57b80764c0 2024-04-02T17:36:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81aebd0 con 0x7f57b80764c0 2024-04-02T17:36:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57b6000640 1 -- 172.21.15.59:0/273901350 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b000f7d0 con 0x7f57b80764c0 2024-04-02T17:36:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.718+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81af140 con 0x7f57b80764c0 2024-04-02T17:36:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.724+0000 7f57c3204740 1 Processor -- start 2024-04-02T17:36:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.724+0000 7f57c3204740 1 -- start start 2024-04-02T17:36:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffded0 0x564ab3ffe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab4005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564ab3f63d70 con 0x564ab4003a30 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564ab3f0c2a0 con 0x564ab3ffded0 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57c3204740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ab3f5fd20 con 0x564ab3ffe8c0 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57b6000640 1 -- 172.21.15.59:0/273901350 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57b000f970 con 0x7f57b80764c0 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.725+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33878/0 (socket says 172.21.15.59:33878) 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 -- 172.21.15.59:0/681865265 learned_addr learned my addr 172.21.15.59:0/681865265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 -- 172.21.15.59:0/681865265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffded0 msgr2=0x564ab3ffe2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffded0 0x564ab3ffe2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 -- 172.21.15.59:0/681865265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 msgr2=0x564ab4005ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T17:36:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab4005ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.726+0000 7f57bdc00640 1 -- 172.21.15.59:0/681865265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab3f20840 con 0x564ab3ffe8c0 2024-04-02T17:36:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.727+0000 7f57bdc00640 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f57b00d6760 tx=0x7f57b00d6c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f6993d01b535f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.727+0000 7f57aaa00640 1 -- 172.21.15.59:0/681865265 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b00e3070 con 0x564ab3ffe8c0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.727+0000 7f57b6000640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 0x7f57a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.727+0000 7f57b6000640 1 -- 172.21.15.59:0/273901350 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f57b00d0510 con 0x7f57b80764c0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.728+0000 7f57aaa00640 1 -- 172.21.15.59:0/681865265 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b009ce90 con 0x564ab3ffe8c0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.728+0000 7f57aaa00640 1 -- 172.21.15.59:0/681865265 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0098a30 con 0x564ab3ffe8c0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.728+0000 7f57be600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 0x7f57a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.728+0000 7f57c3204740 1 -- 172.21.15.59:0/681865265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 msgr2=0x564ab40033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.729+0000 7f57c3204740 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f57b00d6760 tx=0x7f57b00d6c20 comp rx=0 tx=0).stop 2024-04-02T17:36:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.729+0000 7f57c3204740 1 -- 172.21.15.59:0/681865265 shutdown_connections 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab4005ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab40033c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 --2- 172.21.15.59:0/681865265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffded0 0x564ab3ffe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 -- 172.21.15.59:0/681865265 >> 172.21.15.59:0/681865265 conn(0x564ab3ff9440 msgr2=0x564ab3ff9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57be600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 0x7f57a00a61e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x564ab3fff120 tx=0x7f57ac006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 -- 172.21.15.59:0/681865265 shutdown_connections 2024-04-02T17:36:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.730+0000 7f57c3204740 1 -- 172.21.15.59:0/681865265 wait complete. 2024-04-02T17:36:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.731+0000 7f57c3204740 1 -- start start 2024-04-02T17:36:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.732+0000 7f57c3204740 1 Processor -- start 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.732+0000 7f57c3204740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab4008560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab4008aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564ab40069e0 con 0x564ab3ffded0 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564ab3f1f660 con 0x564ab4003a30 2024-04-02T17:36:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564ab4006b60 con 0x564ab3ffe8c0 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57c3204740 10 client.0 _ll_register_callbacks cb 0x564ab3f1f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57bd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab4008aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.733+0000 7f57be600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.734+0000 7f57bd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab4008aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33886/0 (socket says 172.21.15.59:33886) 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.734+0000 7f57bd200640 1 -- 172.21.15.59:0/3217620699 learned_addr learned my addr 172.21.15.59:0/3217620699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.734+0000 7f57bdc00640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab4008560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57bdc00640 1 -- 172.21.15.59:0/3217620699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 msgr2=0x564ab4008aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57bdc00640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab4008aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57bdc00640 1 -- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 msgr2=0x564ab400fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57bdc00640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57bdc00640 1 -- 172.21.15.59:0/3217620699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab3f2ee80 con 0x564ab3ffded0 2024-04-02T17:36:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.735+0000 7f57be600640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.736+0000 7f57bdc00640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab4008560 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f57b009d040 tx=0x7f57b00d3b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d97e869a9d987c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.736+0000 7f579f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-02T17:36:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.736+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b001d040 con 0x564ab3ffded0 2024-04-02T17:36:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.737+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f57b0087070 con 0x564ab3ffded0 2024-04-02T17:36:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.737+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b009ce90 con 0x564ab3ffded0 2024-04-02T17:36:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.738+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f57b009c0f0 con 0x564ab3ffded0 2024-04-02T17:36:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.738+0000 7f57c3204740 10 client.17547 fetch_fsmap learned FSMap version 63 2024-04-02T17:36:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.738+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564ab3c78210 con 0x564ab3ffded0 2024-04-02T17:36:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.740+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b009ce90 con 0x564ab3ffded0 2024-04-02T17:36:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.740+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f57b009c9f0 con 0x564ab3ffded0 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.741+0000 7f57c3204740 10 client.17547 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.741+0000 7f57c3204740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.741+0000 7f57c3204740 20 client.17547 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.741+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab4016350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.741+0000 7f57c3204740 4 client.17547 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.742+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab3f13cd0 con 0x564ab4013f60 2024-04-02T17:36:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.742+0000 7f57bd200640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab4016350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.747+0000 7f579f400640 10 client.17547 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.747+0000 7f57bd200640 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab4016350 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1966c7aef0161f8 server_cookie=3e81328e2e388dd in_seq=0 out_seq=0 2024-04-02T17:36:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.750+0000 7f579f400640 1 -- 172.21.15.59:0/3217620699 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ab3f13cd0 con 0x564ab4013f60 2024-04-02T17:36:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.750+0000 7f579f400640 10 client.17547 handle_command_reply: tid=0 2024-04-02T17:36:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.750+0000 7f57c3204740 1 client.17547 shutdown 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.750+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 msgr2=0x564ab4008560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.750+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab4008560 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f57b009d040 tx=0x7f57b00d3b80 comp rx=0 tx=0).stop 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 shutdown_connections 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab4016350 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab4008aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f579f400640 0 client.17547 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 --2- 172.21.15.59:0/3217620699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab4008560 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 >> 172.21.15.59:0/3217620699 conn(0x564ab3ff9440 msgr2=0x564ab3ff8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 shutdown_connections 2024-04-02T17:36:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.751+0000 7f57c3204740 1 -- 172.21.15.59:0/3217620699 wait complete. 2024-04-02T17:36:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.752+0000 7f57c3204740 20 client.17547 trim_cache size 0 max 16384 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 Processor -- start 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 -- start start 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 0x564ab3fff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 0x564ab4006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab40074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b8210 con 0x564ab4003a30 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b9ed0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.787+0000 7f57c3204740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b89a0 con 0x564ab3ffded0 2024-04-02T17:36:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.788+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 0x564ab3fff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.788+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 0x564ab3fff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33900/0 (socket says 172.21.15.59:33900) 2024-04-02T17:36:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.788+0000 7f57bdc00640 1 -- 172.21.15.59:0/839297861 learned_addr learned my addr 172.21.15.59:0/839297861 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.788+0000 7f57be600640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab40074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.788+0000 7f57bd200640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 0x564ab4006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.789+0000 7f57bd200640 1 -- 172.21.15.59:0/839297861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 msgr2=0x564ab3fff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.789+0000 7f57bd200640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 0x564ab3fff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.789+0000 7f57bd200640 1 -- 172.21.15.59:0/839297861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 msgr2=0x564ab40074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.789+0000 7f57bd200640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab40074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.790+0000 7f57bd200640 1 -- 172.21.15.59:0/839297861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab3d44620 con 0x564ab3ffe8c0 2024-04-02T17:36:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.790+0000 7f57be600640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab40074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.790+0000 7f57bd200640 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 0x564ab4006f60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f57a402c8e0 tx=0x7f57a4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd499c4262400213 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.791+0000 7f57aa000640 1 -- 172.21.15.59:0/839297861 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004040 con 0x564ab3ffe8c0 2024-04-02T17:36:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.791+0000 7f57aa000640 1 -- 172.21.15.59:0/839297861 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a40054e0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.791+0000 7f57aa000640 1 -- 172.21.15.59:0/839297861 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004960 con 0x564ab3ffe8c0 2024-04-02T17:36:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.792+0000 7f57c3204740 1 -- 172.21.15.59:0/839297861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 msgr2=0x564ab4006f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.792+0000 7f57c3204740 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 0x564ab4006f60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f57a402c8e0 tx=0x7f57a4000bc0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 -- 172.21.15.59:0/839297861 shutdown_connections 2024-04-02T17:36:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab4003a30 0x564ab40074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab3ffe8c0 0x564ab4006f60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 --2- 172.21.15.59:0/839297861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffded0 0x564ab3fff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 -- 172.21.15.59:0/839297861 >> 172.21.15.59:0/839297861 conn(0x564ab3ff9440 msgr2=0x564ab3ff9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 -- 172.21.15.59:0/839297861 shutdown_connections 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 -- 172.21.15.59:0/839297861 wait complete. 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.793+0000 7f57c3204740 1 -- start start 2024-04-02T17:36:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.794+0000 7f57c3204740 1 Processor -- start 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab400ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b9800 con 0x564ab3ffded0 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b9070 con 0x564ab4003a30 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b61f0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57c3204740 10 client.0 _ll_register_callbacks cb 0x7f57b00d66e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:36:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57bd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57bd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:33906/0 (socket says 172.21.15.59:33906) 2024-04-02T17:36:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.795+0000 7f57bd200640 1 -- 172.21.15.59:0/2362302245 learned_addr learned my addr 172.21.15.59:0/2362302245 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.796+0000 7f57be600640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.796+0000 7f57bdc00640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab400ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.796+0000 7f57bd200640 1 -- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 msgr2=0x564ab400c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.796+0000 7f57bd200640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.796+0000 7f57bd200640 1 -- 172.21.15.59:0/2362302245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 msgr2=0x564ab400ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f57bd200640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab400ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f57bd200640 1 -- 172.21.15.59:0/2362302245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab4078370 con 0x564ab3ffe8c0 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f57be600640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:36:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f57bdc00640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab400ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:36:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f57bd200640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f57a402c7b0 tx=0x7f57a40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19af5d3f2968cedb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:36:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.797+0000 7f579f400640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:36:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.798+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4005210 con 0x564ab3ffe8c0 2024-04-02T17:36:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.798+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f57a4004860 con 0x564ab3ffe8c0 2024-04-02T17:36:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.799+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a40054e0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.799+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f57a4004a60 con 0x564ab3ffe8c0 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.799+0000 7f57c3204740 10 client.27781 fetch_fsmap learned FSMap version 63 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.799+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564ab40796b0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a40054e0 con 0x564ab3ffe8c0 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f57a405e020 con 0x564ab3ffe8c0 2024-04-02T17:36:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 10 client.27781 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 10 client.27781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 20 client.27781 populate_metadata read hostname 'smithi059' 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab407c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 4 client.27781 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ab3f712b0 con 0x564ab4013f60 2024-04-02T17:36:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.800+0000 7f57bdc00640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab407c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:36:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.804+0000 7f579f400640 10 client.27781 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:57.804+0000 7f57bdc00640 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab407c1c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ddde99e90b7978 server_cookie=51da732ef217b325 in_seq=0 out_seq=0 2024-04-02T17:36:58.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:57 smithi059 ceph-mon[31664]: pgmap v1106: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 824 op/s 2024-04-02T17:36:58.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:57 smithi164 ceph-mon[36499]: pgmap v1106: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 824 op/s 2024-04-02T17:36:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:57 smithi154 ceph-mon[36479]: pgmap v1106: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 824 op/s 2024-04-02T17:36:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.491+0000 7f579f400640 1 -- 172.21.15.59:0/2362302245 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ab3f712b0 con 0x564ab4013f60 2024-04-02T17:36:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f579f400640 10 client.27781 handle_command_reply: tid=0 2024-04-02T17:36:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 client.27781 shutdown 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 msgr2=0x564ab400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f57a402c7b0 tx=0x7f57a40095f0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 shutdown_connections 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564ab4013f60 0x564ab407c1c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f579f400640 0 client.27781 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564ab4003a30 0x564ab400c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564ab3ffe8c0 0x564ab400ba60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 --2- 172.21.15.59:0/2362302245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564ab3ffded0 0x564ab400ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.492+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 >> 172.21.15.59:0/2362302245 conn(0x564ab3ff9440 msgr2=0x564ab3fff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.493+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 shutdown_connections 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.493+0000 7f57c3204740 1 -- 172.21.15.59:0/2362302245 wait complete. 2024-04-02T17:36:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.493+0000 7f57c3204740 20 client.27781 trim_cache size 0 max 16384 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 msgr2=0x7f57a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 0x7f57a00a61e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x564ab3fff120 tx=0x7f57ac006c20 comp rx=0 tx=0).stop 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 msgr2=0x7f57b8070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:36:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f57b000cce0 tx=0x7f57b000a620 comp rx=0 tx=0).stop 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 shutdown_connections 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f57a00a3d20 0x7f57a00a61e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b8077700 0x7f57b80709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57b80780a0 0x7f57b81ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 --2- 172.21.15.59:0/273901350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57b80764c0 0x7f57b8070490 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:36:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.495+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 >> 172.21.15.59:0/273901350 conn(0x7f57b80ff2c0 msgr2=0x7f57b8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:36:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.497+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 shutdown_connections 2024-04-02T17:36:58.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:36:58.497+0000 7f57c0600640 1 -- 172.21.15.59:0/273901350 wait complete. 2024-04-02T17:36:58.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 443209 36.91 MB/sec execute 505 sec latency 402.736 ms 2024-04-02T17:36:59.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 443933 36.92 MB/sec execute 506 sec latency 79.495 ms 2024-04-02T17:37:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:59 smithi059 ceph-mon[31664]: pgmap v1107: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 725 op/s 2024-04-02T17:37:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:36:59 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:37:00.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:59 smithi164 ceph-mon[36499]: pgmap v1107: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 725 op/s 2024-04-02T17:37:00.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:36:59 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:37:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:59 smithi154 ceph-mon[36479]: pgmap v1107: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 725 op/s 2024-04-02T17:37:00.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:36:59 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:37:00.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 444458 36.90 MB/sec execute 507 sec latency 83.211 ms 2024-04-02T17:37:01.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 444717 36.85 MB/sec execute 508 sec latency 406.863 ms 2024-04-02T17:37:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:01 smithi059 ceph-mon[31664]: pgmap v1108: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.0 MiB/s wr, 633 op/s 2024-04-02T17:37:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:37:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:01 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:37:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:01 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:37:02.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:01 smithi164 ceph-mon[36499]: pgmap v1108: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.0 MiB/s wr, 633 op/s 2024-04-02T17:37:02.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:37:02.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:01 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:37:02.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:01 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:37:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:01 smithi154 ceph-mon[36479]: pgmap v1108: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.0 MiB/s wr, 633 op/s 2024-04-02T17:37:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:37:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:01 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:37:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:01 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:37:02.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 445242 36.82 MB/sec execute 509 sec latency 293.792 ms 2024-04-02T17:37:03.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 445842 36.81 MB/sec execute 510 sec latency 71.264 ms 2024-04-02T17:37:04.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:03 smithi059 ceph-mon[31664]: pgmap v1109: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-04-02T17:37:04.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:03 smithi164 ceph-mon[36499]: pgmap v1109: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-04-02T17:37:04.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:03 smithi154 ceph-mon[36479]: pgmap v1109: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-04-02T17:37:04.664 INFO:tasks.workunit.client.0.smithi059.stdout: 10 446664 36.82 MB/sec execute 511 sec latency 126.141 ms 2024-04-02T17:37:05.665 INFO:tasks.workunit.client.0.smithi059.stdout: 10 447382 36.85 MB/sec execute 512 sec latency 105.389 ms 2024-04-02T17:37:06.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:05 smithi059 ceph-mon[31664]: pgmap v1110: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 589 op/s 2024-04-02T17:37:06.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:05 smithi164 ceph-mon[36499]: pgmap v1110: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 589 op/s 2024-04-02T17:37:06.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:05 smithi154 ceph-mon[36479]: pgmap v1110: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 589 op/s 2024-04-02T17:37:06.666 INFO:tasks.workunit.client.0.smithi059.stdout: 10 447948 36.84 MB/sec execute 513 sec latency 86.566 ms 2024-04-02T17:37:07.666 INFO:tasks.workunit.client.0.smithi059.stdout: 10 448028 36.78 MB/sec execute 514 sec latency 800.024 ms 2024-04-02T17:37:08.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:07 smithi059 ceph-mon[31664]: pgmap v1111: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-04-02T17:37:08.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:07 smithi164 ceph-mon[36499]: pgmap v1111: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-04-02T17:37:08.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:07 smithi154 ceph-mon[36479]: pgmap v1111: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-04-02T17:37:08.668 INFO:tasks.workunit.client.0.smithi059.stdout: 10 448321 36.74 MB/sec execute 515 sec latency 138.848 ms 2024-04-02T17:37:09.666 INFO:tasks.workunit.client.0.smithi059.stdout: 10 448849 36.70 MB/sec execute 516 sec latency 98.497 ms 2024-04-02T17:37:10.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:09 smithi059 ceph-mon[31664]: pgmap v1112: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2024-04-02T17:37:10.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:09 smithi164 ceph-mon[36499]: pgmap v1112: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2024-04-02T17:37:10.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:09 smithi154 ceph-mon[36479]: pgmap v1112: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2024-04-02T17:37:10.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 449519 36.70 MB/sec execute 517 sec latency 179.691 ms 2024-04-02T17:37:11.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 450283 36.73 MB/sec execute 518 sec latency 107.472 ms 2024-04-02T17:37:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.673+0000 7f188be00640 1 -- 172.21.15.59:0/896554640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 msgr2=0x7f18840768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.673+0000 7f188be00640 1 --2- 172.21.15.59:0/896554640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f18840768e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1878009900 tx=0x7f187802bc50 comp rx=0 tx=0).stop 2024-04-02T17:37:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 -- 172.21.15.59:0/896554640 shutdown_connections 2024-04-02T17:37:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 --2- 172.21.15.59:0/896554640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f1884077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 --2- 172.21.15.59:0/896554640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f1884078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 --2- 172.21.15.59:0/896554640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f18840768e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 -- 172.21.15.59:0/896554640 >> 172.21.15.59:0/896554640 conn(0x7f18840ff2f0 msgr2=0x7f1884101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 -- 172.21.15.59:0/896554640 shutdown_connections 2024-04-02T17:37:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.674+0000 7f188be00640 1 -- 172.21.15.59:0/896554640 wait complete. 2024-04-02T17:37:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.675+0000 7f188be00640 1 Processor -- start 2024-04-02T17:37:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.675+0000 7f188be00640 1 -- start start 2024-04-02T17:37:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f188419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f1888a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f188419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f18841a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1884113df0 con 0x7f1884077700 2024-04-02T17:37:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1884113c70 con 0x7f18840764c0 2024-04-02T17:37:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f188be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1884113f70 con 0x7f18840780a0 2024-04-02T17:37:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f1889400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f1889400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42490/0 (socket says 172.21.15.59:42490) 2024-04-02T17:37:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.677+0000 7f1889400640 1 -- 172.21.15.59:0/1892839980 learned_addr learned my addr 172.21.15.59:0/1892839980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889e00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f18841a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889400640 1 -- 172.21.15.59:0/1892839980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 msgr2=0x7f18841a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889400640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f18841a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889400640 1 -- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 msgr2=0x7f188419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889400640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f188419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889400640 1 -- 172.21.15.59:0/1892839980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18841a5f70 con 0x7f18840764c0 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1888a00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f188419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.678+0000 7f1889e00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f18841a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.679+0000 7f1889400640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1878009850 tx=0x7f1878004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.679+0000 7f1882000640 1 -- 172.21.15.59:0/1892839980 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878041070 con 0x7f18840764c0 2024-04-02T17:37:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.680+0000 7f1882000640 1 -- 172.21.15.59:0/1892839980 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18780054e0 con 0x7f18840764c0 2024-04-02T17:37:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.680+0000 7f1882000640 1 -- 172.21.15.59:0/1892839980 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878003e20 con 0x7f18840764c0 2024-04-02T17:37:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.680+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18841a6200 con 0x7f18840764c0 2024-04-02T17:37:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.680+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a6700 con 0x7f18840764c0 2024-04-02T17:37:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.684+0000 7f1882000640 1 -- 172.21.15.59:0/1892839980 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1878044070 con 0x7f18840764c0 2024-04-02T17:37:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.686+0000 7f188ec17740 1 Processor -- start 2024-04-02T17:37:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.686+0000 7f188ec17740 1 -- start start 2024-04-02T17:37:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x5593643752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x55936437a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5593642dad70 con 0x559364374ed0 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5593642832a0 con 0x55936437aa30 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.687+0000 7f188ec17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5593642d6d20 con 0x5593643758c0 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1882000640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 0x7f186c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1889e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x5593643752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1889e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x5593643752f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36730/0 (socket says 172.21.15.59:36730) 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1889e00640 1 -- 172.21.15.59:0/463777822 learned_addr learned my addr 172.21.15.59:0/463777822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42502/0 (socket says 172.21.15.59:42502) 2024-04-02T17:37:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1889400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x55936437a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 -- 172.21.15.59:0/463777822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 msgr2=0x55936437a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x55936437a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 -- 172.21.15.59:0/463777822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 msgr2=0x5593643752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x5593643752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1882000640 1 -- 172.21.15.59:0/1892839980 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f18780f1880 con 0x7f18840764c0 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.688+0000 7f1888a00640 1 -- 172.21.15.59:0/463777822 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559364297840 con 0x55936437aa30 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.689+0000 7f1888a00640 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1870009fa0 tx=0x7f18700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e391b7d3594f47f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.689+0000 7f1889e00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 0x7f186c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.689+0000 7f1876600640 1 -- 172.21.15.59:0/463777822 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000f5a0 con 0x55936437aa30 2024-04-02T17:37:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.689+0000 7f1876600640 1 -- 172.21.15.59:0/463777822 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187000fc20 con 0x55936437aa30 2024-04-02T17:37:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.689+0000 7f1876600640 1 -- 172.21.15.59:0/463777822 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870019dc0 con 0x55936437aa30 2024-04-02T17:37:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f1889e00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 0x7f186c0a61e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f187c005f40 tx=0x7f187c009e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 -- 172.21.15.59:0/463777822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 msgr2=0x55936437cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1870009fa0 tx=0x7f18700118d0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 -- 172.21.15.59:0/463777822 shutdown_connections 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x55936437cec0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x55936437a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 --2- 172.21.15.59:0/463777822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x5593643752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 -- 172.21.15.59:0/463777822 >> 172.21.15.59:0/463777822 conn(0x559364370440 msgr2=0x559364370840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 -- 172.21.15.59:0/463777822 shutdown_connections 2024-04-02T17:37:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.691+0000 7f188ec17740 1 -- 172.21.15.59:0/463777822 wait complete. 2024-04-02T17:37:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.692+0000 7f188ec17740 1 -- start start 2024-04-02T17:37:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.692+0000 7f188ec17740 1 Processor -- start 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 0x5593643903c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 0x559364390900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364397940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55936438e9f0 con 0x55936437aa30 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559364296660 con 0x5593643758c0 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55936438eb70 con 0x559364374ed0 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f188ec17740 10 client.0 _ll_register_callbacks cb 0x559364296050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f1888a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 0x559364390900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f1889400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 0x5593643903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f1888a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 0x559364390900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42506/0 (socket says 172.21.15.59:42506) 2024-04-02T17:37:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f1888a00640 1 -- 172.21.15.59:0/756783564 learned_addr learned my addr 172.21.15.59:0/756783564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.693+0000 7f1889e00640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364397940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 -- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 msgr2=0x559364390900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 0x559364390900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 -- 172.21.15.59:0/756783564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 msgr2=0x559364397940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364397940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 -- 172.21.15.59:0/756783564 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593642a5e80 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889e00640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364397940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f1889400640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 0x5593643903c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f18780ef370 tx=0x7f1878004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7e9f65082e34db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.694+0000 7f186aa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187804b070 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f18780ef430 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187000fc20 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f18780f4e60 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f188ec17740 10 client.27790 fetch_fsmap learned FSMap version 63 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x559363fef210 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.695+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187000fc20 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f18780c62e0 con 0x559364374ed0 2024-04-02T17:37:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 10 client.27790 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 10 client.27790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 20 client.27790 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55936439c2c0 0x55936439e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 4 client.27790 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55936428acd0 con 0x55936439c2c0 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.696+0000 7f1888a00640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55936439c2c0 0x55936439e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.697+0000 7f1888a00640 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55936439c2c0 0x55936439e780 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feabbc687f11c712 server_cookie=dd70dbc043167ac7 in_seq=0 out_seq=0 2024-04-02T17:37:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.697+0000 7f186aa00640 10 client.27790 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f186aa00640 1 -- 172.21.15.59:0/756783564 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55936428acd0 con 0x55936439c2c0 2024-04-02T17:37:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f186aa00640 10 client.27790 handle_command_reply: tid=0 2024-04-02T17:37:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f188ec17740 1 client.27790 shutdown 2024-04-02T17:37:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 msgr2=0x5593643903c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 0x5593643903c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f18780ef370 tx=0x7f1878004510 comp rx=0 tx=0).stop 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 shutdown_connections 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.702+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55936439c2c0 0x55936439e780 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364397940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5593643758c0 0x559364390900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 --2- 172.21.15.59:0/756783564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559364374ed0 0x5593643903c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 >> 172.21.15.59:0/756783564 conn(0x559364370440 msgr2=0x55936436f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f186aa00640 0 client.27790 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 shutdown_connections 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 1 -- 172.21.15.59:0/756783564 wait complete. 2024-04-02T17:37:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.703+0000 7f188ec17740 20 client.27790 trim_cache size 0 max 16384 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 Processor -- start 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 -- start start 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x559364390860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x559364373db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a6eb0 con 0x559364374ed0 2024-04-02T17:37:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a8b70 con 0x55936437aa30 2024-04-02T17:37:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.732+0000 7f188ec17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a7640 con 0x5593643758c0 2024-04-02T17:37:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:42782/0 (socket says 172.21.15.59:42782) 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 -- 172.21.15.59:0/4048557216 learned_addr learned my addr 172.21.15.59:0/4048557216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1889400640 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x559364390860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1889e00640 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x559364373db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 -- 172.21.15.59:0/4048557216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 msgr2=0x559364373db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x559364373db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 -- 172.21.15.59:0/4048557216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 msgr2=0x559364390860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x559364390860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.733+0000 7f1888a00640 1 -- 172.21.15.59:0/4048557216 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593640bb620 con 0x5593643758c0 2024-04-02T17:37:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.734+0000 7f1888a00640 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1870009250 tx=0x7f1870009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def81678e739df2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.734+0000 7f1875c00640 1 -- 172.21.15.59:0/4048557216 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870031070 con 0x5593643758c0 2024-04-02T17:37:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.734+0000 7f1875c00640 1 -- 172.21.15.59:0/4048557216 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187002bde0 con 0x5593643758c0 2024-04-02T17:37:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.734+0000 7f1875c00640 1 -- 172.21.15.59:0/4048557216 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870031070 con 0x5593643758c0 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 -- 172.21.15.59:0/4048557216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 msgr2=0x559364397900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1870009250 tx=0x7f1870009770 comp rx=0 tx=0).stop 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 -- 172.21.15.59:0/4048557216 shutdown_connections 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55936437aa30 0x559364373db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x559364397900 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 --2- 172.21.15.59:0/4048557216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559364374ed0 0x559364390860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.735+0000 7f188ec17740 1 -- 172.21.15.59:0/4048557216 >> 172.21.15.59:0/4048557216 conn(0x559364370440 msgr2=0x5593643736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.736+0000 7f188ec17740 1 -- 172.21.15.59:0/4048557216 shutdown_connections 2024-04-02T17:37:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.736+0000 7f188ec17740 1 -- 172.21.15.59:0/4048557216 wait complete. 2024-04-02T17:37:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.736+0000 7f188ec17740 1 -- start start 2024-04-02T17:37:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.736+0000 7f188ec17740 1 Processor -- start 2024-04-02T17:37:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x5593643953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f1889400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364396400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f1889400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42518/0 (socket says 172.21.15.59:42518) 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a84a0 con 0x55936437aa30 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a7d10 con 0x559364374ed0 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f188410a410 con 0x5593643758c0 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f188ec17740 10 client.0 _ll_register_callbacks cb 0x7f1878005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.737+0000 7f1889400640 1 -- 172.21.15.59:0/3335993161 learned_addr learned my addr 172.21.15.59:0/3335993161 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1888a00640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x5593643953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889e00640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364396400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889400640 1 -- 172.21.15.59:0/3335993161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 msgr2=0x5593643953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889400640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x5593643953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889400640 1 -- 172.21.15.59:0/3335993161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 msgr2=0x559364396400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889400640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364396400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.738+0000 7f1889400640 1 -- 172.21.15.59:0/3335993161 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593643969f0 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f1889400640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18780f1cd0 tx=0x7f1878009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ce2ba521ef0ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f186aa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187804b070 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f18780035f0 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187002bde0 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f18780bdb00 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f188ec17740 10 client.27737 fetch_fsmap learned FSMap version 63 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.739+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5593643969f0 con 0x559364374ed0 2024-04-02T17:37:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1878005690 con 0x559364374ed0 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f18780bcf50 con 0x559364374ed0 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 10 client.27737 fetch_fsmap finished waiting for FSMap version 63 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 10 client.27737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 20 client.27737 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559364413ef0 0x559364416310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 4 client.27737 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:37:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f187002bde0 con 0x559364413ef0 2024-04-02T17:37:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.740+0000 7f1888a00640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559364413ef0 0x559364416310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.741+0000 7f1888a00640 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559364413ef0 0x559364416310 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bf141486840b6c server_cookie=9d3140342419d976 in_seq=0 out_seq=0 2024-04-02T17:37:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:11.741+0000 7f186aa00640 10 client.27737 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.040+0000 7f186aa00640 1 -- 172.21.15.59:0/3335993161 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f187002bde0 con 0x559364413ef0 2024-04-02T17:37:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.040+0000 7f186aa00640 10 client.27737 handle_command_reply: tid=0 2024-04-02T17:37:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.040+0000 7f188ec17740 1 client.27737 shutdown 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.040+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 msgr2=0x559364394eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.040+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18780f1cd0 tx=0x7f1878009560 comp rx=0 tx=0).stop 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 shutdown_connections 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x559364413ef0 0x559364416310 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55936437aa30 0x559364396400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f186aa00640 0 client.27737 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593643758c0 0x5593643953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 --2- 172.21.15.59:0/3335993161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559364374ed0 0x559364394eb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 >> 172.21.15.59:0/3335993161 conn(0x559364370440 msgr2=0x5593643906e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 shutdown_connections 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.041+0000 7f188ec17740 1 -- 172.21.15.59:0/3335993161 wait complete. 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.042+0000 7f188ec17740 20 client.27737 trim_cache size 0 max 16384 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:37:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:37:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:37:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:37:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.042+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 msgr2=0x7f186c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.042+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 0x7f186c0a61e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f187c005f40 tx=0x7f187c009e90 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.042+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 msgr2=0x7f188419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.042+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1878009850 tx=0x7f1878004e30 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 shutdown_connections 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f186c0a3d20 0x7f186c0a61e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1884077700 0x7f188419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18840780a0 0x7f18841a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 --2- 172.21.15.59:0/1892839980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18840764c0 0x7f188419e350 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.043+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 >> 172.21.15.59:0/1892839980 conn(0x7f18840ff2f0 msgr2=0x7f1884100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.044+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 shutdown_connections 2024-04-02T17:37:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:12.044+0000 7f188be00640 1 -- 172.21.15.59:0/1892839980 wait complete. 2024-04-02T17:37:12.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:11 smithi059 ceph-mon[31664]: pgmap v1113: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-04-02T17:37:12.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:11 smithi164 ceph-mon[36499]: pgmap v1113: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-04-02T17:37:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:11 smithi154 ceph-mon[36479]: pgmap v1113: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-04-02T17:37:12.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 450863 36.74 MB/sec execute 519 sec latency 178.337 ms 2024-04-02T17:37:13.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 451390 36.74 MB/sec execute 520 sec latency 83.623 ms 2024-04-02T17:37:14.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:13 smithi059 ceph-mon[31664]: pgmap v1114: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-04-02T17:37:14.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:13 smithi164 ceph-mon[36499]: pgmap v1114: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-04-02T17:37:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:13 smithi154 ceph-mon[36479]: pgmap v1114: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 690 op/s 2024-04-02T17:37:14.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 451949 36.71 MB/sec execute 521 sec latency 58.511 ms 2024-04-02T17:37:15.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:15 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:15.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:15 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:15.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:15 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:15.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 452568 36.69 MB/sec execute 522 sec latency 90.952 ms 2024-04-02T17:37:16.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:16 smithi059 ceph-mon[31664]: pgmap v1115: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 802 op/s 2024-04-02T17:37:16.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:16 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:16.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:16 smithi164 ceph-mon[36499]: pgmap v1115: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 802 op/s 2024-04-02T17:37:16.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:16 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:16 smithi154 ceph-mon[36479]: pgmap v1115: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 802 op/s 2024-04-02T17:37:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:16 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:16.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 453314 36.71 MB/sec execute 523 sec latency 67.477 ms 2024-04-02T17:37:17.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 454118 36.71 MB/sec execute 524 sec latency 60.275 ms 2024-04-02T17:37:18.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:18 smithi059 ceph-mon[31664]: pgmap v1116: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 677 op/s 2024-04-02T17:37:18.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:18 smithi164 ceph-mon[36499]: pgmap v1116: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 677 op/s 2024-04-02T17:37:18.402 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:18 smithi154 ceph-mon[36479]: pgmap v1116: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 15 MiB/s wr, 677 op/s 2024-04-02T17:37:18.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 454786 36.73 MB/sec execute 525 sec latency 43.663 ms 2024-04-02T17:37:19.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 455391 36.71 MB/sec execute 526 sec latency 127.613 ms 2024-04-02T17:37:20.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:20 smithi059 ceph-mon[31664]: pgmap v1117: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 682 op/s 2024-04-02T17:37:20.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:20 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:20.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:20 smithi164 ceph-mon[36499]: pgmap v1117: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 682 op/s 2024-04-02T17:37:20.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:20 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:20 smithi154 ceph-mon[36479]: pgmap v1117: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 682 op/s 2024-04-02T17:37:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:20 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:20.669 INFO:tasks.workunit.client.0.smithi059.stdout: 10 455690 36.68 MB/sec execute 527 sec latency 439.894 ms 2024-04-02T17:37:21.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 456432 36.67 MB/sec execute 528 sec latency 102.485 ms 2024-04-02T17:37:22.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:22 smithi059 ceph-mon[31664]: pgmap v1118: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:37:22.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:22 smithi164 ceph-mon[36499]: pgmap v1118: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:37:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:22 smithi154 ceph-mon[36479]: pgmap v1118: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-04-02T17:37:22.667 INFO:tasks.workunit.client.0.smithi059.stdout: 10 457115 36.69 MB/sec execute 529 sec latency 70.872 ms 2024-04-02T17:37:23.668 INFO:tasks.workunit.client.0.smithi059.stdout: 10 457897 36.73 MB/sec execute 530 sec latency 40.474 ms 2024-04-02T17:37:24.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:24 smithi059 ceph-mon[31664]: pgmap v1119: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 561 op/s 2024-04-02T17:37:24.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:24 smithi164 ceph-mon[36499]: pgmap v1119: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 561 op/s 2024-04-02T17:37:24.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:24 smithi154 ceph-mon[36479]: pgmap v1119: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 561 op/s 2024-04-02T17:37:24.668 INFO:tasks.workunit.client.0.smithi059.stdout: 10 458526 36.72 MB/sec execute 531 sec latency 42.481 ms 2024-04-02T17:37:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 -- 172.21.15.59:0/2986079622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a4410de20 msgr2=0x7f9a4410e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/2986079622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a4410de20 0x7f9a4410e2a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009ca0 tx=0x7f9a3c02c100 comp rx=0 tx=0).stop 2024-04-02T17:37:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 -- 172.21.15.59:0/2986079622 shutdown_connections 2024-04-02T17:37:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/2986079622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a4410de20 0x7f9a4410e2a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/2986079622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a4410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/2986079622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a44108b00 0x7f9a4410af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.229+0000 7f9a4ac00640 1 -- 172.21.15.59:0/2986079622 >> 172.21.15.59:0/2986079622 conn(0x7f9a4406eb40 msgr2=0x7f9a44070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.230+0000 7f9a4ac00640 1 -- 172.21.15.59:0/2986079622 shutdown_connections 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.230+0000 7f9a4ac00640 1 -- 172.21.15.59:0/2986079622 wait complete. 2024-04-02T17:37:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.231+0000 7f9a4ac00640 1 Processor -- start 2024-04-02T17:37:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.231+0000 7f9a4ac00640 1 -- start start 2024-04-02T17:37:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a441155f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a4410de20 0x7f9a441b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4411c5a0 con 0x7f9a4410de20 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4411c420 con 0x7f9a44108b00 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a4ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4411c720 con 0x7f9a4410b470 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a43400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a441155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53654/0 (socket says 172.21.15.59:53654) 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a4410de20 0x7f9a441b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.232+0000 7f9a43e00640 1 -- 172.21.15.59:0/746682427 learned_addr learned my addr 172.21.15.59:0/746682427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43400640 1 -- 172.21.15.59:0/746682427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 msgr2=0x7f9a441150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43400640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43400640 1 -- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a4410de20 msgr2=0x7f9a441b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43400640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a4410de20 0x7f9a441b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43400640 1 -- 172.21.15.59:0/746682427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44118720 con 0x7f9a4410b470 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.233+0000 7f9a43e00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.234+0000 7f9a43400640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a441155f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9f0 tx=0x7f9a3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.234+0000 7f9a40c00640 1 -- 172.21.15.59:0/746682427 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000ccc0 con 0x7f9a4410b470 2024-04-02T17:37:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.234+0000 7f9a40c00640 1 -- 172.21.15.59:0/746682427 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a30010910 con 0x7f9a4410b470 2024-04-02T17:37:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.234+0000 7f9a40c00640 1 -- 172.21.15.59:0/746682427 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30018960 con 0x7f9a4410b470 2024-04-02T17:37:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.235+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441b71b0 con 0x7f9a4410b470 2024-04-02T17:37:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.235+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441b7680 con 0x7f9a4410b470 2024-04-02T17:37:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.241+0000 7f9a4d9e7740 1 Processor -- start 2024-04-02T17:37:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.241+0000 7f9a4d9e7740 1 -- start start 2024-04-02T17:37:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.242+0000 7f9a40c00640 1 -- 172.21.15.59:0/746682427 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a30010ac0 con 0x7f9a4410b470 2024-04-02T17:37:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afcd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afcfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56022af2dd70 con 0x56022afc88c0 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56022aed62a0 con 0x56022afc7ed0 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56022af29d20 con 0x56022afcda30 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53662/0 (socket says 172.21.15.59:53662) 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a48c00640 1 -- 172.21.15.59:0/100924419 learned_addr learned my addr 172.21.15.59:0/100924419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.245+0000 7f9a43400640 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afcfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.246+0000 7f9a48c00640 1 -- 172.21.15.59:0/100924419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 msgr2=0x56022afcfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.246+0000 7f9a48c00640 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afcfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.246+0000 7f9a48c00640 1 -- 172.21.15.59:0/100924419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 msgr2=0x56022afcd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.246+0000 7f9a48c00640 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afcd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.246+0000 7f9a48c00640 1 -- 172.21.15.59:0/100924419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022aeea840 con 0x56022afc7ed0 2024-04-02T17:37:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.247+0000 7f9a48c00640 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c02c710 tx=0x7f9a3c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84fae712d4c89d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.247+0000 7f9a40c00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 0x7f9a280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.248+0000 7f9a40c00640 1 -- 172.21.15.59:0/746682427 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9a300d0870 con 0x7f9a4410b470 2024-04-02T17:37:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.248+0000 7f9a39200640 1 -- 172.21.15.59:0/100924419 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c043020 con 0x56022afc7ed0 2024-04-02T17:37:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.248+0000 7f9a39200640 1 -- 172.21.15.59:0/100924419 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c003970 con 0x56022afc7ed0 2024-04-02T17:37:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.248+0000 7f9a39200640 1 -- 172.21.15.59:0/100924419 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c039040 con 0x56022afc7ed0 2024-04-02T17:37:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.249+0000 7f9a43e00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 0x7f9a280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/100924419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 msgr2=0x56022afc82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c02c710 tx=0x7f9a3c009800 comp rx=0 tx=0).stop 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/100924419 shutdown_connections 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afcd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afcfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/100924419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afc82f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/100924419 >> 172.21.15.59:0/100924419 conn(0x56022afc3440 msgr2=0x56022afc3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/100924419 shutdown_connections 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.250+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/100924419 wait complete. 2024-04-02T17:37:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.251+0000 7f9a4d9e7740 1 -- start start 2024-04-02T17:37:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.251+0000 7f9a4d9e7740 1 Processor -- start 2024-04-02T17:37:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.252+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.252+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.252+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afcda30 0x56022afd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.252+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36762/0 (socket says 172.21.15.59:36762) 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a43e00640 1 -- 172.21.15.59:0/2964034699 learned_addr learned my addr 172.21.15.59:0/2964034699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56022afd0980 con 0x56022afc88c0 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56022aee9660 con 0x56022afcda30 2024-04-02T17:37:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56022afd0b00 con 0x56022afc7ed0 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.253+0000 7f9a4d9e7740 10 client.0 _ll_register_callbacks cb 0x56022aee9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43400640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 -- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afcda30 msgr2=0x56022afd99f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afcda30 0x56022afd99f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 -- 172.21.15.59:0/2964034699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 msgr2=0x56022afd2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022aef8e80 con 0x56022afc7ed0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 0x7f9a280a6210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x56022afc43b0 tx=0x7f9a340046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.254+0000 7f9a43e00640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a34010de0 tx=0x7f9a3400c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd8e96e4c2ca58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.255+0000 7f9a2d600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.255+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34028060 con 0x56022afc7ed0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.255+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9a34023410 con 0x56022afc7ed0 2024-04-02T17:37:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.256+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3c003970 con 0x56022afc7ed0 2024-04-02T17:37:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.256+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9a3401f030 con 0x56022afc7ed0 2024-04-02T17:37:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.256+0000 7f9a4d9e7740 10 client.27808 fetch_fsmap learned FSMap version 67 2024-04-02T17:37:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.256+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x56022ac42210 con 0x56022afc7ed0 2024-04-02T17:37:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c003970 con 0x56022afc7ed0 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 67) ==== 8406+0+0 (secure 0 0 0) 0x7f9a3402c4a0 con 0x56022afc7ed0 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 10 client.27808 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 10 client.27808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 20 client.27808 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022afde370 0x56022afe0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 4 client.27808 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:37:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56022aeddcd0 con 0x56022afde370 2024-04-02T17:37:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.257+0000 7f9a43400640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022afde370 0x56022afe0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.258+0000 7f9a43400640 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022afde370 0x56022afe0810 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799260178b7d923d server_cookie=f4b3904a1ec16087 in_seq=0 out_seq=0 2024-04-02T17:37:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.258+0000 7f9a2d600640 10 client.27808 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.260+0000 7f9a2d600640 1 -- 172.21.15.59:0/2964034699 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56022aeddcd0 con 0x56022afde370 2024-04-02T17:37:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.260+0000 7f9a2d600640 10 client.27808 handle_command_reply: tid=0 2024-04-02T17:37:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.260+0000 7f9a4d9e7740 1 client.27808 shutdown 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 msgr2=0x56022afd2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9a34010de0 tx=0x7f9a3400c700 comp rx=0 tx=0).stop 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 shutdown_connections 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022afde370 0x56022afe0810 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afcda30 0x56022afd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/2964034699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc7ed0 0x56022afd2450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a2d600640 0 client.27808 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.261+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 >> 172.21.15.59:0/2964034699 conn(0x56022afc3440 msgr2=0x56022afc2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.262+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 shutdown_connections 2024-04-02T17:37:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.262+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/2964034699 wait complete. 2024-04-02T17:37:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.262+0000 7f9a4d9e7740 20 client.27808 trim_cache size 0 max 16384 2024-04-02T17:37:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.291+0000 7f9a4d9e7740 1 Processor -- start 2024-04-02T17:37:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 -- start start 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 0x56022afd99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 0x56022afc1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441c0800 con 0x56022afcda30 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441c2580 con 0x56022afc7ed0 2024-04-02T17:37:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.292+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441c0f90 con 0x56022afc88c0 2024-04-02T17:37:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.293+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.293+0000 7f9a43e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:53688/0 (socket says 172.21.15.59:53688) 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.293+0000 7f9a43e00640 1 -- 172.21.15.59:0/3105081130 learned_addr learned my addr 172.21.15.59:0/3105081130 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43e00640 1 -- 172.21.15.59:0/3105081130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 msgr2=0x56022afd99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.293+0000 7f9a43400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 0x56022afd99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.293+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 0x56022afc1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43e00640 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 0x56022afd99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43e00640 1 -- 172.21.15.59:0/3105081130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 msgr2=0x56022afc1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43e00640 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 0x56022afc1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43e00640 1 -- 172.21.15.59:0/3105081130 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022ad0e620 con 0x56022afc7ed0 2024-04-02T17:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.294+0000 7f9a43400640 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 0x56022afd99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.295+0000 7f9a48c00640 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 0x56022afc1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.295+0000 7f9a43e00640 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000c00 tx=0x7f9a3400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a38180b8887607 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.296+0000 7f9a2fe00640 1 -- 172.21.15.59:0/3105081130 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34032080 con 0x56022afc7ed0 2024-04-02T17:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.297+0000 7f9a2fe00640 1 -- 172.21.15.59:0/3105081130 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3402cab0 con 0x56022afc7ed0 2024-04-02T17:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.297+0000 7f9a2fe00640 1 -- 172.21.15.59:0/3105081130 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34016070 con 0x56022afc7ed0 2024-04-02T17:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/3105081130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 msgr2=0x56022afd2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9a34000c00 tx=0x7f9a3400cc40 comp rx=0 tx=0).stop 2024-04-02T17:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/3105081130 shutdown_connections 2024-04-02T17:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afcda30 0x56022afc1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afc88c0 0x56022afd99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/3105081130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd2790 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.298+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/3105081130 >> 172.21.15.59:0/3105081130 conn(0x56022afc3440 msgr2=0x56022afd2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.299+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/3105081130 shutdown_connections 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.299+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/3105081130 wait complete. 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.299+0000 7f9a4d9e7740 1 -- start start 2024-04-02T17:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 Processor -- start 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441c1eb0 con 0x56022afc88c0 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441c1720 con 0x56022afc7ed0 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441be7e0 con 0x56022afcda30 2024-04-02T17:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.300+0000 7f9a4d9e7740 10 client.0 _ll_register_callbacks cb 0x7f9a34010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a48c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:36776/0 (socket says 172.21.15.59:36776) 2024-04-02T17:37:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a48c00640 1 -- 172.21.15.59:0/4235814790 learned_addr learned my addr 172.21.15.59:0/4235814790 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a43400640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a48c00640 1 -- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 msgr2=0x56022afd6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.301+0000 7f9a48c00640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a48c00640 1 -- 172.21.15.59:0/4235814790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 msgr2=0x56022afd7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a48c00640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a48c00640 1 -- 172.21.15.59:0/4235814790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022afd8950 con 0x56022afcda30 2024-04-02T17:37:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a43400640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:37:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a48c00640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009430 tx=0x7f9a3c005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5cfa839b12b96d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.302+0000 7f9a2d600640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c04d060 con 0x56022afcda30 2024-04-02T17:37:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f9a3c003970 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3402cab0 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9a3c043030 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a4d9e7740 10 client.27811 fetch_fsmap learned FSMap version 67 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x56022afd9c40 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.303+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3402cab0 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 67) ==== 8406+0+0 (secure 0 0 0) 0x7f9a3c051030 con 0x56022afcda30 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 10 client.27811 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 10 client.27811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 20 client.27811 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022b044c00 0x56022b047020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 4 client.27811 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:37:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56022af3b2b0 con 0x56022b044c00 2024-04-02T17:37:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.304+0000 7f9a43e00640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022b044c00 0x56022b047020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.305+0000 7f9a43e00640 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022b044c00 0x56022b047020 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26924c258341de34 server_cookie=2e50cc88bc09014 in_seq=0 out_seq=0 2024-04-02T17:37:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.305+0000 7f9a2d600640 10 client.27811 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:25.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:25 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:25 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:25.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:25 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a2d600640 1 -- 172.21.15.59:0/4235814790 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56022af3b2b0 con 0x56022b044c00 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a2d600640 10 client.27811 handle_command_reply: tid=0 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 client.27811 shutdown 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 msgr2=0x56022afd8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009430 tx=0x7f9a3c005c60 comp rx=0 tx=0).stop 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 shutdown_connections 2024-04-02T17:37:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x56022b044c00 0x56022b047020 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56022afc88c0 0x56022afd7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56022afcda30 0x56022afd8360 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 --2- 172.21.15.59:0/4235814790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56022afc7ed0 0x56022afd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 >> 172.21.15.59:0/4235814790 conn(0x56022afc3440 msgr2=0x56022afc66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.620+0000 7f9a2d600640 0 client.27811 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.621+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 shutdown_connections 2024-04-02T17:37:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.621+0000 7f9a4d9e7740 1 -- 172.21.15.59:0/4235814790 wait complete. 2024-04-02T17:37:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.621+0000 7f9a4d9e7740 20 client.27811 trim_cache size 0 max 16384 2024-04-02T17:37:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:37:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:37:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:37:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 msgr2=0x7f9a280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 0x7f9a280a6210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x56022afc43b0 tx=0x7f9a340046c0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 msgr2=0x7f9a441155f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a441155f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9f0 tx=0x7f9a3000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 shutdown_connections 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9a280a3d50 0x7f9a280a6210 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a4410de20 0x7f9a441b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4410b470 0x7f9a441155f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 --2- 172.21.15.59:0/746682427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a44108b00 0x7f9a441150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 >> 172.21.15.59:0/746682427 conn(0x7f9a4406eb40 msgr2=0x7f9a44109780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 shutdown_connections 2024-04-02T17:37:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:25.622+0000 7f9a4ac00640 1 -- 172.21.15.59:0/746682427 wait complete. 2024-04-02T17:37:25.668 INFO:tasks.workunit.client.0.smithi059.stdout: 10 459031 36.69 MB/sec execute 532 sec latency 221.765 ms 2024-04-02T17:37:26.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:26 smithi164 ceph-mon[36499]: pgmap v1120: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 609 op/s 2024-04-02T17:37:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:26 smithi154 ceph-mon[36479]: pgmap v1120: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 609 op/s 2024-04-02T17:37:26.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:26 smithi059 ceph-mon[31664]: pgmap v1120: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 609 op/s 2024-04-02T17:37:26.668 INFO:tasks.workunit.client.0.smithi059.stdout: 10 459682 36.69 MB/sec execute 533 sec latency 99.685 ms 2024-04-02T17:37:27.670 INFO:tasks.workunit.client.0.smithi059.stdout: 10 460389 36.69 MB/sec execute 534 sec latency 105.387 ms 2024-04-02T17:37:28.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:28 smithi164 ceph-mon[36499]: pgmap v1121: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-04-02T17:37:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:28 smithi154 ceph-mon[36479]: pgmap v1121: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-04-02T17:37:28.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:28 smithi059 ceph-mon[31664]: pgmap v1121: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-04-02T17:37:28.671 INFO:tasks.workunit.client.0.smithi059.stdout: 10 460567 36.64 MB/sec execute 535 sec latency 618.310 ms 2024-04-02T17:37:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 461018 36.62 MB/sec execute 536 sec latency 89.107 ms 2024-04-02T17:37:30.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:30 smithi164 ceph-mon[36499]: pgmap v1122: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-04-02T17:37:30.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:30 smithi154 ceph-mon[36479]: pgmap v1122: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-04-02T17:37:30.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:30 smithi059 ceph-mon[31664]: pgmap v1122: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-04-02T17:37:30.671 INFO:tasks.workunit.client.0.smithi059.stdout: 10 461614 36.64 MB/sec execute 537 sec latency 50.075 ms 2024-04-02T17:37:31.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 462285 36.64 MB/sec execute 538 sec latency 46.739 ms 2024-04-02T17:37:32.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:32 smithi164 ceph-mon[36499]: pgmap v1123: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-04-02T17:37:32.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:32 smithi154 ceph-mon[36479]: pgmap v1123: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-04-02T17:37:32.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:32 smithi059 ceph-mon[31664]: pgmap v1123: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-04-02T17:37:32.671 INFO:tasks.workunit.client.0.smithi059.stdout: 10 462999 36.64 MB/sec execute 539 sec latency 111.901 ms 2024-04-02T17:37:33.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 463409 36.61 MB/sec execute 540 sec latency 433.294 ms 2024-04-02T17:37:34.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:34 smithi154 ceph-mon[36479]: pgmap v1124: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-04-02T17:37:34.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:34 smithi059 ceph-mon[31664]: pgmap v1124: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-04-02T17:37:34.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:34 smithi164 ceph-mon[36499]: pgmap v1124: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-04-02T17:37:34.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 463587 36.57 MB/sec execute 541 sec latency 1065.612 ms 2024-04-02T17:37:35.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 463982 36.54 MB/sec execute 542 sec latency 136.016 ms 2024-04-02T17:37:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:36 smithi154 ceph-mon[36479]: pgmap v1125: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.9 MiB/s wr, 683 op/s 2024-04-02T17:37:36.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:36 smithi059 ceph-mon[31664]: pgmap v1125: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.9 MiB/s wr, 683 op/s 2024-04-02T17:37:36.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:36 smithi164 ceph-mon[36499]: pgmap v1125: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.9 MiB/s wr, 683 op/s 2024-04-02T17:37:36.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 464486 36.54 MB/sec execute 543 sec latency 171.207 ms 2024-04-02T17:37:37.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 465016 36.51 MB/sec execute 544 sec latency 62.096 ms 2024-04-02T17:37:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:38 smithi154 ceph-mon[36479]: pgmap v1126: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 688 op/s 2024-04-02T17:37:38.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:38 smithi059 ceph-mon[31664]: pgmap v1126: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 688 op/s 2024-04-02T17:37:38.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:38 smithi164 ceph-mon[36499]: pgmap v1126: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 688 op/s 2024-04-02T17:37:38.673 INFO:tasks.workunit.client.0.smithi059.stdout: 10 465593 36.51 MB/sec execute 545 sec latency 82.329 ms 2024-04-02T17:37:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.831+0000 7fa1c3a00640 1 -- 172.21.15.59:0/1579089866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc0744e0 msgr2=0x7fa1bc074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.831+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/1579089866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc074940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802bcd0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.832+0000 7fa1c3a00640 1 -- 172.21.15.59:0/1579089866 shutdown_connections 2024-04-02T17:37:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.832+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/1579089866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc1099e0 0x7fa1bc10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.832+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/1579089866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc074940 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.832+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/1579089866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc075ac0 0x7fa1bc073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.832+0000 7fa1c3a00640 1 -- 172.21.15.59:0/1579089866 >> 172.21.15.59:0/1579089866 conn(0x7fa1bc0fd0c0 msgr2=0x7fa1bc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.833+0000 7fa1c3a00640 1 -- 172.21.15.59:0/1579089866 shutdown_connections 2024-04-02T17:37:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.833+0000 7fa1c3a00640 1 -- 172.21.15.59:0/1579089866 wait complete. 2024-04-02T17:37:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.835+0000 7fa1c3a00640 1 Processor -- start 2024-04-02T17:37:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.835+0000 7fa1c3a00640 1 -- start start 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc075ac0 0x7fa1bc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc1099e0 0x7fa1bc1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc101b70 con 0x7fa1bc075ac0 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1019f0 con 0x7fa1bc1099e0 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c3a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc101cf0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44206/0 (socket says 172.21.15.59:44206) 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c1000640 1 -- 172.21.15.59:0/3092827239 learned_addr learned my addr 172.21.15.59:0/3092827239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.836+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc1099e0 0x7fa1bc1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 -- 172.21.15.59:0/3092827239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc1099e0 msgr2=0x7fa1bc1a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc1099e0 0x7fa1bc1a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 -- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc075ac0 msgr2=0x7fa1bc19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc075ac0 0x7fa1bc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 -- 172.21.15.59:0/3092827239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc1a5ee0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.837+0000 7fa1c1000640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c920 tx=0x7fa1ac00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.838+0000 7fa1b9600640 1 -- 172.21.15.59:0/3092827239 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac007bb0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.838+0000 7fa1b9600640 1 -- 172.21.15.59:0/3092827239 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac007d50 con 0x7fa1bc0744e0 2024-04-02T17:37:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.838+0000 7fa1b9600640 1 -- 172.21.15.59:0/3092827239 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0187d0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.838+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc1a61d0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.839+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a66d0 con 0x7fa1bc0744e0 2024-04-02T17:37:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.846+0000 7fa1b9600640 1 -- 172.21.15.59:0/3092827239 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1ac00f450 con 0x7fa1bc0744e0 2024-04-02T17:37:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.846+0000 7fa1c6613740 1 Processor -- start 2024-04-02T17:37:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.846+0000 7fa1c6613740 1 -- start start 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b181642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b1816bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b180c9d70 con 0x564b18169a30 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564b180722a0 con 0x564b18163ed0 2024-04-02T17:37:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.847+0000 7fa1c6613740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b180c5d20 con 0x564b181648c0 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b181642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44210/0 (socket says 172.21.15.59:44210) 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 -- 172.21.15.59:0/2196171837 learned_addr learned my addr 172.21.15.59:0/2196171837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b1816bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 -- 172.21.15.59:0/2196171837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 msgr2=0x564b181642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b181642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 -- 172.21.15.59:0/2196171837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 msgr2=0x564b1816bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b1816bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1c1000640 1 -- 172.21.15.59:0/2196171837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b18086840 con 0x564b181648c0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.848+0000 7fa1b9600640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 0x7fa1a00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0d6600 tx=0x7fa1ac0d66f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ae5c2e5597e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1b9600640 1 -- 172.21.15.59:0/3092827239 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa1ac0d0490 con 0x7fa1bc0744e0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1b2000640 1 -- 172.21.15.59:0/2196171837 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0989b0 con 0x564b181648c0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1b2000640 1 -- 172.21.15.59:0/2196171837 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac09ce10 con 0x564b181648c0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1b2000640 1 -- 172.21.15.59:0/2196171837 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac09c070 con 0x564b181648c0 2024-04-02T17:37:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.849+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 0x7fa1a00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 0x7fa1a00a62d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x564b18165120 tx=0x7fa1b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 -- 172.21.15.59:0/2196171837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 msgr2=0x564b181693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0d6600 tx=0x7fa1ac0d66f0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 -- 172.21.15.59:0/2196171837 shutdown_connections 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b1816bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181693c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2196171837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b181642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.851+0000 7fa1c6613740 1 -- 172.21.15.59:0/2196171837 >> 172.21.15.59:0/2196171837 conn(0x564b1815f440 msgr2=0x564b1815f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.852+0000 7fa1c6613740 1 -- 172.21.15.59:0/2196171837 shutdown_connections 2024-04-02T17:37:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.852+0000 7fa1c6613740 1 -- 172.21.15.59:0/2196171837 wait complete. 2024-04-02T17:37:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 -- start start 2024-04-02T17:37:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 Processor -- start 2024-04-02T17:37:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 0x564b1816e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.853+0000 7fa1c6613740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b1816c9e0 con 0x564b18163ed0 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c6613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564b18085660 con 0x564b181648c0 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c6613740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564b1816cb60 con 0x564b18169a30 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c6613740 10 client.0 _ll_register_callbacks cb 0x564b18085050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44218/0 (socket says 172.21.15.59:44218) 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 -- 172.21.15.59:0/2583005667 learned_addr learned my addr 172.21.15.59:0/2583005667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 0x564b1816e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 -- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 msgr2=0x564b1816e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 0x564b1816e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 -- 172.21.15.59:0/2583005667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 msgr2=0x564b1816e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.854+0000 7fa1c1a00640 1 -- 172.21.15.59:0/2583005667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b18094e80 con 0x564b18169a30 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 0x564b1816e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4003fd0 tx=0x7fa1b4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf49caef2e305b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400f8f0 con 0x564b18169a30 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1ac09ce10 con 0x564b18169a30 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.855+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa1b400fa90 con 0x564b18169a30 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.856+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa1b400fc90 con 0x564b18169a30 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.856+0000 7fa1c6613740 10 client.27820 fetch_fsmap learned FSMap version 67 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.856+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x564b17dde210 con 0x564b18169a30 2024-04-02T17:37:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.856+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac09ce10 con 0x564b18169a30 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 67) ==== 8406+0+0 (secure 0 0 0) 0x7fa1b4031020 con 0x564b18169a30 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 10 client.27820 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 10 client.27820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 20 client.27820 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b1817a400 0x564b1817c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 4 client.27820 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.857+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b18079cd0 con 0x564b1817a400 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.858+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b1817a400 0x564b1817c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.858+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b1817a400 0x564b1817c840 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cc12ff9cbb0e3 server_cookie=9e9dd3539cc58087 in_seq=0 out_seq=0 2024-04-02T17:37:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.858+0000 7fa1a6a00640 10 client.27820 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.859+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2583005667 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b18079cd0 con 0x564b1817a400 2024-04-02T17:37:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.859+0000 7fa1a6a00640 10 client.27820 handle_command_reply: tid=0 2024-04-02T17:37:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.859+0000 7fa1c6613740 1 client.27820 shutdown 2024-04-02T17:37:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 msgr2=0x564b18175990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4003fd0 tx=0x7fa1b4004ab0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 shutdown_connections 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b1817a400 0x564b1817c840 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b18169a30 0x564b18175990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b181648c0 0x564b1816e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2583005667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 >> 172.21.15.59:0/2583005667 conn(0x564b1815f440 msgr2=0x564b1815e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.860+0000 7fa1a6a00640 0 client.27820 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.861+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 shutdown_connections 2024-04-02T17:37:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.861+0000 7fa1c6613740 1 -- 172.21.15.59:0/2583005667 wait complete. 2024-04-02T17:37:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.861+0000 7fa1c6613740 20 client.27820 trim_cache size 0 max 16384 2024-04-02T17:37:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.895+0000 7fa1c6613740 1 Processor -- start 2024-04-02T17:37:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.895+0000 7fa1c6613740 1 -- start start 2024-04-02T17:37:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18169a30 0x564b18175950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1a6e70 con 0x564b18163ed0 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1a8bf0 con 0x564b18169a30 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c6613740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1a7600 con 0x564b181648c0 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1c1a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18169a30 0x564b18175950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.896+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44222/0 (socket says 172.21.15.59:44222) 2024-04-02T17:37:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 -- 172.21.15.59:0/1217897892 learned_addr learned my addr 172.21.15.59:0/1217897892 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 -- 172.21.15.59:0/1217897892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18169a30 msgr2=0x564b18175950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18169a30 0x564b18175950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 -- 172.21.15.59:0/1217897892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 msgr2=0x564b1816cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.897+0000 7fa1bbe00640 1 -- 172.21.15.59:0/1217897892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b17eaa620 con 0x564b181648c0 2024-04-02T17:37:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.898+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa14bfd3c8519ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.898+0000 7fa1b1600640 1 -- 172.21.15.59:0/1217897892 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80043b0 con 0x564b181648c0 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.899+0000 7fa1b1600640 1 -- 172.21.15.59:0/1217897892 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8005510 con 0x564b181648c0 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.899+0000 7fa1b1600640 1 -- 172.21.15.59:0/1217897892 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8003850 con 0x564b181648c0 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.899+0000 7fa1c6613740 1 -- 172.21.15.59:0/1217897892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 msgr2=0x564b1816d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.899+0000 7fa1c6613740 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a8004040 comp rx=0 tx=0).stop 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 -- 172.21.15.59:0/1217897892 shutdown_connections 2024-04-02T17:37:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18169a30 0x564b18175950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b1816d440 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 --2- 172.21.15.59:0/1217897892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18163ed0 0x564b1816cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 -- 172.21.15.59:0/1217897892 >> 172.21.15.59:0/1217897892 conn(0x564b1815f440 msgr2=0x564b1816e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 -- 172.21.15.59:0/1217897892 shutdown_connections 2024-04-02T17:37:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.900+0000 7fa1c6613740 1 -- 172.21.15.59:0/1217897892 wait complete. 2024-04-02T17:37:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.901+0000 7fa1c6613740 1 -- start start 2024-04-02T17:37:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.901+0000 7fa1c6613740 1 Processor -- start 2024-04-02T17:37:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b18175910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b181728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1a8520 con 0x564b18169a30 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1a7d90 con 0x564b18163ed0 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc072190 con 0x564b181648c0 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c6613740 10 client.0 _ll_register_callbacks cb 0x7fa1b4003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44224/0 (socket says 172.21.15.59:44224) 2024-04-02T17:37:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 -- 172.21.15.59:0/2855522876 learned_addr learned my addr 172.21.15.59:0/2855522876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b18175910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 -- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 msgr2=0x564b18175910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b181728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b18175910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 -- 172.21.15.59:0/2855522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 msgr2=0x564b181728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b181728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.902+0000 7fa1bbe00640 1 -- 172.21.15.59:0/2855522876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b181ef9c0 con 0x564b181648c0 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.903+0000 7fa1c1a00640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b181728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.903+0000 7fa1bbe00640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80096c0 tx=0x7fa1a802c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82bb8a9bc226787 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.903+0000 7fa1a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.904+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8005270 con 0x564b181648c0 2024-04-02T17:37:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.904+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa1a80035f0 con 0x564b181648c0 2024-04-02T17:37:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.905+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a8005510 con 0x564b181648c0 2024-04-02T17:37:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.905+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa1a80037f0 con 0x564b181648c0 2024-04-02T17:37:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.906+0000 7fa1c6613740 10 client.27826 fetch_fsmap learned FSMap version 67 2024-04-02T17:37:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.906+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x564b181f0cb0 con 0x564b181648c0 2024-04-02T17:37:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8005510 con 0x564b181648c0 2024-04-02T17:37:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 67) ==== 8406+0+0 (secure 0 0 0) 0x7fa1a803f9d0 con 0x564b181648c0 2024-04-02T17:37:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1c6613740 10 client.27826 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T17:37:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1c6613740 10 client.27826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1c6613740 20 client.27826 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.907+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b181f16b0 0x564b181f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.908+0000 7fa1c6613740 4 client.27826 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:37:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.908+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b180d72b0 con 0x564b181f16b0 2024-04-02T17:37:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.908+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b181f16b0 0x564b181f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.909+0000 7fa1c1000640 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b181f16b0 0x564b181f3ad0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d3b4a5e4a6f847 server_cookie=192d6982841ea2b3 in_seq=0 out_seq=0 2024-04-02T17:37:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:38.909+0000 7fa1a6a00640 10 client.27826 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.179+0000 7fa1a6a00640 1 -- 172.21.15.59:0/2855522876 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b180d72b0 con 0x564b181f16b0 2024-04-02T17:37:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.179+0000 7fa1a6a00640 10 client.27826 handle_command_reply: tid=0 2024-04-02T17:37:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.179+0000 7fa1c6613740 1 client.27826 shutdown 2024-04-02T17:37:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 msgr2=0x564b181723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80096c0 tx=0x7fa1a802c6e0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 shutdown_connections 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564b181f16b0 0x564b181f3ad0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b18169a30 0x564b181728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564b181648c0 0x564b181723b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 --2- 172.21.15.59:0/2855522876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564b18163ed0 0x564b18175910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 >> 172.21.15.59:0/2855522876 conn(0x564b1815f440 msgr2=0x564b1816e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.180+0000 7fa1a6a00640 0 client.27826 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.181+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 shutdown_connections 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.181+0000 7fa1c6613740 1 -- 172.21.15.59:0/2855522876 wait complete. 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.181+0000 7fa1c6613740 20 client.27826 trim_cache size 0 max 16384 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:37:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:37:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:37:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.183+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 msgr2=0x7fa1a00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.183+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 0x7fa1a00a62d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x564b18165120 tx=0x7fa1b4009500 comp rx=0 tx=0).stop 2024-04-02T17:37:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.183+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 msgr2=0x7fa1bc19e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.183+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c920 tx=0x7fa1ac00cd20 comp rx=0 tx=0).stop 2024-04-02T17:37:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 shutdown_connections 2024-04-02T17:37:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa1a00a3e10 0x7fa1a00a62d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc075ac0 0x7fa1bc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1bc1099e0 0x7fa1bc1a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 --2- 172.21.15.59:0/3092827239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1bc0744e0 0x7fa1bc19e2c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 >> 172.21.15.59:0/3092827239 conn(0x7fa1bc0fd0c0 msgr2=0x7fa1bc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 shutdown_connections 2024-04-02T17:37:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:39.185+0000 7fa1c3a00640 1 -- 172.21.15.59:0/3092827239 wait complete. 2024-04-02T17:37:39.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 466191 36.51 MB/sec execute 546 sec latency 187.521 ms 2024-04-02T17:37:40.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:40 smithi154 ceph-mon[36479]: pgmap v1127: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 754 op/s 2024-04-02T17:37:40.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:40 smithi059 ceph-mon[31664]: pgmap v1127: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 754 op/s 2024-04-02T17:37:40.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:40 smithi164 ceph-mon[36499]: pgmap v1127: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 754 op/s 2024-04-02T17:37:40.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 466973 36.53 MB/sec execute 547 sec latency 67.405 ms 2024-04-02T17:37:41.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:41 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:41.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:41 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:41.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:41 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:41.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 467831 36.56 MB/sec execute 548 sec latency 38.258 ms 2024-04-02T17:37:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:42 smithi154 ceph-mon[36479]: pgmap v1128: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 852 op/s 2024-04-02T17:37:42.449 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:42 smithi059 ceph-mon[31664]: pgmap v1128: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 852 op/s 2024-04-02T17:37:42.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:42 smithi164 ceph-mon[36499]: pgmap v1128: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 852 op/s 2024-04-02T17:37:42.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 468529 36.57 MB/sec execute 549 sec latency 52.954 ms 2024-04-02T17:37:43.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 469071 36.54 MB/sec execute 550 sec latency 65.932 ms 2024-04-02T17:37:44.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:44 smithi164 ceph-mon[36499]: pgmap v1129: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 721 op/s 2024-04-02T17:37:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:44 smithi154 ceph-mon[36479]: pgmap v1129: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 721 op/s 2024-04-02T17:37:44.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:44 smithi059 ceph-mon[31664]: pgmap v1129: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 721 op/s 2024-04-02T17:37:44.673 INFO:tasks.workunit.client.0.smithi059.stdout: 10 469627 36.53 MB/sec execute 551 sec latency 256.680 ms 2024-04-02T17:37:45.672 INFO:tasks.workunit.client.0.smithi059.stdout: 10 470091 36.51 MB/sec execute 552 sec latency 485.767 ms 2024-04-02T17:37:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:46 smithi154 ceph-mon[36479]: pgmap v1130: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 767 op/s 2024-04-02T17:37:46.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:46 smithi059 ceph-mon[31664]: pgmap v1130: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 767 op/s 2024-04-02T17:37:46.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:46 smithi164 ceph-mon[36499]: pgmap v1130: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 767 op/s 2024-04-02T17:37:46.674 INFO:tasks.workunit.client.0.smithi059.stdout: 10 470989 36.57 MB/sec execute 553 sec latency 37.262 ms 2024-04-02T17:37:47.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 471712 36.58 MB/sec execute 554 sec latency 42.943 ms 2024-04-02T17:37:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:48 smithi154 ceph-mon[36479]: pgmap v1131: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-02T17:37:48.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:48 smithi059 ceph-mon[31664]: pgmap v1131: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-02T17:37:48.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:48 smithi164 ceph-mon[36499]: pgmap v1131: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-02T17:37:48.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 472209 36.55 MB/sec execute 555 sec latency 83.258 ms 2024-04-02T17:37:49.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 472918 36.55 MB/sec execute 556 sec latency 124.328 ms 2024-04-02T17:37:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:50 smithi154 ceph-mon[36479]: pgmap v1132: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 621 op/s 2024-04-02T17:37:50.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:50 smithi059 ceph-mon[31664]: pgmap v1132: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 621 op/s 2024-04-02T17:37:50.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:50 smithi164 ceph-mon[36499]: pgmap v1132: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 621 op/s 2024-04-02T17:37:50.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 473699 36.56 MB/sec execute 557 sec latency 70.768 ms 2024-04-02T17:37:51.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:51 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:51.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:51 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:51.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:51 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:37:51.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 474502 36.57 MB/sec execute 558 sec latency 57.562 ms 2024-04-02T17:37:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 -- 172.21.15.59:0/2885034596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 msgr2=0x7fc5ec0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/2885034596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec0734c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009900 tx=0x7fc5d802bc30 comp rx=0 tx=0).stop 2024-04-02T17:37:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 -- 172.21.15.59:0/2885034596 shutdown_connections 2024-04-02T17:37:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/2885034596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec10fe40 0x7fc5ec112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/2885034596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec0734c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/2885034596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 -- 172.21.15.59:0/2885034596 >> 172.21.15.59:0/2885034596 conn(0x7fc5ec06e970 msgr2=0x7fc5ec070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 -- 172.21.15.59:0/2885034596 shutdown_connections 2024-04-02T17:37:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.375+0000 7fc5f2a00640 1 -- 172.21.15.59:0/2885034596 wait complete. 2024-04-02T17:37:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.376+0000 7fc5f2a00640 1 Processor -- start 2024-04-02T17:37:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.376+0000 7fc5f2a00640 1 -- start start 2024-04-02T17:37:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec1a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec10fe40 0x7fc5ec1a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec115e60 con 0x7fc5ec1096e0 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec115ce0 con 0x7fc5ec10fe40 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec115fe0 con 0x7fc5ec073040 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54654/0 (socket says 172.21.15.59:54654) 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f1600640 1 -- 172.21.15.59:0/4076663344 learned_addr learned my addr 172.21.15.59:0/4076663344 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec1a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.377+0000 7fc5f2000640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec10fe40 0x7fc5ec1a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 -- 172.21.15.59:0/4076663344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec10fe40 msgr2=0x7fc5ec1a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec10fe40 0x7fc5ec1a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 -- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 msgr2=0x7fc5ec1a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec1a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 -- 172.21.15.59:0/4076663344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ec1aa2f0 con 0x7fc5ec073040 2024-04-02T17:37:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec1a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.378+0000 7fc5f1600640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e9f0 tx=0x7fc5e000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.379+0000 7fc5ea000640 1 -- 172.21.15.59:0/4076663344 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000cd80 con 0x7fc5ec073040 2024-04-02T17:37:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.379+0000 7fc5ea000640 1 -- 172.21.15.59:0/4076663344 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e0005500 con 0x7fc5ec073040 2024-04-02T17:37:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.379+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ec1aa5e0 con 0x7fc5ec073040 2024-04-02T17:37:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.379+0000 7fc5ea000640 1 -- 172.21.15.59:0/4076663344 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0010690 con 0x7fc5ec073040 2024-04-02T17:37:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.379+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ec1aaab0 con 0x7fc5ec073040 2024-04-02T17:37:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.383+0000 7fc5f6dc1740 1 Processor -- start 2024-04-02T17:37:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.383+0000 7fc5ea000640 1 -- 172.21.15.59:0/4076663344 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc5e00046c0 con 0x7fc5ec073040 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.383+0000 7fc5f6dc1740 1 -- start start 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d4bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d53900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564275ca8d70 con 0x564275d4b910 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564275c512a0 con 0x564275d4c300 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564275ca4d20 con 0x564275d51470 2024-04-02T17:37:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d4bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f1600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.384+0000 7fc5f0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d53900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.385+0000 7fc5f1600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50030/0 (socket says 172.21.15.59:50030) 2024-04-02T17:37:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.385+0000 7fc5f1600640 1 -- 172.21.15.59:0/2377565185 learned_addr learned my addr 172.21.15.59:0/2377565185 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.386+0000 7fc5f1600640 1 -- 172.21.15.59:0/2377565185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 msgr2=0x564275d53900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.386+0000 7fc5f1600640 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d53900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.386+0000 7fc5f1600640 1 -- 172.21.15.59:0/2377565185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 msgr2=0x564275d4bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.386+0000 7fc5f1600640 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d4bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.386+0000 7fc5f1600640 1 -- 172.21.15.59:0/2377565185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275c65840 con 0x564275d4c300 2024-04-02T17:37:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.387+0000 7fc5f1600640 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00d6a90 tx=0x7fc5e00d6b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570a90f2113b1bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.388+0000 7fc5ea000640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 0x7fc5d40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.388+0000 7fc5ea000640 1 -- 172.21.15.59:0/4076663344 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc5e00d08c0 con 0x7fc5ec073040 2024-04-02T17:37:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.388+0000 7fc5de600640 1 -- 172.21.15.59:0/2377565185 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00df070 con 0x564275d4c300 2024-04-02T17:37:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.388+0000 7fc5de600640 1 -- 172.21.15.59:0/2377565185 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e009d1e0 con 0x564275d4c300 2024-04-02T17:37:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.388+0000 7fc5de600640 1 -- 172.21.15.59:0/2377565185 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e009c620 con 0x564275d4c300 2024-04-02T17:37:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.389+0000 7fc5f2000640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 0x7fc5d40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2377565185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 msgr2=0x564275d50e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00d6a90 tx=0x7fc5e00d6b80 comp rx=0 tx=0).stop 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2377565185 shutdown_connections 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d53900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d50e00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2377565185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d4bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2377565185 >> 172.21.15.59:0/2377565185 conn(0x564275d3e670 msgr2=0x564275d3ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.390+0000 7fc5f2000640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 0x7fc5d40a6060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x564275d4cb60 tx=0x7fc5e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.391+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2377565185 shutdown_connections 2024-04-02T17:37:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.391+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2377565185 wait complete. 2024-04-02T17:37:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.392+0000 7fc5f6dc1740 1 -- start start 2024-04-02T17:37:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.392+0000 7fc5f6dc1740 1 Processor -- start 2024-04-02T17:37:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d5d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564275d54420 con 0x564275d4b910 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564275c64660 con 0x564275d4c300 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564275d545a0 con 0x564275d51470 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.393+0000 7fc5f6dc1740 10 client.0 _ll_register_callbacks cb 0x564275c64050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50044/0 (socket says 172.21.15.59:50044) 2024-04-02T17:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 -- 172.21.15.59:0/826357765 learned_addr learned my addr 172.21.15.59:0/826357765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 -- 172.21.15.59:0/826357765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 msgr2=0x564275d5d440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d5d440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f0c00640 1 -- 172.21.15.59:0/826357765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 msgr2=0x564275d55ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.394+0000 7fc5f1600640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.395+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d55ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.395+0000 7fc5f0c00640 1 -- 172.21.15.59:0/826357765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564275c73e80 con 0x564275d4c300 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.395+0000 7fc5f1600640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.395+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009900 tx=0x7fc5d80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1760595b8f940094 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.396+0000 7fc5d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.396+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8005d50 con 0x564275d4c300 2024-04-02T17:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.396+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc5d8004e70 con 0x564275d4c300 2024-04-02T17:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.396+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e009d1e0 con 0x564275d4c300 2024-04-02T17:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.397+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc5d802cd30 con 0x564275d4c300 2024-04-02T17:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.397+0000 7fc5f6dc1740 10 client.27776 fetch_fsmap learned FSMap version 69 2024-04-02T17:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.397+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5642759bd210 con 0x564275d4c300 2024-04-02T17:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e009d1e0 con 0x564275d4c300 2024-04-02T17:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) ==== 8418+0+0 (secure 0 0 0) 0x7fc5d8003ad0 con 0x564275d4c300 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 10 client.27776 fetch_fsmap finished waiting for FSMap version 69 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 10 client.27776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 20 client.27776 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275d64240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 4 client.27776 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.398+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564275c58cd0 con 0x564275d61de0 2024-04-02T17:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.399+0000 7fc5f1600640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275d64240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.400+0000 7fc5f1600640 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275d64240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4189a22e5fc7b51e server_cookie=ba0da41442eab776 in_seq=0 out_seq=0 2024-04-02T17:37:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.400+0000 7fc5d2a00640 10 client.27776 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:52.398 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:52 smithi154 ceph-mon[36479]: pgmap v1133: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.401+0000 7fc5d2a00640 1 -- 172.21.15.59:0/826357765 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564275c58cd0 con 0x564275d61de0 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.401+0000 7fc5d2a00640 10 client.27776 handle_command_reply: tid=0 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.401+0000 7fc5f6dc1740 1 client.27776 shutdown 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.401+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 msgr2=0x564275d56400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009900 tx=0x7fc5d80048f0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 shutdown_connections 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275d64240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5d2a00640 0 client.27776 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d51470 0x564275d5d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d56400 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/826357765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 >> 172.21.15.59:0/826357765 conn(0x564275d3e670 msgr2=0x564275d3d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 shutdown_connections 2024-04-02T17:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/826357765 wait complete. 2024-04-02T17:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.402+0000 7fc5f6dc1740 20 client.27776 trim_cache size 0 max 16384 2024-04-02T17:37:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.425+0000 7fc5f6dc1740 1 Processor -- start 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.425+0000 7fc5f6dc1740 1 -- start start 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4b910 0x564275d54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1af730 con 0x564275d51470 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1b14b0 con 0x564275d4c300 2024-04-02T17:37:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1afec0 con 0x564275d4b910 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51592/0 (socket says 172.21.15.59:51592) 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f2000640 1 -- 172.21.15.59:0/2393162688 learned_addr learned my addr 172.21.15.59:0/2393162688 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.426+0000 7fc5f1600640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4b910 0x564275d54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f2000640 1 -- 172.21.15.59:0/2393162688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4b910 msgr2=0x564275d54940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f2000640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4b910 0x564275d54940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f2000640 1 -- 172.21.15.59:0/2393162688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 msgr2=0x564275d5d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f2000640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d5d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f2000640 1 -- 172.21.15.59:0/2393162688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275a89620 con 0x564275d51470 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.427+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.428+0000 7fc5f2000640 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009cd0 tx=0x7fc5e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1429610d80a04cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.428+0000 7fc5ddc00640 1 -- 172.21.15.59:0/2393162688 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400f4e0 con 0x564275d51470 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.429+0000 7fc5ddc00640 1 -- 172.21.15.59:0/2393162688 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e400fb60 con 0x564275d51470 2024-04-02T17:37:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.429+0000 7fc5ddc00640 1 -- 172.21.15.59:0/2393162688 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4028d80 con 0x564275d51470 2024-04-02T17:37:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2393162688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 msgr2=0x564275d54e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009cd0 tx=0x7fc5e4016000 comp rx=0 tx=0).stop 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2393162688 shutdown_connections 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d51470 0x564275d54e80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d4c300 0x564275d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/2393162688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4b910 0x564275d54940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2393162688 >> 172.21.15.59:0/2393162688 conn(0x564275d3e670 msgr2=0x564275d4d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2393162688 shutdown_connections 2024-04-02T17:37:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.430+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/2393162688 wait complete. 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.431+0000 7fc5f6dc1740 1 -- start start 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.431+0000 7fc5f6dc1740 1 Processor -- start 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.431+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 0x564275d5bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1b0de0 con 0x564275d4b910 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1b0650 con 0x564275d51470 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1ad710 con 0x564275d4c300 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f6dc1740 10 client.0 _ll_register_callbacks cb 0x7fc5d8004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:54702/0 (socket says 172.21.15.59:54702) 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f0c00640 1 -- 172.21.15.59:0/1427122745 learned_addr learned my addr 172.21.15.59:0/1427122745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:37:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.432+0000 7fc5f2000640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 0x564275d5bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 -- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 msgr2=0x564275d5bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 0x564275d5bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 -- 172.21.15.59:0/1427122745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 msgr2=0x564275d5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 -- 172.21.15.59:0/1427122745 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564275d5c2a0 con 0x564275d4c300 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f1600640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f2000640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 0x564275d5bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f0c00640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5d802c170 tx=0x7fc5d8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca47be132f1fbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d802cd30 con 0x564275d4c300 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc5d8004e70 con 0x564275d4c300 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.433+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e400fb60 con 0x564275d4c300 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.434+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc5d8005070 con 0x564275d4c300 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.434+0000 7fc5f6dc1740 10 client.27841 fetch_fsmap learned FSMap version 69 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.434+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x564275d5d5c0 con 0x564275d4c300 2024-04-02T17:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.434+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e400fb60 con 0x564275d4c300 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 69) ==== 8418+0+0 (secure 0 0 0) 0x7fc5d8066050 con 0x564275d4c300 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 10 client.27841 fetch_fsmap finished waiting for FSMap version 69 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 10 client.27841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 20 client.27841 populate_metadata read hostname 'smithi059' 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275dcaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 4 client.27841 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564275cb62b0 con 0x564275d61de0 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.435+0000 7fc5f1600640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275dcaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.436+0000 7fc5f1600640 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275dcaa80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b470e5cbee5221c server_cookie=29e94494fb86c05c in_seq=0 out_seq=0 2024-04-02T17:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.436+0000 7fc5d2a00640 10 client.27841 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:52.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:52 smithi059 ceph-mon[31664]: pgmap v1133: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-04-02T17:37:52.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:52 smithi164 ceph-mon[36499]: pgmap v1133: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-04-02T17:37:52.675 INFO:tasks.workunit.client.0.smithi059.stdout: 10 475007 36.57 MB/sec execute 559 sec latency 186.138 ms 2024-04-02T17:37:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5d2a00640 1 -- 172.21.15.59:0/1427122745 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564275cb62b0 con 0x564275d61de0 2024-04-02T17:37:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5d2a00640 10 client.27841 handle_command_reply: tid=0 2024-04-02T17:37:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 client.27841 shutdown 2024-04-02T17:37:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 msgr2=0x564275d5aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5d802c170 tx=0x7fc5d8003820 comp rx=0 tx=0).stop 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 shutdown_connections 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x564275d61de0 0x564275dcaa80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564275d51470 0x564275d5bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564275d4c300 0x564275d5aca0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 --2- 172.21.15.59:0/1427122745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564275d4b910 0x564275d5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 >> 172.21.15.59:0/1427122745 conn(0x564275d3e670 msgr2=0x564275d54940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.800+0000 7fc5d2a00640 0 client.27841 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.801+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 shutdown_connections 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.801+0000 7fc5f6dc1740 1 -- 172.21.15.59:0/1427122745 wait complete. 2024-04-02T17:37:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.801+0000 7fc5f6dc1740 20 client.27841 trim_cache size 0 max 16384 2024-04-02T17:37:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:37:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:37:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:37:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:37:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 msgr2=0x7fc5d40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 0x7fc5d40a6060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x564275d4cb60 tx=0x7fc5e4009500 comp rx=0 tx=0).stop 2024-04-02T17:37:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 msgr2=0x7fc5ec1a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e9f0 tx=0x7fc5e000edf0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 shutdown_connections 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fc5d40a3ba0 0x7fc5d40a6060 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1096e0 0x7fc5ec1a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5ec10fe40 0x7fc5ec1a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 --2- 172.21.15.59:0/4076663344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5ec073040 0x7fc5ec1a26d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 >> 172.21.15.59:0/4076663344 conn(0x7fc5ec06e970 msgr2=0x7fc5ec109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:37:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.803+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 shutdown_connections 2024-04-02T17:37:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:37:52.805+0000 7fc5f2a00640 1 -- 172.21.15.59:0/4076663344 wait complete. 2024-04-02T17:37:53.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 475313 36.53 MB/sec execute 560 sec latency 388.143 ms 2024-04-02T17:37:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:54 smithi154 ceph-mon[36479]: pgmap v1134: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-02T17:37:54.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:54 smithi059 ceph-mon[31664]: pgmap v1134: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-02T17:37:54.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:54 smithi164 ceph-mon[36499]: pgmap v1134: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-02T17:37:54.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 475620 36.49 MB/sec execute 561 sec latency 472.591 ms 2024-04-02T17:37:55.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 475826 36.44 MB/sec execute 562 sec latency 449.293 ms 2024-04-02T17:37:56.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:56 smithi059 ceph-mon[31664]: pgmap v1135: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 518 op/s 2024-04-02T17:37:56.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:56 smithi164 ceph-mon[36499]: pgmap v1135: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 518 op/s 2024-04-02T17:37:56.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:56 smithi154 ceph-mon[36479]: pgmap v1135: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 518 op/s 2024-04-02T17:37:56.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 476582 36.44 MB/sec execute 563 sec latency 64.716 ms 2024-04-02T17:37:57.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 477390 36.46 MB/sec execute 564 sec latency 40.866 ms 2024-04-02T17:37:58.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:37:58 smithi059 ceph-mon[31664]: pgmap v1136: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 499 op/s 2024-04-02T17:37:58.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:37:58 smithi164 ceph-mon[36499]: pgmap v1136: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 499 op/s 2024-04-02T17:37:58.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:37:58 smithi154 ceph-mon[36479]: pgmap v1136: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 499 op/s 2024-04-02T17:37:58.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 478069 36.46 MB/sec execute 565 sec latency 49.086 ms 2024-04-02T17:37:59.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 478666 36.46 MB/sec execute 566 sec latency 67.162 ms 2024-04-02T17:38:00.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:00 smithi059 ceph-mon[31664]: pgmap v1137: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-04-02T17:38:00.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:00 smithi164 ceph-mon[36499]: pgmap v1137: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-04-02T17:38:00.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:00 smithi154 ceph-mon[36479]: pgmap v1137: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-04-02T17:38:00.676 INFO:tasks.workunit.client.0.smithi059.stdout: 10 478790 36.41 MB/sec execute 567 sec latency 817.089 ms 2024-04-02T17:38:01.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:01 smithi059 ceph-mon[31664]: pgmap v1138: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 555 op/s 2024-04-02T17:38:01.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:01 smithi164 ceph-mon[36499]: pgmap v1138: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 555 op/s 2024-04-02T17:38:01.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:01 smithi154 ceph-mon[36479]: pgmap v1138: 97 pgs: 97 active+clean; 1.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 555 op/s 2024-04-02T17:38:01.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 479013 36.35 MB/sec execute 568 sec latency 960.162 ms 2024-04-02T17:38:02.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:02 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:38:02.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:02 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:38:02.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:02 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:38:02.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 479298 36.31 MB/sec execute 569 sec latency 128.413 ms 2024-04-02T17:38:03.549 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:03 smithi059 ceph-mon[31664]: pgmap v1139: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-04-02T17:38:03.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:03 smithi164 ceph-mon[36499]: pgmap v1139: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-04-02T17:38:03.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:03 smithi154 ceph-mon[36479]: pgmap v1139: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-04-02T17:38:03.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 479740 36.30 MB/sec execute 570 sec latency 206.147 ms 2024-04-02T17:38:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:38:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:04 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:38:04.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:04 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:38:04.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:38:04.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:04 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:38:04.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:04 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:38:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:38:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:04 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:38:04.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:04 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:38:04.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 480126 36.27 MB/sec execute 571 sec latency 506.022 ms 2024-04-02T17:38:05.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 480742 36.27 MB/sec execute 572 sec latency 69.744 ms 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 -- 172.21.15.59:0/2103776909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc102d50 msgr2=0x7fdfcc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/2103776909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc102d50 0x7fdfcc1031d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009900 tx=0x7fdfb802bc40 comp rx=0 tx=0).stop 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 -- 172.21.15.59:0/2103776909 shutdown_connections 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/2103776909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 0x7fdfcc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/2103776909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc102d50 0x7fdfcc1031d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/2103776909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc101b60 0x7fdfcc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 -- 172.21.15.59:0/2103776909 >> 172.21.15.59:0/2103776909 conn(0x7fdfcc0fd0d0 msgr2=0x7fdfcc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 -- 172.21.15.59:0/2103776909 shutdown_connections 2024-04-02T17:38:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.012+0000 7fdfd2a00640 1 -- 172.21.15.59:0/2103776909 wait complete. 2024-04-02T17:38:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.014+0000 7fdfd2a00640 1 Processor -- start 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.014+0000 7fdfd2a00640 1 -- start start 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 0x7fdfcc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 0x7fdfcc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc10d890 con 0x7fdfcc103710 2024-04-02T17:38:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc10d710 con 0x7fdfcc102d50 2024-04-02T17:38:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc10da10 con 0x7fdfcc101b60 2024-04-02T17:38:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32920/0 (socket says 172.21.15.59:32920) 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfcbe00640 1 -- 172.21.15.59:0/1326681397 learned_addr learned my addr 172.21.15.59:0/1326681397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 0x7fdfcc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.015+0000 7fdfcb400640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 0x7fdfcc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcb400640 1 -- 172.21.15.59:0/1326681397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 msgr2=0x7fdfcc19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcb400640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcb400640 1 -- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 msgr2=0x7fdfcc1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcb400640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 0x7fdfcc1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcb400640 1 -- 172.21.15.59:0/1326681397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a1c30 con 0x7fdfcc102d50 2024-04-02T17:38:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.016+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfcb400640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 0x7fdfcc19a550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8036040 tx=0x7fdfb8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfc8c00640 1 -- 172.21.15.59:0/1326681397 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8041070 con 0x7fdfcc102d50 2024-04-02T17:38:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfc8c00640 1 -- 172.21.15.59:0/1326681397 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb80044d0 con 0x7fdfcc102d50 2024-04-02T17:38:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfc8c00640 1 -- 172.21.15.59:0/1326681397 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8034440 con 0x7fdfcc102d50 2024-04-02T17:38:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a1ec0 con 0x7fdfcc102d50 2024-04-02T17:38:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.017+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a22d0 con 0x7fdfcc102d50 2024-04-02T17:38:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.021+0000 7fdfc8c00640 1 -- 172.21.15.59:0/1326681397 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdfb8044070 con 0x7fdfcc102d50 2024-04-02T17:38:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 Processor -- start 2024-04-02T17:38:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 -- start start 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad77fee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad77ff33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad77ff5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad77f53d70 con 0x55ad77ff3a30 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad77efc2a0 con 0x55ad77feded0 2024-04-02T17:38:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.023+0000 7fdfd581a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad77f4fd20 con 0x55ad77fee8c0 2024-04-02T17:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad77fee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad77fee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42470/0 (socket says 172.21.15.59:42470) 2024-04-02T17:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfd0a00640 1 -- 172.21.15.59:0/3206926800 learned_addr learned my addr 172.21.15.59:0/3206926800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad77ff33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfc8c00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 0x7fdfb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfc8c00640 1 -- 172.21.15.59:0/1326681397 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdfb80f2360 con 0x7fdfcc102d50 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 -- 172.21.15.59:0/3206926800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 msgr2=0x55ad77fee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad77fee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 -- 172.21.15.59:0/3206926800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 msgr2=0x55ad77ff5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad77ff5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 0x7fdfb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.024+0000 7fdfcbe00640 1 -- 172.21.15.59:0/3206926800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad77f10840 con 0x55ad77fee8c0 2024-04-02T17:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.025+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad77ff33c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0114c0 tx=0x7fdfbc011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0536cbf08c6e473 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.025+0000 7fdfc1200640 1 -- 172.21.15.59:0/3206926800 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc005020 con 0x55ad77fee8c0 2024-04-02T17:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.025+0000 7fdfc1200640 1 -- 172.21.15.59:0/3206926800 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc0047a0 con 0x55ad77fee8c0 2024-04-02T17:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.025+0000 7fdfc1200640 1 -- 172.21.15.59:0/3206926800 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00f980 con 0x55ad77fee8c0 2024-04-02T17:38:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.026+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 0x7fdfb00a6190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009fd0 tx=0x7fdfc4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.027+0000 7fdfd581a740 1 -- 172.21.15.59:0/3206926800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 msgr2=0x55ad77ff33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.027+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad77ff33c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0114c0 tx=0x7fdfbc011980 comp rx=0 tx=0).stop 2024-04-02T17:38:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.028+0000 7fdfd581a740 1 -- 172.21.15.59:0/3206926800 shutdown_connections 2024-04-02T17:38:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.028+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad77ff5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.028+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad77ff33c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.028+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3206926800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad77fee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.028+0000 7fdfd581a740 1 -- 172.21.15.59:0/3206926800 >> 172.21.15.59:0/3206926800 conn(0x55ad77fe9440 msgr2=0x55ad77fe9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.029+0000 7fdfd581a740 1 -- 172.21.15.59:0/3206926800 shutdown_connections 2024-04-02T17:38:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.029+0000 7fdfd581a740 1 -- 172.21.15.59:0/3206926800 wait complete. 2024-04-02T17:38:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.030+0000 7fdfd581a740 1 -- start start 2024-04-02T17:38:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.030+0000 7fdfd581a740 1 Processor -- start 2024-04-02T17:38:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77feded0 0x55ad78000cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 0x55ad780011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad77fff1f0 con 0x55ad77fee8c0 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad77f0f660 con 0x55ad77ff3a30 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad77fff370 con 0x55ad77feded0 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd581a740 10 client.0 _ll_register_callbacks cb 0x55ad77f0f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.031+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42472/0 (socket says 172.21.15.59:42472) 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 -- 172.21.15.59:0/3609167600 learned_addr learned my addr 172.21.15.59:0/3609167600 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77feded0 0x55ad78000cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 -- 172.21.15.59:0/3609167600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77feded0 msgr2=0x55ad78000cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77feded0 0x55ad78000cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 -- 172.21.15.59:0/3609167600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 msgr2=0x55ad780011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfcb400640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 0x55ad780011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 0x55ad780011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfd0a00640 1 -- 172.21.15.59:0/3609167600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad77f1ee80 con 0x55ad77ff3a30 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.032+0000 7fdfcb400640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 0x55ad780011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.033+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5805f2977cdd072e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.033+0000 7fdfb5600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.033+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007970 con 0x55ad77ff3a30 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.033+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdfc4007b10 con 0x55ad77ff3a30 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.033+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc0047a0 con 0x55ad77ff3a30 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.034+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4007d10 con 0x55ad77ff3a30 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.034+0000 7fdfd581a740 10 client.27791 fetch_fsmap learned FSMap version 69 2024-04-02T17:38:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.034+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ad77c68210 con 0x55ad77ff3a30 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.034+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc0047a0 con 0x55ad77ff3a30 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.034+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) ==== 8418+0+0 (secure 0 0 0) 0x7fdfc40174b0 con 0x55ad77ff3a30 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 10 client.27791 fetch_fsmap finished waiting for FSMap version 69 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 10 client.27791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 20 client.27791 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad7800f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 4 client.27791 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad77f03cd0 con 0x55ad7800cc20 2024-04-02T17:38:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.035+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad7800f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.036+0000 7fdfb5600640 10 client.27791 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.036+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad7800f080 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78591a48623906b server_cookie=27b502ae1ab27c53 in_seq=0 out_seq=0 2024-04-02T17:38:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.039+0000 7fdfb5600640 1 -- 172.21.15.59:0/3609167600 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad77f03cd0 con 0x55ad7800cc20 2024-04-02T17:38:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.039+0000 7fdfb5600640 10 client.27791 handle_command_reply: tid=0 2024-04-02T17:38:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.039+0000 7fdfd581a740 1 client.27791 shutdown 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.039+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 msgr2=0x55ad78008230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.039+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc400d040 comp rx=0 tx=0).stop 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 shutdown_connections 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad7800f080 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfb5600640 0 client.27791 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77fee8c0 0x55ad780011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77ff3a30 0x55ad78008230 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 --2- 172.21.15.59:0/3609167600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77feded0 0x55ad78000cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 >> 172.21.15.59:0/3609167600 conn(0x55ad77fe9440 msgr2=0x55ad77fe8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 shutdown_connections 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 1 -- 172.21.15.59:0/3609167600 wait complete. 2024-04-02T17:38:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.040+0000 7fdfd581a740 20 client.27791 trim_cache size 0 max 16384 2024-04-02T17:38:06.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:05 smithi059 ceph-mon[31664]: pgmap v1140: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T17:38:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.066+0000 7fdfd581a740 1 Processor -- start 2024-04-02T17:38:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.066+0000 7fdfd581a740 1 -- start start 2024-04-02T17:38:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 0x55ad77fecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77fee8c0 0x55ad780081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a6f10 con 0x55ad77feded0 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a8c90 con 0x55ad77fee8c0 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd581a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a76a0 con 0x55ad77ff3a30 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77fee8c0 0x55ad780081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32950/0 (socket says 172.21.15.59:32950) 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 -- 172.21.15.59:0/448893291 learned_addr learned my addr 172.21.15.59:0/448893291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 0x55ad77fecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 -- 172.21.15.59:0/448893291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77fee8c0 msgr2=0x55ad780081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77fee8c0 0x55ad780081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 -- 172.21.15.59:0/448893291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 msgr2=0x55ad77fecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 0x55ad77fecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.067+0000 7fdfd0a00640 1 -- 172.21.15.59:0/448893291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad77d34620 con 0x55ad77ff3a30 2024-04-02T17:38:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.068+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 0x55ad77fecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:38:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.068+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400ae70 tx=0x7fdfc4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621b42b5cf05d5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.068+0000 7fdfb7e00640 1 -- 172.21.15.59:0/448893291 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4017420 con 0x55ad77ff3a30 2024-04-02T17:38:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.068+0000 7fdfb7e00640 1 -- 172.21.15.59:0/448893291 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4035020 con 0x55ad77ff3a30 2024-04-02T17:38:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.068+0000 7fdfb7e00640 1 -- 172.21.15.59:0/448893291 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40306f0 con 0x55ad77ff3a30 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 -- 172.21.15.59:0/448893291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 msgr2=0x55ad78013940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400ae70 tx=0x7fdfc4019000 comp rx=0 tx=0).stop 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 -- 172.21.15.59:0/448893291 shutdown_connections 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77ff3a30 0x55ad78013940 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77fee8c0 0x55ad780081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 --2- 172.21.15.59:0/448893291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77feded0 0x55ad77fecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.069+0000 7fdfd581a740 1 -- 172.21.15.59:0/448893291 >> 172.21.15.59:0/448893291 conn(0x55ad77fe9440 msgr2=0x55ad77fec6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.070+0000 7fdfd581a740 1 -- 172.21.15.59:0/448893291 shutdown_connections 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.070+0000 7fdfd581a740 1 -- 172.21.15.59:0/448893291 wait complete. 2024-04-02T17:38:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.070+0000 7fdfd581a740 1 -- start start 2024-04-02T17:38:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.070+0000 7fdfd581a740 1 Processor -- start 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad78005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad78006bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a85c0 con 0x55ad77ff3a30 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a7e30 con 0x55ad77feded0 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1a4f10 con 0x55ad77fee8c0 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd581a740 10 client.0 _ll_register_callbacks cb 0x7fdfc4019720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad78005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:42492/0 (socket says 172.21.15.59:42492) 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad78005bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:32964/0 (socket says 172.21.15.59:32964) 2024-04-02T17:38:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfcb400640 1 -- 172.21.15.59:0/2718785524 learned_addr learned my addr 172.21.15.59:0/2718785524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.071+0000 7fdfd0a00640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad78006bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 -- 172.21.15.59:0/2718785524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 msgr2=0x55ad78005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad78005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 -- 172.21.15.59:0/2718785524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 msgr2=0x55ad78006bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad78006bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 -- 172.21.15.59:0/2718785524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad780071c0 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfcbe00640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55ad77fea3b0 tx=0x7fdfbc0047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e3e9e1749057ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfb5600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01d040 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.072+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc4035020 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.073+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdfbc009c10 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.073+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc038050 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.073+0000 7fdfd581a740 10 client.27797 fetch_fsmap learned FSMap version 69 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.073+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ad780084d0 con 0x55ad77feded0 2024-04-02T17:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.073+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4035020 con 0x55ad77feded0 2024-04-02T17:38:06.074 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:05 smithi164 ceph-mon[36499]: pgmap v1140: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) ==== 8418+0+0 (secure 0 0 0) 0x7fdfbc02d020 con 0x55ad77feded0 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 10 client.27797 fetch_fsmap finished waiting for FSMap version 69 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 10 client.27797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 20 client.27797 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad78075810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 4 client.27797 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad77f612b0 con 0x55ad7800cc20 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.074+0000 7fdfcb400640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad78075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.075+0000 7fdfcb400640 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad78075810 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c474591254edc5 server_cookie=6f5696bf7e2d4c4e in_seq=0 out_seq=0 2024-04-02T17:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.075+0000 7fdfb5600640 10 client.27797 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:06.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:05 smithi154 ceph-mon[36479]: pgmap v1140: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T17:38:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.280+0000 7fdfb5600640 1 -- 172.21.15.59:0/2718785524 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad77f612b0 con 0x55ad7800cc20 2024-04-02T17:38:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.280+0000 7fdfb5600640 10 client.27797 handle_command_reply: tid=0 2024-04-02T17:38:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.280+0000 7fdfd581a740 1 client.27797 shutdown 2024-04-02T17:38:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.281+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 msgr2=0x55ad78005680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.281+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55ad77fea3b0 tx=0x7fdfbc0047a0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 shutdown_connections 2024-04-02T17:38:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55ad7800cc20 0x55ad78075810 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad77ff3a30 0x55ad78006bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfb5600640 0 client.27797 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad77fee8c0 0x55ad78005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 --2- 172.21.15.59:0/2718785524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad77feded0 0x55ad78005680 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.283+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 >> 172.21.15.59:0/2718785524 conn(0x55ad77fe9440 msgr2=0x55ad77fecf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.284+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 shutdown_connections 2024-04-02T17:38:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.284+0000 7fdfd581a740 1 -- 172.21.15.59:0/2718785524 wait complete. 2024-04-02T17:38:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.284+0000 7fdfd581a740 20 client.27797 trim_cache size 0 max 16384 2024-04-02T17:38:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:38:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:38:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:38:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 msgr2=0x7fdfb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 0x7fdfb00a6190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009fd0 tx=0x7fdfc4004a40 comp rx=0 tx=0).stop 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 msgr2=0x7fdfcc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 0x7fdfcc19a550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8036040 tx=0x7fdfb8005d50 comp rx=0 tx=0).stop 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 shutdown_connections 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fdfb00a3cd0 0x7fdfb00a6190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfcc103710 0x7fdfcc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc102d50 0x7fdfcc19a550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 --2- 172.21.15.59:0/1326681397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfcc101b60 0x7fdfcc19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.286+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 >> 172.21.15.59:0/1326681397 conn(0x7fdfcc0fd0d0 msgr2=0x7fdfcc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.287+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 shutdown_connections 2024-04-02T17:38:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:06.287+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1326681397 wait complete. 2024-04-02T17:38:06.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 481281 36.25 MB/sec execute 573 sec latency 204.543 ms 2024-04-02T17:38:07.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 481917 36.26 MB/sec execute 574 sec latency 84.388 ms 2024-04-02T17:38:08.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:07 smithi059 ceph-mon[31664]: pgmap v1141: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 803 op/s 2024-04-02T17:38:08.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:07 smithi164 ceph-mon[36499]: pgmap v1141: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 803 op/s 2024-04-02T17:38:08.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:07 smithi154 ceph-mon[36479]: pgmap v1141: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 803 op/s 2024-04-02T17:38:08.677 INFO:tasks.workunit.client.0.smithi059.stdout: 10 482562 36.25 MB/sec execute 575 sec latency 68.115 ms 2024-04-02T17:38:09.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 483264 36.25 MB/sec execute 576 sec latency 82.950 ms 2024-04-02T17:38:10.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:09 smithi164 ceph-mon[36499]: pgmap v1142: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 828 op/s 2024-04-02T17:38:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:09 smithi154 ceph-mon[36479]: pgmap v1142: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 828 op/s 2024-04-02T17:38:10.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:09 smithi059 ceph-mon[31664]: pgmap v1142: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 828 op/s 2024-04-02T17:38:10.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 484067 36.25 MB/sec execute 577 sec latency 57.335 ms 2024-04-02T17:38:11.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 484657 36.26 MB/sec execute 578 sec latency 327.362 ms 2024-04-02T17:38:12.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:11 smithi164 ceph-mon[36499]: pgmap v1143: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 793 op/s 2024-04-02T17:38:12.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:11 smithi059 ceph-mon[31664]: pgmap v1143: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 793 op/s 2024-04-02T17:38:12.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:11 smithi154 ceph-mon[36479]: pgmap v1143: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 793 op/s 2024-04-02T17:38:12.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 485353 36.27 MB/sec execute 579 sec latency 92.942 ms 2024-04-02T17:38:13.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 486039 36.26 MB/sec execute 580 sec latency 60.244 ms 2024-04-02T17:38:14.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:13 smithi164 ceph-mon[36499]: pgmap v1144: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 17 MiB/s wr, 830 op/s 2024-04-02T17:38:14.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:13 smithi154 ceph-mon[36479]: pgmap v1144: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 17 MiB/s wr, 830 op/s 2024-04-02T17:38:14.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:13 smithi059 ceph-mon[31664]: pgmap v1144: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 17 MiB/s wr, 830 op/s 2024-04-02T17:38:14.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 486708 36.26 MB/sec execute 581 sec latency 66.684 ms 2024-04-02T17:38:15.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 487554 36.26 MB/sec execute 582 sec latency 64.328 ms 2024-04-02T17:38:16.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:15 smithi164 ceph-mon[36499]: pgmap v1145: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 748 op/s 2024-04-02T17:38:16.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:15 smithi154 ceph-mon[36479]: pgmap v1145: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 748 op/s 2024-04-02T17:38:16.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:15 smithi059 ceph-mon[31664]: pgmap v1145: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 748 op/s 2024-04-02T17:38:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 488388 36.29 MB/sec execute 583 sec latency 49.132 ms 2024-04-02T17:38:17.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 489066 36.30 MB/sec execute 584 sec latency 40.365 ms 2024-04-02T17:38:18.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:17 smithi164 ceph-mon[36499]: pgmap v1146: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 694 op/s 2024-04-02T17:38:18.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:17 smithi154 ceph-mon[36479]: pgmap v1146: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 694 op/s 2024-04-02T17:38:18.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:17 smithi059 ceph-mon[31664]: pgmap v1146: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 694 op/s 2024-04-02T17:38:18.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 489712 36.29 MB/sec execute 585 sec latency 49.641 ms 2024-04-02T17:38:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 -- 172.21.15.59:0/3272988625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c06ced0 msgr2=0x7ff02c06b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 --2- 172.21.15.59:0/3272988625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c06ced0 0x7ff02c06b320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a910 tx=0x7ff01c03cc70 comp rx=0 tx=0).stop 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 -- 172.21.15.59:0/3272988625 shutdown_connections 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 --2- 172.21.15.59:0/3272988625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c116730 0x7ff02c118b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 --2- 172.21.15.59:0/3272988625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06b8f0 0x7ff02c06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 --2- 172.21.15.59:0/3272988625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c06ced0 0x7ff02c06b320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 -- 172.21.15.59:0/3272988625 >> 172.21.15.59:0/3272988625 conn(0x7ff02c070920 msgr2=0x7ff02c072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 -- 172.21.15.59:0/3272988625 shutdown_connections 2024-04-02T17:38:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.480+0000 7ff031c00640 1 -- 172.21.15.59:0/3272988625 wait complete. 2024-04-02T17:38:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 Processor -- start 2024-04-02T17:38:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 -- start start 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 0x7ff02c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 0x7ff02c1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110ba0 con 0x7ff02c06b8f0 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.482+0000 7ff031c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110a20 con 0x7ff02c06ced0 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff031c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110d20 con 0x7ff02c116730 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff02b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff02b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44582/0 (socket says 172.21.15.59:44582) 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff02b400640 1 -- 172.21.15.59:0/48518412 learned_addr learned my addr 172.21.15.59:0/48518412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff031200640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 0x7ff02c1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.483+0000 7ff02be00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 0x7ff02c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 -- 172.21.15.59:0/48518412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 msgr2=0x7ff02c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 -- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 msgr2=0x7ff02c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 0x7ff02c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 -- 172.21.15.59:0/48518412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c1a5eb0 con 0x7ff02c116730 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff02b400640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff02be00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 0x7ff02c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:38:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.484+0000 7ff031200640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 0x7ff02c1a5840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff02400da10 tx=0x7ff02400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.485+0000 7ff028c00640 1 -- 172.21.15.59:0/48518412 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400bd70 con 0x7ff02c116730 2024-04-02T17:38:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.485+0000 7ff028c00640 1 -- 172.21.15.59:0/48518412 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024003e50 con 0x7ff02c116730 2024-04-02T17:38:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.485+0000 7ff028c00640 1 -- 172.21.15.59:0/48518412 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240106a0 con 0x7ff02c116730 2024-04-02T17:38:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.486+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a61a0 con 0x7ff02c116730 2024-04-02T17:38:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.486+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a6640 con 0x7ff02c116730 2024-04-02T17:38:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.491+0000 7ff028c00640 1 -- 172.21.15.59:0/48518412 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff024003970 con 0x7ff02c116730 2024-04-02T17:38:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.492+0000 7ff035e5c740 1 Processor -- start 2024-04-02T17:38:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.492+0000 7ff035e5c740 1 -- start start 2024-04-02T17:38:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d255520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d25a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d25d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff028c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 0x7ff0100a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da1d1bad70 con 0x55da1d255100 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da1d1632a0 con 0x55da1d25ac60 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.493+0000 7ff035e5c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da1d1b6d20 con 0x55da1d255af0 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff02b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d255520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff02b400640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 0x7ff0100a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff02b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d255520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45392/0 (socket says 172.21.15.59:45392) 2024-04-02T17:38:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff02b400640 1 -- 172.21.15.59:0/267414387 learned_addr learned my addr 172.21.15.59:0/267414387 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff028c00640 1 -- 172.21.15.59:0/48518412 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff0240d0040 con 0x7ff02c116730 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.494+0000 7ff031200640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d25d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d25a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 -- 172.21.15.59:0/267414387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 msgr2=0x55da1d25d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d25d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 -- 172.21.15.59:0/267414387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 msgr2=0x55da1d255520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d255520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff02be00640 1 -- 172.21.15.59:0/267414387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1d177840 con 0x55da1d255af0 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.495+0000 7ff031200640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d25d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.496+0000 7ff02be00640 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d25a5f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff01c03d280 tx=0x7ff01c01a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a674dd17ec4170 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.496+0000 7ff02b400640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 0x7ff0100a60e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff018009640 tx=0x7ff018009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.496+0000 7ff021600640 1 -- 172.21.15.59:0/267414387 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c004a00 con 0x55da1d255af0 2024-04-02T17:38:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.496+0000 7ff021600640 1 -- 172.21.15.59:0/267414387 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff01c004ba0 con 0x55da1d255af0 2024-04-02T17:38:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.496+0000 7ff021600640 1 -- 172.21.15.59:0/267414387 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0051a0 con 0x55da1d255af0 2024-04-02T17:38:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.497+0000 7ff035e5c740 1 -- 172.21.15.59:0/267414387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 msgr2=0x55da1d25a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.497+0000 7ff035e5c740 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d25a5f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff01c03d280 tx=0x7ff01c01a640 comp rx=0 tx=0).stop 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 -- 172.21.15.59:0/267414387 shutdown_connections 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d25d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d25a5f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 --2- 172.21.15.59:0/267414387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d255520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 -- 172.21.15.59:0/267414387 >> 172.21.15.59:0/267414387 conn(0x55da1d250670 msgr2=0x55da1d250a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 -- 172.21.15.59:0/267414387 shutdown_connections 2024-04-02T17:38:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.498+0000 7ff035e5c740 1 -- 172.21.15.59:0/267414387 wait complete. 2024-04-02T17:38:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.499+0000 7ff035e5c740 1 -- start start 2024-04-02T17:38:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.499+0000 7ff035e5c740 1 Processor -- start 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d25f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d25ac60 0x55da1d266bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da1d25dbb0 con 0x55da1d255100 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55da1d176660 con 0x55da1d255af0 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da1d25dd30 con 0x55da1d25ac60 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff035e5c740 10 client.0 _ll_register_callbacks cb 0x55da1d176050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff02b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff02b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:44594/0 (socket says 172.21.15.59:44594) 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff02b400640 1 -- 172.21.15.59:0/581314581 learned_addr learned my addr 172.21.15.59:0/581314581 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff02be00640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d25f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.500+0000 7ff031200640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d25ac60 0x55da1d266bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 -- 172.21.15.59:0/581314581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d25ac60 msgr2=0x55da1d266bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d25ac60 0x55da1d266bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 -- 172.21.15.59:0/581314581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 msgr2=0x55da1d25f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d25f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 -- 172.21.15.59:0/581314581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1d185e80 con 0x55da1d255af0 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff02b400640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff01801f960 tx=0x7ff01801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10ece75e1f124fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff016000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018019040 con 0x55da1d255af0 2024-04-02T17:38:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff01c004ba0 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.501+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff01801ead0 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff01801fe20 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff035e5c740 10 client.27809 fetch_fsmap learned FSMap version 70 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55da1cecf210 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff01c004ba0 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 70) ==== 8422+0+0 (secure 0 0 0) 0x7ff018004a00 con 0x55da1d255af0 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff035e5c740 10 client.27809 fetch_fsmap finished waiting for FSMap version 70 2024-04-02T17:38:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff035e5c740 10 client.27809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.502+0000 7ff035e5c740 20 client.27809 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.503+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d26b5c0 0x55da1d26da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.503+0000 7ff035e5c740 4 client.27809 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.503+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1d16acd0 con 0x55da1d26b5c0 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.503+0000 7ff02be00640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d26b5c0 0x55da1d26da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.504+0000 7ff02be00640 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d26b5c0 0x55da1d26da60 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c561b918102fff server_cookie=a67ed7a38905b7bc in_seq=0 out_seq=0 2024-04-02T17:38:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.504+0000 7ff016000640 10 client.27809 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff016000640 1 -- 172.21.15.59:0/581314581 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da1d16acd0 con 0x55da1d26b5c0 2024-04-02T17:38:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff016000640 10 client.27809 handle_command_reply: tid=0 2024-04-02T17:38:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 client.27809 shutdown 2024-04-02T17:38:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 msgr2=0x55da1d25fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff01801f960 tx=0x7ff01801d000 comp rx=0 tx=0).stop 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 shutdown_connections 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d26b5c0 0x55da1d26da60 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d25ac60 0x55da1d266bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d25fb80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff016000640 0 client.27809 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 --2- 172.21.15.59:0/581314581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d25f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.507+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 >> 172.21.15.59:0/581314581 conn(0x55da1d250670 msgr2=0x55da1d24f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.508+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 shutdown_connections 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.508+0000 7ff035e5c740 1 -- 172.21.15.59:0/581314581 wait complete. 2024-04-02T17:38:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.508+0000 7ff035e5c740 20 client.27809 trim_cache size 0 max 16384 2024-04-02T17:38:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 Processor -- start 2024-04-02T17:38:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 -- start start 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d24f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d266b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1af5a0 con 0x55da1d255100 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b1320 con 0x55da1d25ac60 2024-04-02T17:38:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.528+0000 7ff035e5c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1afd30 con 0x55da1d255af0 2024-04-02T17:38:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44242/0 (socket says 172.21.15.59:44242) 2024-04-02T17:38:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 -- 172.21.15.59:0/1063324673 learned_addr learned my addr 172.21.15.59:0/1063324673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d24f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff031200640 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d266b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 -- 172.21.15.59:0/1063324673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 msgr2=0x55da1d266b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d266b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 -- 172.21.15.59:0/1063324673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 msgr2=0x55da1d24f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d24f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.529+0000 7ff02b400640 1 -- 172.21.15.59:0/1063324673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1cf9b620 con 0x55da1d255af0 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.530+0000 7ff031200640 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d266b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:38:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.530+0000 7ff02b400640 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff01801e3a0 tx=0x7ff0180275c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91b2a3d687b7fef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.530+0000 7ff020c00640 1 -- 172.21.15.59:0/1063324673 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018019040 con 0x55da1d255af0 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.530+0000 7ff020c00640 1 -- 172.21.15.59:0/1063324673 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018004b60 con 0x55da1d255af0 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.530+0000 7ff020c00640 1 -- 172.21.15.59:0/1063324673 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018005220 con 0x55da1d255af0 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.531+0000 7ff035e5c740 1 -- 172.21.15.59:0/1063324673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 msgr2=0x55da1d24fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.531+0000 7ff035e5c740 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff01801e3a0 tx=0x7ff0180275c0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 -- 172.21.15.59:0/1063324673 shutdown_connections 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d25ac60 0x55da1d266b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255af0 0x55da1d24fc30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 --2- 172.21.15.59:0/1063324673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d255100 0x55da1d24f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 -- 172.21.15.59:0/1063324673 >> 172.21.15.59:0/1063324673 conn(0x55da1d250670 msgr2=0x55da1d24efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 -- 172.21.15.59:0/1063324673 shutdown_connections 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 -- 172.21.15.59:0/1063324673 wait complete. 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 -- start start 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.532+0000 7ff035e5c740 1 Processor -- start 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d264ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 0x55da1d265ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b0c50 con 0x55da1d25ac60 2024-04-02T17:38:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1b04c0 con 0x55da1d255af0 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ad580 con 0x55da1d255100 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff035e5c740 10 client.0 _ll_register_callbacks cb 0x7ff01801f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:44256/0 (socket says 172.21.15.59:44256) 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 -- 172.21.15.59:0/102363857 learned_addr learned my addr 172.21.15.59:0/102363857 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02b400640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d264ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 -- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 msgr2=0x55da1d264ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d264ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff02be00640 1 -- 172.21.15.59:0/102363857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 msgr2=0x55da1d265ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.533+0000 7ff031200640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 0x55da1d265ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff02be00640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 0x55da1d265ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff02be00640 1 -- 172.21.15.59:0/102363857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1d2664d0 con 0x55da1d255100 2024-04-02T17:38:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff031200640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 0x55da1d265ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T17:38:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff02be00640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a910 tx=0x7ff01c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581b63f5fc132815 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff016000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:38:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c05fbc0 con 0x55da1d255100 2024-04-02T17:38:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.534+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff01c05fd60 con 0x55da1d255100 2024-04-02T17:38:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.535+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da1d1c82b0 con 0x55da1d255100 2024-04-02T17:38:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.535+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff01c06e050 con 0x55da1d255100 2024-04-02T17:38:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.535+0000 7ff035e5c740 10 client.27877 fetch_fsmap learned FSMap version 70 2024-04-02T17:38:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.535+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55da1d2d2210 con 0x55da1d255100 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.535+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55da1d1c82b0 con 0x55da1d255100 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 70) ==== 8422+0+0 (secure 0 0 0) 0x7ff01c05f740 con 0x55da1d255100 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 10 client.27877 fetch_fsmap finished waiting for FSMap version 70 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 10 client.27877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 20 client.27877 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d2d2880 0x55da1d2d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 4 client.27877 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da1d1c2600 con 0x55da1d2d2880 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.536+0000 7ff02b400640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d2d2880 0x55da1d2d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.537+0000 7ff02b400640 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d2d2880 0x55da1d2d4c70 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47535aa2457a6896 server_cookie=50f3e2e4b39bdd2f in_seq=0 out_seq=0 2024-04-02T17:38:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.537+0000 7ff016000640 10 client.27877 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:19.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 490175 36.27 MB/sec execute 586 sec latency 124.134 ms 2024-04-02T17:38:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.868+0000 7ff016000640 1 -- 172.21.15.59:0/102363857 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da1d1c2600 con 0x55da1d2d2880 2024-04-02T17:38:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.868+0000 7ff016000640 10 client.27877 handle_command_reply: tid=0 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.868+0000 7ff035e5c740 1 client.27877 shutdown 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 msgr2=0x55da1d264990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a910 tx=0x7ff01c000bc0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 shutdown_connections 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x55da1d2d2880 0x55da1d2d4c70 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da1d25ac60 0x55da1d265ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55da1d255af0 0x55da1d264ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 --2- 172.21.15.59:0/102363857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da1d255100 0x55da1d264990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 >> 172.21.15.59:0/102363857 conn(0x55da1d250670 msgr2=0x55da1d24f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:38:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:38:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:38:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff016000640 0 client.27877 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.869+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 shutdown_connections 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.870+0000 7ff035e5c740 1 -- 172.21.15.59:0/102363857 wait complete. 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.870+0000 7ff035e5c740 20 client.27877 trim_cache size 0 max 16384 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.871+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 msgr2=0x7ff0100a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 0x7ff0100a60e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff018009640 tx=0x7ff018009d90 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 msgr2=0x7ff02c1a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 0x7ff02c1a5840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff02400da10 tx=0x7ff02400ded0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 shutdown_connections 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c116730 0x7ff02c1a5840 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff02c06ced0 0x7ff02c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7ff0100a3c20 0x7ff0100a60e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 --2- 172.21.15.59:0/48518412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c06b8f0 0x7ff02c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 >> 172.21.15.59:0/48518412 conn(0x7ff02c070920 msgr2=0x7ff02c071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 shutdown_connections 2024-04-02T17:38:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:19.872+0000 7ff031c00640 1 -- 172.21.15.59:0/48518412 wait complete. 2024-04-02T17:38:20.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:19 smithi154 ceph-mon[36479]: pgmap v1147: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 584 op/s 2024-04-02T17:38:20.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:19 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:20.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:19 smithi059 ceph-mon[31664]: pgmap v1147: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 584 op/s 2024-04-02T17:38:20.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:19 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:20.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:19 smithi164 ceph-mon[36499]: pgmap v1147: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.7 MiB/s wr, 584 op/s 2024-04-02T17:38:20.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:19 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:20.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 490521 36.24 MB/sec execute 587 sec latency 278.466 ms 2024-04-02T17:38:21.678 INFO:tasks.workunit.client.0.smithi059.stdout: 10 491426 36.25 MB/sec execute 588 sec latency 84.648 ms 2024-04-02T17:38:22.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:21 smithi154 ceph-mon[36479]: pgmap v1148: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 499 op/s 2024-04-02T17:38:22.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:21 smithi059 ceph-mon[31664]: pgmap v1148: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 499 op/s 2024-04-02T17:38:22.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:21 smithi164 ceph-mon[36499]: pgmap v1148: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.1 MiB/s wr, 499 op/s 2024-04-02T17:38:22.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 492168 36.26 MB/sec execute 589 sec latency 75.667 ms 2024-04-02T17:38:23.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:22 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:23.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:22 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:23.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:22 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:23.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 492848 36.28 MB/sec execute 590 sec latency 54.993 ms 2024-04-02T17:38:24.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:23 smithi154 ceph-mon[36479]: pgmap v1149: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 558 op/s 2024-04-02T17:38:24.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:23 smithi059 ceph-mon[31664]: pgmap v1149: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 558 op/s 2024-04-02T17:38:24.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:23 smithi164 ceph-mon[36499]: pgmap v1149: 97 pgs: 97 active+clean; 1.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 558 op/s 2024-04-02T17:38:24.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 493400 36.26 MB/sec execute 591 sec latency 64.054 ms 2024-04-02T17:38:25.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:24 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:25.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:24 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:25.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:24 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:25.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 493960 36.25 MB/sec execute 592 sec latency 72.751 ms 2024-04-02T17:38:26.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:25 smithi154 ceph-mon[36479]: pgmap v1150: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-04-02T17:38:26.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:25 smithi059 ceph-mon[31664]: pgmap v1150: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-04-02T17:38:26.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:25 smithi164 ceph-mon[36499]: pgmap v1150: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.8 MiB/s wr, 530 op/s 2024-04-02T17:38:26.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 494741 36.24 MB/sec execute 593 sec latency 68.405 ms 2024-04-02T17:38:27.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 495671 36.28 MB/sec execute 594 sec latency 40.747 ms 2024-04-02T17:38:28.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:27 smithi154 ceph-mon[36479]: pgmap v1151: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-04-02T17:38:28.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:27 smithi059 ceph-mon[31664]: pgmap v1151: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-04-02T17:38:28.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:27 smithi164 ceph-mon[36499]: pgmap v1151: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-04-02T17:38:28.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 496366 36.29 MB/sec execute 595 sec latency 54.042 ms 2024-04-02T17:38:29.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 496901 36.27 MB/sec execute 596 sec latency 145.687 ms 2024-04-02T17:38:30.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:29 smithi154 ceph-mon[36479]: pgmap v1152: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 567 op/s 2024-04-02T17:38:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:29 smithi059 ceph-mon[31664]: pgmap v1152: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 567 op/s 2024-04-02T17:38:30.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:29 smithi164 ceph-mon[36499]: pgmap v1152: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 567 op/s 2024-04-02T17:38:30.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 497089 36.23 MB/sec execute 597 sec latency 525.970 ms 2024-04-02T17:38:31.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 497530 36.20 MB/sec execute 598 sec latency 99.201 ms 2024-04-02T17:38:32.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:31 smithi154 ceph-mon[36479]: pgmap v1153: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-04-02T17:38:32.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:31 smithi059 ceph-mon[31664]: pgmap v1153: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-04-02T17:38:32.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:31 smithi164 ceph-mon[36499]: pgmap v1153: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-04-02T17:38:32.679 INFO:tasks.workunit.client.0.smithi059.stdout: 10 498305 36.20 MB/sec execute 599 sec latency 90.540 ms 2024-04-02T17:38:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 -- 172.21.15.59:0/600303065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e781020e0 msgr2=0x7f4e78102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 --2- 172.21.15.59:0/600303065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e781020e0 0x7f4e78102500 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009860 tx=0x7f4e6802bbd0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 -- 172.21.15.59:0/600303065 shutdown_connections 2024-04-02T17:38:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 --2- 172.21.15.59:0/600303065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e78108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 --2- 172.21.15.59:0/600303065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e78100ca0 0x7f4e78101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 --2- 172.21.15.59:0/600303065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e781020e0 0x7f4e78102500 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 -- 172.21.15.59:0/600303065 >> 172.21.15.59:0/600303065 conn(0x7f4e780fb750 msgr2=0x7f4e780fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 -- 172.21.15.59:0/600303065 shutdown_connections 2024-04-02T17:38:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.052+0000 7f4e7d000640 1 -- 172.21.15.59:0/600303065 wait complete. 2024-04-02T17:38:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.053+0000 7f4e7d000640 1 Processor -- start 2024-04-02T17:38:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.053+0000 7f4e7d000640 1 -- start start 2024-04-02T17:38:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e7819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e781020e0 0x7f4e781a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7810b630 con 0x7f4e78101660 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7810b4b0 con 0x7f4e781020e0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e7d000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7810b7b0 con 0x7f4e78100ca0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e76800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e781020e0 0x7f4e781a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e7819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40036/0 (socket says 172.21.15.59:40036) 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 -- 172.21.15.59:0/2862123978 learned_addr learned my addr 172.21.15.59:0/2862123978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 -- 172.21.15.59:0/2862123978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e781020e0 msgr2=0x7f4e781a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e781020e0 0x7f4e781a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 -- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 msgr2=0x7f4e7819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e7819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.054+0000 7f4e75e00640 1 -- 172.21.15.59:0/2862123978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e781a1c10 con 0x7f4e78100ca0 2024-04-02T17:38:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e75400640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e7819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005940 tx=0x7f4e68032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e6ea00640 1 -- 172.21.15.59:0/2862123978 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68041020 con 0x7f4e78100ca0 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e6ea00640 1 -- 172.21.15.59:0/2862123978 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6802c9f0 con 0x7f4e78100ca0 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e781a1ea0 con 0x7f4e78100ca0 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e6ea00640 1 -- 172.21.15.59:0/2862123978 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68039040 con 0x7f4e78100ca0 2024-04-02T17:38:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.055+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e781a2300 con 0x7f4e78100ca0 2024-04-02T17:38:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.058+0000 7f4e6ea00640 1 -- 172.21.15.59:0/2862123978 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e68044020 con 0x7f4e78100ca0 2024-04-02T17:38:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.059+0000 7f4e7fdde740 1 Processor -- start 2024-04-02T17:38:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.059+0000 7f4e7fdde740 1 -- start start 2024-04-02T17:38:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 0x5585282952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x55852829a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 0x55852829cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585281fad70 con 0x55852829aa30 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5585281a32a0 con 0x558528294ed0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5585281f6d20 con 0x5585282958c0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e6ea00640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 0x7f4e5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e6ea00640 1 -- 172.21.15.59:0/2862123978 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4e680f1dc0 con 0x7f4e78100ca0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e76800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 0x5585282952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e76800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 0x5585282952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:37686/0 (socket says 172.21.15.59:37686) 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e75400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 0x55852829cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e76800640 1 -- 172.21.15.59:0/702463788 learned_addr learned my addr 172.21.15.59:0/702463788 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e75e00640 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x55852829a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.060+0000 7f4e76800640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 0x7f4e5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 -- 172.21.15.59:0/702463788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 msgr2=0x5585282952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 0x5585282952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 -- 172.21.15.59:0/702463788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 msgr2=0x55852829cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 0x55852829cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 -- 172.21.15.59:0/702463788 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585281b7840 con 0x5585282958c0 2024-04-02T17:38:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75400640 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 0x55852829cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e75e00640 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x55852829a3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e680bcea0 tx=0x7f4e6802bbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac40d490f7ba2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e76800640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 0x7f4e5c0a6190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x558528296120 tx=0x7f4e70009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e63400640 1 -- 172.21.15.59:0/702463788 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680be6e0 con 0x5585282958c0 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e63400640 1 -- 172.21.15.59:0/702463788 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e680bdfe0 con 0x5585282958c0 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.061+0000 7f4e63400640 1 -- 172.21.15.59:0/702463788 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680ef910 con 0x5585282958c0 2024-04-02T17:38:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 -- 172.21.15.59:0/702463788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 msgr2=0x55852829a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x55852829a3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e680bcea0 tx=0x7f4e6802bbd0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 -- 172.21.15.59:0/702463788 shutdown_connections 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55852829aa30 0x55852829cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x55852829a3c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/702463788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558528294ed0 0x5585282952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 -- 172.21.15.59:0/702463788 >> 172.21.15.59:0/702463788 conn(0x558528290440 msgr2=0x558528290840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 -- 172.21.15.59:0/702463788 shutdown_connections 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.062+0000 7f4e7fdde740 1 -- 172.21.15.59:0/702463788 wait complete. 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 -- start start 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 Processor -- start 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x55852829f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585281f6d20 con 0x5585282958c0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5585281b6660 con 0x55852829aa30 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55852829d7a0 con 0x558528294ed0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.063+0000 7f4e7fdde740 10 client.0 _ll_register_callbacks cb 0x5585281b6050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e76800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40062/0 (socket says 172.21.15.59:40062) 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 -- 172.21.15.59:0/753493755 learned_addr learned my addr 172.21.15.59:0/753493755 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75400640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x55852829f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 -- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 msgr2=0x5585282a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 -- 172.21.15.59:0/753493755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 msgr2=0x55852829f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x55852829f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 -- 172.21.15.59:0/753493755 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585281c5e80 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e75e00640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4e680bc530 tx=0x7f4e68032900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19fcf599f50865a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e57e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.064+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680db070 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4e68032e30 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e680bdfe0 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4e680d6070 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 10 client.27889 fetch_fsmap learned FSMap version 73 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x558527f0f210 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e680bdfe0 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 73) ==== 8426+0+0 (secure 0 0 0) 0x7f4e6803c030 con 0x558528294ed0 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 10 client.27889 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 10 client.27889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 20 client.27889 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5585282aabe0 0x5585282ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 4 client.27889 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.065+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585281aacd0 con 0x5585282aabe0 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.066+0000 7f4e75400640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5585282aabe0 0x5585282ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.066+0000 7f4e75400640 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5585282aabe0 0x5585282ad000 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8044d8dc4f19387 server_cookie=5b1ec78c1703f82e in_seq=0 out_seq=0 2024-04-02T17:38:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.066+0000 7f4e57e00640 10 client.27889 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e57e00640 1 -- 172.21.15.59:0/753493755 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585281aacd0 con 0x5585282aabe0 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e57e00640 10 client.27889 handle_command_reply: tid=0 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 client.27889 shutdown 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 msgr2=0x55852829f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4e680bc530 tx=0x7f4e68032900 comp rx=0 tx=0).stop 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 shutdown_connections 2024-04-02T17:38:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x5585282aabe0 0x5585282ad000 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x55852829f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/753493755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x55852829f0b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e57e00640 0 client.27889 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 >> 172.21.15.59:0/753493755 conn(0x558528290440 msgr2=0x55852828f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 shutdown_connections 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.067+0000 7f4e7fdde740 1 -- 172.21.15.59:0/753493755 wait complete. 2024-04-02T17:38:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.068+0000 7f4e7fdde740 20 client.27889 trim_cache size 0 max 16384 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.083+0000 7f4e7fdde740 1 Processor -- start 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.083+0000 7f4e7fdde740 1 -- start start 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.083+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558528294ed0 0x55852829f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.083+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.083+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x55852828edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ab2c0 con 0x558528294ed0 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781acfe0 con 0x55852829aa30 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ab9f0 con 0x5585282958c0 2024-04-02T17:38:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558528294ed0 0x55852829f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40072/0 (socket says 172.21.15.59:40072) 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 -- 172.21.15.59:0/414759409 learned_addr learned my addr 172.21.15.59:0/414759409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e76800640 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x55852828edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 -- 172.21.15.59:0/414759409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 msgr2=0x55852828edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x55852828edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 -- 172.21.15.59:0/414759409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558528294ed0 msgr2=0x55852829f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558528294ed0 0x55852829f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 -- 172.21.15.59:0/414759409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558527fdb620 con 0x5585282958c0 2024-04-02T17:38:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.084+0000 7f4e75400640 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab2695b837ef2e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e62a00640 1 -- 172.21.15.59:0/414759409 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004310 con 0x5585282958c0 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e62a00640 1 -- 172.21.15.59:0/414759409 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e640044b0 con 0x5585282958c0 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e62a00640 1 -- 172.21.15.59:0/414759409 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64010430 con 0x5585282958c0 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 -- 172.21.15.59:0/414759409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 msgr2=0x5585282a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e64009260 comp rx=0 tx=0).stop 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 -- 172.21.15.59:0/414759409 shutdown_connections 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x55852828edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5585282958c0 0x5585282a65f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/414759409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558528294ed0 0x55852829f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 -- 172.21.15.59:0/414759409 >> 172.21.15.59:0/414759409 conn(0x558528290440 msgr2=0x55852829ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 -- 172.21.15.59:0/414759409 shutdown_connections 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.085+0000 7f4e7fdde740 1 -- 172.21.15.59:0/414759409 wait complete. 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.086+0000 7f4e7fdde740 1 -- start start 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.086+0000 7f4e7fdde740 1 Processor -- start 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.086+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.086+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x5585282a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e7fdde740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ac910 con 0x5585282958c0 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781ac180 con 0x55852829aa30 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e7fdde740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e781a92a0 con 0x558528294ed0 2024-04-02T17:38:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e7fdde740 10 client.0 _ll_register_callbacks cb 0x7f4e680ba350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x5585282a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:40084/0 (socket says 172.21.15.59:40084) 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 -- 172.21.15.59:0/2775390811 learned_addr learned my addr 172.21.15.59:0/2775390811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e76800640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 -- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 msgr2=0x5585282a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 -- 172.21.15.59:0/2775390811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 msgr2=0x5585282a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x5585282a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 -- 172.21.15.59:0/2775390811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585282a5380 con 0x558528294ed0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75400640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x5585282a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e75e00640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ef8e0 tx=0x7f4e68032850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93524c53b31835a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.087+0000 7f4e57e00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-02T17:38:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680db070 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4e68004340 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e640044b0 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4e680a7270 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 10 client.27895 fetch_fsmap learned FSMap version 73 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5585282a5380 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e640044b0 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 73) ==== 8426+0+0 (secure 0 0 0) 0x7f4e680d72b0 con 0x558528294ed0 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 10 client.27895 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 10 client.27895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 20 client.27895 populate_metadata read hostname 'smithi059' 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558528311220 0x558528313640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 4 client.27895 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-02T17:38:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.088+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 --> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585282082b0 con 0x558528311220 2024-04-02T17:38:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.089+0000 7f4e75400640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558528311220 0x558528313640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.089+0000 7f4e75400640 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558528311220 0x558528313640 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e78b82b952421a1 server_cookie=6e23c82c25c8def in_seq=0 out_seq=0 2024-04-02T17:38:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.089+0000 7f4e57e00640 10 client.27895 ms_handle_connect on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:33.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:32 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:33.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:32 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:33.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:32 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e57e00640 1 -- 172.21.15.59:0/2775390811 <== mds.0 v2:172.21.15.154:6838/3218637046 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585282082b0 con 0x558528311220 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e57e00640 10 client.27895 handle_command_reply: tid=0 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 client.27895 shutdown 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 msgr2=0x5585282a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ef8e0 tx=0x7f4e68032850 comp rx=0 tx=0).stop 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 shutdown_connections 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:6838/3218637046,v1:172.21.15.154:6839/3218637046] conn(0x558528311220 0x558528313640 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585282958c0 0x5585282a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55852829aa30 0x5585282a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 --2- 172.21.15.59:0/2775390811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558528294ed0 0x5585282a3840 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ef8e0 tx=0x7f4e68032850 comp rx=0 tx=0).stop 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 >> 172.21.15.59:0/2775390811 conn(0x558528290440 msgr2=0x5585282936a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.556+0000 7f4e57e00640 0 client.27895 ms_handle_reset on v2:172.21.15.154:6838/3218637046 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.557+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 shutdown_connections 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.557+0000 7f4e7fdde740 1 -- 172.21.15.59:0/2775390811 wait complete. 2024-04-02T17:38:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.557+0000 7f4e7fdde740 20 client.27895 trim_cache size 0 max 16384 2024-04-02T17:38:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-04-02T17:38:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-04-02T17:38:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-04-02T17:38:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-04-02T17:38:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 msgr2=0x7f4e5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 0x7f4e5c0a6190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x558528296120 tx=0x7f4e70009500 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 msgr2=0x7f4e78199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005940 tx=0x7f4e68032470 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 shutdown_connections 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f4e5c0a3cd0 0x7f4e5c0a6190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e78101660 0x7f4e7819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e781020e0 0x7f4e781a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 --2- 172.21.15.59:0/2862123978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e78100ca0 0x7f4e78199ff0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.561+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 >> 172.21.15.59:0/2862123978 conn(0x7f4e780fb750 msgr2=0x7f4e78190cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.562+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 shutdown_connections 2024-04-02T17:38:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-04-02T17:38:33.562+0000 7f4e7d000640 1 -- 172.21.15.59:0/2862123978 wait complete. 2024-04-02T17:38:33.680 INFO:tasks.workunit.client.0.smithi059.stdout: 10 cleanup 600 sec 2024-04-02T17:38:34.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:33 smithi059 ceph-mon[31664]: pgmap v1154: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-04-02T17:38:34.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:33 smithi164 ceph-mon[36499]: pgmap v1154: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-04-02T17:38:34.398 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:33 smithi154 ceph-mon[36479]: pgmap v1154: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-04-02T17:38:34.454 INFO:tasks.workunit.client.0.smithi059.stdout: 0 cleanup 601 sec 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Operation Count AvgLat MaxLat 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: ---------------------------------------- 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: NTCreateX 691363 3.578 1253.107 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Close 507879 0.008 3.815 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Rename 29279 16.389 1015.240 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Unlink 139589 4.088 751.959 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Deltree 18 184.698 447.484 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Mkdir 9 0.019 0.023 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Qpathinfo 626696 2.593 1663.092 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Qfileinfo 109848 0.003 0.847 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Qfsinfo 114912 0.510 30.923 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Sfileinfo 56352 0.114 125.889 2024-04-02T17:38:34.455 INFO:tasks.workunit.client.0.smithi059.stdout: Find 242326 0.350 663.671 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: WriteX 344845 0.064 440.261 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: ReadX 1083692 0.051 784.940 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: LockX 2250 3.130 252.968 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: UnlockX 2250 2.555 232.284 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: Flush 48466 12.086 1033.973 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout: 2024-04-02T17:38:34.456 INFO:tasks.workunit.client.0.smithi059.stdout:Throughput 36.2039 MB/sec 10 clients 10 procs max_latency=1663.103 ms 2024-04-02T17:38:34.728 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T17:38:34.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T17:38:36.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:35 smithi059 ceph-mon[31664]: pgmap v1155: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.7 MiB/s wr, 592 op/s 2024-04-02T17:38:36.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:35 smithi164 ceph-mon[36499]: pgmap v1155: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.7 MiB/s wr, 592 op/s 2024-04-02T17:38:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:35 smithi154 ceph-mon[36479]: pgmap v1155: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.7 MiB/s wr, 592 op/s 2024-04-02T17:38:37.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:36 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:37.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:36 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:37.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:36 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:38.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:37 smithi059 ceph-mon[31664]: pgmap v1156: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.5 MiB/s wr, 663 op/s 2024-04-02T17:38:38.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:37 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:38.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:37 smithi164 ceph-mon[36499]: pgmap v1156: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.5 MiB/s wr, 663 op/s 2024-04-02T17:38:38.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:37 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:37 smithi154 ceph-mon[36479]: pgmap v1156: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.5 MiB/s wr, 663 op/s 2024-04-02T17:38:38.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:37 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:38.758 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-02T17:38:38.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T17:38:39.978 DEBUG:teuthology.parallel:result is None 2024-04-02T17:38:39.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T17:38:40.084 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T17:38:40.084 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-02T17:38:40.094 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:38:40.121 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-02T17:38:40.135 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-02T17:38:40.135 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-02T17:38:40.136 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-02T17:38:40.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:38:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:39 smithi059 ceph-mon[31664]: pgmap v1157: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 735 op/s 2024-04-02T17:38:40.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:39 smithi164 ceph-mon[36499]: pgmap v1157: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 735 op/s 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 -- 172.21.15.59:0/2389502518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 msgr2=0x7fb8ac108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 --2- 172.21.15.59:0/2389502518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac108350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009900 tx=0x7fb89c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 -- 172.21.15.59:0/2389502518 shutdown_connections 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 --2- 172.21.15.59:0/2389502518 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac109aa0 0x7fb8ac1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 --2- 172.21.15.59:0/2389502518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac1090e0 0x7fb8ac109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 --2- 172.21.15.59:0/2389502518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac108350 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 -- 172.21.15.59:0/2389502518 >> 172.21.15.59:0/2389502518 conn(0x7fb8ac076f10 msgr2=0x7fb8ac079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 -- 172.21.15.59:0/2389502518 shutdown_connections 2024-04-02T17:38:40.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.390+0000 7fb8b1600640 1 -- 172.21.15.59:0/2389502518 wait complete. 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.391+0000 7fb8b1600640 1 Processor -- start 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.391+0000 7fb8b1600640 1 -- start start 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.391+0000 7fb8b1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b1600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac1090e0 0x7fb8ac19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac109aa0 0x7fb8ac1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac113c10 con 0x7fb8ac107f10 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b1600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac113a90 con 0x7fb8ac1090e0 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac113d90 con 0x7fb8ac109aa0 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47298/0 (socket says 172.21.15.59:47298) 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 -- 172.21.15.59:0/3672188300 learned_addr learned my addr 172.21.15.59:0/3672188300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8ab400640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac1090e0 0x7fb8ac19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8b0c00640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac109aa0 0x7fb8ac1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 -- 172.21.15.59:0/3672188300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac109aa0 msgr2=0x7fb8ac1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac109aa0 0x7fb8ac1a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 -- 172.21.15.59:0/3672188300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac1090e0 msgr2=0x7fb8ac19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac1090e0 0x7fb8ac19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.392+0000 7fb8abe00640 1 -- 172.21.15.59:0/3672188300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac1a5e80 con 0x7fb8ac107f10 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8abe00640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000c00 tx=0x7fb89c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c043020 con 0x7fb8ac107f10 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89c0054e0 con 0x7fb8ac107f10 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003bd0 con 0x7fb8ac107f10 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac1a6110 con 0x7fb8ac107f10 2024-04-02T17:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.393+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac1a66b0 con 0x7fb8ac107f10 2024-04-02T17:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.394+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac10f370 con 0x7fb8ac107f10 2024-04-02T17:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.396+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb89c046020 con 0x7fb8ac107f10 2024-04-02T17:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.397+0000 7fb8a8c00640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 0x7fb8900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.398+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fb89c0f23d0 con 0x7fb8ac107f10 2024-04-02T17:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.398+0000 7fb8ab400640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 0x7fb8900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.399+0000 7fb8ab400640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 0x7fb8900a60f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac19f770 tx=0x7fb898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:40.398 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:39 smithi154 ceph-mon[36479]: pgmap v1157: 97 pgs: 97 active+clean; 1.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 735 op/s 2024-04-02T17:38:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.403+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb89c04d020 con 0x7fb8ac107f10 2024-04-02T17:38:40.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.615+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8ac02d590 con 0x7fb8ac107f10 2024-04-02T17:38:40.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.619+0000 7fb8a8c00640 1 -- 172.21.15.59:0/3672188300 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22857 (secure 0 0 0) 0x7fb89c0becf0 con 0x7fb8ac107f10 2024-04-02T17:38:40.619 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:38:40.619 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:32:50.136490+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 msgr2=0x7fb8900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 0x7fb8900a60f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac19f770 tx=0x7fb898005f90 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 msgr2=0x7fb8ac19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000c00 tx=0x7fb89c02c650 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 shutdown_connections 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8ac109aa0 0x7fb8ac1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8ac1090e0 0x7fb8ac19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fb8900a3c30 0x7fb8900a60f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 --2- 172.21.15.59:0/3672188300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac107f10 0x7fb8ac19e260 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 >> 172.21.15.59:0/3672188300 conn(0x7fb8ac076f10 msgr2=0x7fb8ac10c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 shutdown_connections 2024-04-02T17:38:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.622+0000 7fb8b1600640 1 -- 172.21.15.59:0/3672188300 wait complete. 2024-04-02T17:38:40.643 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:38:40.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 -- 172.21.15.59:0/1793104984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 msgr2=0x7f3b14110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 --2- 172.21.15.59:0/1793104984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b14110350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009ca0 tx=0x7f3b0c02c100 comp rx=0 tx=0).stop 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 -- 172.21.15.59:0/1793104984 shutdown_connections 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 --2- 172.21.15.59:0/1793104984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b14110350 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 --2- 172.21.15.59:0/1793104984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 0x7f3b141094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 --2- 172.21.15.59:0/1793104984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b14108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.873+0000 7f3b13e00640 1 -- 172.21.15.59:0/1793104984 >> 172.21.15.59:0/1793104984 conn(0x7f3b14076ec0 msgr2=0x7f3b14079300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.874+0000 7f3b13e00640 1 -- 172.21.15.59:0/1793104984 shutdown_connections 2024-04-02T17:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.874+0000 7f3b13e00640 1 -- 172.21.15.59:0/1793104984 wait complete. 2024-04-02T17:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.874+0000 7f3b13e00640 1 Processor -- start 2024-04-02T17:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 -- start start 2024-04-02T17:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 0x7f3b1419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b141a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14113bd0 con 0x7f3b14107f20 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14113a50 con 0x7f3b14109a30 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b14113d50 con 0x7f3b14109070 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b12a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b12a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60080/0 (socket says 172.21.15.59:60080) 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b12a00640 1 -- 172.21.15.59:0/3406693737 learned_addr learned my addr 172.21.15.59:0/3406693737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b12000640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 0x7f3b1419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.875+0000 7f3b13400640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b141a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 -- 172.21.15.59:0/3406693737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 msgr2=0x7f3b1419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 0x7f3b1419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 -- 172.21.15.59:0/3406693737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 msgr2=0x7f3b141a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b141a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 -- 172.21.15.59:0/3406693737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b141a5dd0 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b13400640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b141a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.876+0000 7f3b12a00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9f0 tx=0x7f3b0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.877+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400cd80 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.877+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b141a60c0 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.877+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b04010910 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.877+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141a6660 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.877+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04026060 con 0x7f3b14107f20 2024-04-02T17:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.878+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1410f230 con 0x7f3b14107f20 2024-04-02T17:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.880+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3b04010ac0 con 0x7f3b14107f20 2024-04-02T17:38:40.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.881+0000 7f3b0b400640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 0x7f3af80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:40.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.882+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f3b040d0890 con 0x7f3b14107f20 2024-04-02T17:38:40.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.882+0000 7f3b12000640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 0x7f3af80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.883+0000 7f3b12000640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 0x7f3af80a6070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3b1419f6c0 tx=0x7f3b00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:40.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:40.886+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3b0409cb60 con 0x7f3b14107f20 2024-04-02T17:38:41.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.098+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b1402d5d0 con 0x7f3b14107f20 2024-04-02T17:38:41.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.099+0000 7f3b0b400640 1 -- 172.21.15.59:0/3406693737 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f3b040bd0f0 con 0x7f3b14107f20 2024-04-02T17:38:41.099 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:38:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 msgr2=0x7f3af80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 0x7f3af80a6070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3b1419f6c0 tx=0x7f3b00005f90 comp rx=0 tx=0).stop 2024-04-02T17:38:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 msgr2=0x7f3b1419e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400e9f0 tx=0x7f3b0400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 shutdown_connections 2024-04-02T17:38:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b14109a30 0x7f3b141a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b14109070 0x7f3b1419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f3af80a3bb0 0x7f3af80a6070 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 --2- 172.21.15.59:0/3406693737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b14107f20 0x7f3b1419e1b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 >> 172.21.15.59:0/3406693737 conn(0x7f3b14076ec0 msgr2=0x7f3b1410c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 shutdown_connections 2024-04-02T17:38:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.102+0000 7f3b13e00640 1 -- 172.21.15.59:0/3406693737 wait complete. 2024-04-02T17:38:41.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:38:41.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:40 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3672188300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:38:41.300 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:40 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3672188300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:38:41.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:40 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3672188300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:38:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 -- 172.21.15.59:0/3038407163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 msgr2=0x7f7d24101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 --2- 172.21.15.59:0/3038407163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d24101e20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009900 tx=0x7f7d1402bc30 comp rx=0 tx=0).stop 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 -- 172.21.15.59:0/3038407163 shutdown_connections 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 --2- 172.21.15.59:0/3038407163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 --2- 172.21.15.59:0/3038407163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 0x7f7d24109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 --2- 172.21.15.59:0/3038407163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d24101e20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.356+0000 7f7d2a200640 1 -- 172.21.15.59:0/3038407163 >> 172.21.15.59:0/3038407163 conn(0x7f7d240fcf70 msgr2=0x7f7d240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.357+0000 7f7d2a200640 1 -- 172.21.15.59:0/3038407163 shutdown_connections 2024-04-02T17:38:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.357+0000 7f7d2a200640 1 -- 172.21.15.59:0/3038407163 wait complete. 2024-04-02T17:38:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 Processor -- start 2024-04-02T17:38:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 -- start start 2024-04-02T17:38:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 0x7f7d24074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d680 con 0x7f7d24102bf0 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d500 con 0x7f7d241035b0 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.358+0000 7f7d2a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d800 con 0x7f7d24101a00 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d28e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d28e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:56366/0 (socket says 172.21.15.59:56366) 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60088/0 (socket says 172.21.15.59:60088) 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d28e00640 1 -- 172.21.15.59:0/52037850 learned_addr learned my addr 172.21.15.59:0/52037850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d29800640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 0x7f7d24074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 -- 172.21.15.59:0/52037850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 msgr2=0x7f7d240758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 -- 172.21.15.59:0/52037850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 msgr2=0x7f7d24074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 0x7f7d24074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d1be00640 1 -- 172.21.15.59:0/52037850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d24074cc0 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.359+0000 7f7d28e00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d1be00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e9f0 tx=0x7f7d0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00ccc0 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c010910 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241a1e40 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c018960 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.360+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a22a0 con 0x7f7d24102bf0 2024-04-02T17:38:41.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.361+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d24108d40 con 0x7f7d24102bf0 2024-04-02T17:38:41.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.364+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d0c010ac0 con 0x7f7d24102bf0 2024-04-02T17:38:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.367+0000 7f7d22000640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 0x7f7d080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.367+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f7d0c0d0c50 con 0x7f7d24102bf0 2024-04-02T17:38:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.367+0000 7f7d28e00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 0x7f7d080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.368+0000 7f7d28e00640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 0x7f7d080a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7d1402c8e0 tx=0x7f7d14005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:41.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.372+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d0c09cf20 con 0x7f7d24102bf0 2024-04-02T17:38:41.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.581+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d2402d5c0 con 0x7f7d24102bf0 2024-04-02T17:38:41.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.583+0000 7f7d22000640 1 -- 172.21.15.59:0/52037850 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7d0c0a1220 con 0x7f7d24102bf0 2024-04-02T17:38:41.582 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 msgr2=0x7f7d080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 0x7f7d080a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7d1402c8e0 tx=0x7f7d14005b50 comp rx=0 tx=0).stop 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 msgr2=0x7f7d24073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e9f0 tx=0x7f7d0c00eeb0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 shutdown_connections 2024-04-02T17:38:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7d080a3bb0 0x7f7d080a6070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d24102bf0 0x7f7d24073ef0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241035b0 0x7f7d24074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 --2- 172.21.15.59:0/52037850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d24101a00 0x7f7d240758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.585+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 >> 172.21.15.59:0/52037850 conn(0x7f7d240fcf70 msgr2=0x7f7d24105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.586+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 shutdown_connections 2024-04-02T17:38:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.586+0000 7f7d2a200640 1 -- 172.21.15.59:0/52037850 wait complete. 2024-04-02T17:38:41.606 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.836+0000 7fcb42200640 1 -- 172.21.15.59:0/352385573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c103c00 msgr2=0x7fcb3c104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.836+0000 7fcb42200640 1 --2- 172.21.15.59:0/352385573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c103c00 0x7fcb3c104020 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009900 tx=0x7fcb3002bc30 comp rx=0 tx=0).stop 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 -- 172.21.15.59:0/352385573 shutdown_connections 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 --2- 172.21.15.59:0/352385573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 --2- 172.21.15.59:0/352385573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c104df0 0x7fcb3c105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 --2- 172.21.15.59:0/352385573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c103c00 0x7fcb3c104020 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 -- 172.21.15.59:0/352385573 >> 172.21.15.59:0/352385573 conn(0x7fcb3c0ff170 msgr2=0x7fcb3c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 -- 172.21.15.59:0/352385573 shutdown_connections 2024-04-02T17:38:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.837+0000 7fcb42200640 1 -- 172.21.15.59:0/352385573 wait complete. 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 Processor -- start 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 -- start start 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c103c00 0x7fcb3c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 0x7fcb3c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10f8c0 con 0x7fcb3c104df0 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10f740 con 0x7fcb3c103c00 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.838+0000 7fcb42200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10fa40 con 0x7fcb3c1057b0 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 0x7fcb3c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 0x7fcb3c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60108/0 (socket says 172.21.15.59:60108) 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb3be00640 1 -- 172.21.15.59:0/2387760994 learned_addr learned my addr 172.21.15.59:0/2387760994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb40e00640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c103c00 0x7fcb3c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 -- 172.21.15.59:0/2387760994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c103c00 msgr2=0x7fcb3c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c103c00 0x7fcb3c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 -- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 msgr2=0x7fcb3c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 0x7fcb3c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 -- 172.21.15.59:0/2387760994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c1a1b50 con 0x7fcb3c1057b0 2024-04-02T17:38:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.839+0000 7fcb41800640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c1a14e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009fd0 tx=0x7fcb3400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.840+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400ce20 con 0x7fcb3c1057b0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.840+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb34005020 con 0x7fcb3c1057b0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.840+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1a1e40 con 0x7fcb3c1057b0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.840+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34010720 con 0x7fcb3c1057b0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.840+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a22b0 con 0x7fcb3c1057b0 2024-04-02T17:38:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.841+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c10af20 con 0x7fcb3c1057b0 2024-04-02T17:38:41.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.843+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb34005540 con 0x7fcb3c1057b0 2024-04-02T17:38:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.844+0000 7fcb39600640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 0x7fcb240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.845+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fcb340d0b70 con 0x7fcb3c1057b0 2024-04-02T17:38:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.845+0000 7fcb40e00640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 0x7fcb240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.846+0000 7fcb40e00640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 0x7fcb240a6070 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcb3002c170 tx=0x7fcb3002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:41.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:41.849+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcb3409d490 con 0x7fcb3c1057b0 2024-04-02T17:38:42.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.060+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb3c062700 con 0x7fcb3c1057b0 2024-04-02T17:38:42.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.061+0000 7fcb39600640 1 -- 172.21.15.59:0/2387760994 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fcb3409cd90 con 0x7fcb3c1057b0 2024-04-02T17:38:42.061 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:38:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 msgr2=0x7fcb240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 0x7fcb240a6070 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcb3002c170 tx=0x7fcb3002df90 comp rx=0 tx=0).stop 2024-04-02T17:38:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 msgr2=0x7fcb3c1a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c1a14e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009fd0 tx=0x7fcb3400ef10 comp rx=0 tx=0).stop 2024-04-02T17:38:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 shutdown_connections 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fcb240a3bb0 0x7fcb240a6070 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c104df0 0x7fcb3c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb3c1057b0 0x7fcb3c1a14e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 --2- 172.21.15.59:0/2387760994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb3c103c00 0x7fcb3c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.064+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 >> 172.21.15.59:0/2387760994 conn(0x7fcb3c0ff170 msgr2=0x7fcb3c1080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.065+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 shutdown_connections 2024-04-02T17:38:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.065+0000 7fcb42200640 1 -- 172.21.15.59:0/2387760994 wait complete. 2024-04-02T17:38:42.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-02T17:38:42.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:41 smithi059 ceph-mon[31664]: pgmap v1158: 97 pgs: 97 active+clean; 1.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T17:38:42.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3406693737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:41 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/52037850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 -- 172.21.15.59:0/2534135640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c81058f0 msgr2=0x7f92c810c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 --2- 172.21.15.59:0/2534135640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c81058f0 0x7f92c810c180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009ca0 tx=0x7f92c002c100 comp rx=0 tx=0).stop 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 -- 172.21.15.59:0/2534135640 shutdown_connections 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 --2- 172.21.15.59:0/2534135640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c81058f0 0x7f92c810c180 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 --2- 172.21.15.59:0/2534135640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 0x7f92c81053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 --2- 172.21.15.59:0/2534135640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c8103d40 0x7f92c8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 -- 172.21.15.59:0/2534135640 >> 172.21.15.59:0/2534135640 conn(0x7f92c80ff2f0 msgr2=0x7f92c8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 -- 172.21.15.59:0/2534135640 shutdown_connections 2024-04-02T17:38:42.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.314+0000 7f92ce600640 1 -- 172.21.15.59:0/2534135640 wait complete. 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 Processor -- start 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 -- start start 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 0x7f92c819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 0x7f92c819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92c810f9c0 con 0x7f92c8103d40 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92c810f840 con 0x7f92c81058f0 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.315+0000 7f92ce600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92c810fb40 con 0x7f92c8104f30 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 0x7f92c819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:42.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:59154/0 (socket says 172.21.15.59:59154) 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 -- 172.21.15.59:0/2511787553 learned_addr learned my addr 172.21.15.59:0/2511787553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c6a00640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 0x7f92c819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 -- 172.21.15.59:0/2511787553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 msgr2=0x7f92c819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 0x7f92c819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 -- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 msgr2=0x7f92c819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 0x7f92c819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 -- 172.21.15.59:0/2511787553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c81a1c20 con 0x7f92c81058f0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7400640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 0x7f92c819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.316+0000 7f92c7e00640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f92c0037040 tx=0x7f92c0035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.317+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0039040 con 0x7f92c81058f0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.317+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c0044070 con 0x7f92c81058f0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.317+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c81a1eb0 con 0x7f92c81058f0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.317+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004460 con 0x7f92c81058f0 2024-04-02T17:38:42.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.317+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a2310 con 0x7f92c81058f0 2024-04-02T17:38:42.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.318+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c810b000 con 0x7f92c81058f0 2024-04-02T17:38:42.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.320+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92c0004600 con 0x7f92c81058f0 2024-04-02T17:38:42.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.321+0000 7f92bfe00640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 0x7f92ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:38:42.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.321+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f92c00f2ba0 con 0x7f92c81058f0 2024-04-02T17:38:42.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.322+0000 7f92c7400640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 0x7f92ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:38:42.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:41 smithi164 ceph-mon[36499]: pgmap v1158: 97 pgs: 97 active+clean; 1.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T17:38:42.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3406693737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:41 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/52037850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.322+0000 7f92c7400640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 0x7f92ac0a6070 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104d90 tx=0x7f92b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:38:42.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.326+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f92c00bf4c0 con 0x7f92c81058f0 2024-04-02T17:38:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:41 smithi154 ceph-mon[36479]: pgmap v1158: 97 pgs: 97 active+clean; 1.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T17:38:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3406693737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:41 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/52037850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:38:42.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.588+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f92c8062700 con 0x7f92c81058f0 2024-04-02T17:38:42.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.597+0000 7f92bfe00640 1 -- 172.21.15.59:0/2511787553 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+745530 (secure 0 0 0) 0x7f92c00bedc0 con 0x7f92c81058f0 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 18, 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout: "active_gid": 14152, 2024-04-02T17:38:42.611 INFO:teuthology.orchestra.run.smithi059.stdout: "active_name": "x", 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addrs": { 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1634390325 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1634390325 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:42.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addr": "172.21.15.59:6801/1634390325", 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "active_change": "2024-04-02T17:02:49.350890+0000", 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "active_mgr_features": 4540701547738038271, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "available": true, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "standbys": [ 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 24109, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "z", 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.613 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.615 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.616 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.618 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-04-02T17:38:42.620 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.624 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.625 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.628 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.629 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:42.630 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:42.631 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.632 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-04-02T17:38:42.633 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.635 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.636 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.639 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.640 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T17:38:42.641 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-04-02T17:38:42.642 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.643 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:42.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-04-02T17:38:42.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-04-02T17:38:42.646 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.647 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.648 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.649 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.651 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.652 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T17:38:42.654 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.656 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.660 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-04-02T17:38:42.661 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.664 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.668 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.671 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-04-02T17:38:42.672 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-04-02T17:38:42.673 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.674 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.675 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.676 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.678 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.679 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:42.680 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T17:38:42.681 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.682 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.683 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.685 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-04-02T17:38:42.686 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.687 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.688 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.689 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.690 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.691 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T17:38:42.692 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.693 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T17:38:42.696 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.698 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.700 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.703 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-04-02T17:38:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-04-02T17:38:42.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-04-02T17:38:42.706 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.709 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.710 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.711 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.712 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.713 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.714 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.715 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-04-02T17:38:42.716 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T17:38:42.718 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T17:38:42.719 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T17:38:42.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.722 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.725 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.730 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T17:38:42.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.735 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T17:38:42.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T17:38:42.737 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.738 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.739 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T17:38:42.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T17:38:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T17:38:42.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T17:38:42.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T17:38:42.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.749 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T17:38:42.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T17:38:42.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T17:38:42.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T17:38:42.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T17:38:42.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.757 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.758 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.759 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.760 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.761 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.762 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.763 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.770 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.771 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.772 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.774 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:42.775 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-04-02T17:38:42.776 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.778 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.779 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:42.780 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.781 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-04-02T17:38:42.782 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T17:38:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T17:38:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-04-02T17:38:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T17:38:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T17:38:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-04-02T17:38:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-04-02T17:38:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.830 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-04-02T17:38:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-04-02T17:38:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.900 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.903 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.904 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.905 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.906 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.907 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.908 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.909 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-04-02T17:38:42.910 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.911 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:42.913 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.915 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.918 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.920 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.921 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.922 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.923 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.924 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.925 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.926 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.928 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:42.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:42.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.932 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.933 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.934 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.937 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.939 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.940 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T17:38:42.942 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.943 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.944 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.945 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.949 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.950 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.951 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.952 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.953 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.954 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.955 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.956 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.957 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.958 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.959 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.960 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.961 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.962 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.963 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.965 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.966 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.967 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.968 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.970 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.972 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.973 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.974 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.975 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.978 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.979 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.980 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.981 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.982 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:42.983 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:42.984 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.985 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.986 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-04-02T17:38:42.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.988 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.989 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.990 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.991 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.992 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.993 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.994 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:42.998 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:42.999 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.000 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.002 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.003 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.004 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.005 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.006 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.008 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.009 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.010 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.011 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.012 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.014 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-04-02T17:38:43.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-04-02T17:38:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-04-02T17:38:43.018 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T17:38:43.019 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.020 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.021 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.022 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.023 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.024 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.025 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.026 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.027 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.028 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.029 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.030 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.031 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 24116, 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "y", 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-04-02T17:38:43.032 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.033 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.034 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.035 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.038 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.041 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.042 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.044 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.045 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.046 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.047 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-04-02T17:38:43.048 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:43.049 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.050 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-04-02T17:38:43.052 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.053 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:43.054 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.055 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.057 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.058 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T17:38:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-04-02T17:38:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T17:38:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T17:38:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.077 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-04-02T17:38:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-04-02T17:38:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T17:38:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-04-02T17:38:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-04-02T17:38:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-04-02T17:38:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T17:38:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T17:38:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T17:38:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-04-02T17:38:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-04-02T17:38:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T17:38:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T17:38:43.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.178 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.179 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T17:38:43.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T17:38:43.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T17:38:43.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.184 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.194 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.195 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-04-02T17:38:43.198 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.201 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.202 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:43.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.205 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.207 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-04-02T17:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.222 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.223 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.228 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-04-02T17:38:43.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.237 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-04-02T17:38:43.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-04-02T17:38:43.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.241 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-04-02T17:38:43.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.246 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.254 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-04-02T17:38:43.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-04-02T17:38:43.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.259 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.261 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.262 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.263 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.265 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.269 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.271 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-04-02T17:38:43.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T17:38:43.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.276 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.283 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.284 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.285 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.286 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.287 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T17:38:43.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.292 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.294 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-04-02T17:38:43.297 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.299 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.301 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.302 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-04-02T17:38:43.303 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.304 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.305 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.307 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.309 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.312 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-04-02T17:38:43.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.321 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.325 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.326 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.327 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.328 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.329 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-04-02T17:38:43.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-04-02T17:38:43.338 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.339 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.341 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-04-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.353 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.356 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.359 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.360 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.361 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.363 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.364 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.367 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.368 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.373 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.374 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.379 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-04-02T17:38:43.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.382 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-04-02T17:38:43.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.391 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.397 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.398 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.399 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.400 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.401 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.402 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.404 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.408 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.409 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.410 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.411 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-04-02T17:38:43.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-04-02T17:38:43.413 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-04-02T17:38:43.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.421 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-04-02T17:38:43.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-04-02T17:38:43.423 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-04-02T17:38:43.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.427 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.428 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.429 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.430 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.433 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.434 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.436 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.439 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.440 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.444 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.446 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.449 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.451 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-04-02T17:38:43.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.458 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "modules": [ 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "iostat", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "nfs", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "restful" 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.459 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.461 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.463 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.464 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.465 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.466 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.472 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.473 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.475 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-04-02T17:38:43.476 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-04-02T17:38:43.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.479 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.481 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.485 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.487 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.488 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.491 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.492 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.493 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-04-02T17:38:43.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.495 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.496 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-04-02T17:38:43.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T17:38:43.500 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T17:38:43.501 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T17:38:43.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.504 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.507 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.508 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.511 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-04-02T17:38:43.512 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.515 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.518 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-04-02T17:38:43.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T17:38:43.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.526 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.528 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.530 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.533 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.534 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T17:38:43.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T17:38:43.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.539 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.541 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.544 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.545 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.546 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-04-02T17:38:43.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-04-02T17:38:43.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.549 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.550 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-04-02T17:38:43.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-04-02T17:38:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-04-02T17:38:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-04-02T17:38:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.567 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.568 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.571 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.573 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.574 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.576 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T17:38:43.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.578 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.579 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.580 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.581 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.582 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T17:38:43.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T17:38:43.584 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T17:38:43.585 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T17:38:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T17:38:43.587 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T17:38:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T17:38:43.589 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T17:38:43.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.593 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T17:38:43.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T17:38:43.598 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.599 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.600 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.601 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.602 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.603 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.604 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.611 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.615 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.616 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.618 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.619 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.676 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.677 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.678 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-04-02T17:38:43.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:43.681 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.683 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.686 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.687 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.689 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.690 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.691 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.692 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.693 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.694 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.695 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.696 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.697 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.699 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.700 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.702 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.703 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.706 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.712 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.714 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.715 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.716 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.717 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.718 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.719 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.720 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.722 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.723 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.724 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.725 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.726 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.728 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.729 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.730 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.731 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.732 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-04-02T17:38:43.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.735 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.739 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.740 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.742 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.744 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.745 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-04-02T17:38:43.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-04-02T17:38:43.747 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-04-02T17:38:43.748 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-04-02T17:38:43.749 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.750 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.753 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.755 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.758 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.760 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.763 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.764 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.765 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:43.767 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.768 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-04-02T17:38:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-04-02T17:38:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.791 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.792 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.794 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-04-02T17:38:43.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-04-02T17:38:43.796 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:43.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.799 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.800 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.801 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-04-02T17:38:43.805 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.807 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.810 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.811 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.812 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-04-02T17:38:43.815 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.816 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.818 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.819 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-04-02T17:38:43.820 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-04-02T17:38:43.821 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.824 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.827 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.828 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.829 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-04-02T17:38:43.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.833 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.834 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.835 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.836 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.837 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.838 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.839 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.840 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.841 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.843 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.844 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.845 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.846 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.847 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.849 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.852 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-04-02T17:38:43.855 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.856 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.858 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.859 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.861 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.862 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-04-02T17:38:43.863 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.865 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.869 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.872 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.875 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.876 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.877 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.878 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.879 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.880 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.881 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.883 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.884 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.885 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.886 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.887 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-04-02T17:38:43.888 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.889 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.890 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.891 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.892 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.893 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.894 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.895 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.896 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.897 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.898 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.899 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.900 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.901 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.902 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.903 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.904 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.906 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.908 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.909 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.910 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.911 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-04-02T17:38:43.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-04-02T17:38:43.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-04-02T17:38:43.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.918 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.919 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.920 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.921 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-04-02T17:38:43.922 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.923 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.925 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-04-02T17:38:43.926 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.927 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.929 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.930 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.932 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.933 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.934 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.935 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.937 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.941 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-04-02T17:38:43.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.944 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.947 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.948 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.949 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-04-02T17:38:43.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-04-02T17:38:43.952 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-04-02T17:38:43.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "services": { 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard": "https://172.21.15.59:8443/" 2024-04-02T17:38:43.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "always_on_modules": { 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "octopus": [ 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "pacific": [ 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-04-02T17:38:43.955 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "quincy": [ 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-04-02T17:38:43.956 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "reef": [ 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-04-02T17:38:43.957 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "squid": [ 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.958 INFO:teuthology.orchestra.run.smithi059.stdout: "last_failure_osd_epoch": 3, 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "active_clients": [ 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2942673111 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "libcephsqlite", 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1842613387 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-04-02T17:38:43.960 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 863400844 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-04-02T17:38:43.961 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3076784263 2024-04-02T17:38:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:43.962 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-02T17:38:43.962 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:38:43.963 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:42 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:43.963 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:42 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2387760994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:38:43.963 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:42 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2511787553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T17:38:43.964 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:42 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:43.964 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:42 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2387760994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:38:43.964 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:42 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2511787553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T17:38:43.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 msgr2=0x7f92ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:43.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 0x7f92ac0a6070 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104d90 tx=0x7f92b8005f90 comp rx=0 tx=0).stop 2024-04-02T17:38:43.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 msgr2=0x7f92c81a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f92c0037040 tx=0x7f92c0035d90 comp rx=0 tx=0).stop 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 shutdown_connections 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92c81058f0 0x7f92c81a15b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92c8104f30 0x7f92c819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f92ac0a3bb0 0x7f92ac0a6070 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 --2- 172.21.15.59:0/2511787553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92c8103d40 0x7f92c819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 >> 172.21.15.59:0/2511787553 conn(0x7f92c80ff2f0 msgr2=0x7f92c81081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:38:43.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 shutdown_connections 2024-04-02T17:38:43.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:38:42.612+0000 7f92ce600640 1 -- 172.21.15.59:0/2511787553 wait complete. 2024-04-02T17:38:43.973 DEBUG:tasks.check_counter:Getting stats from a 2024-04-02T17:38:43.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-04-02T17:38:43.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:42 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:43.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:42 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2387760994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:38:43.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:42 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2511787553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T17:38:44.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:43 smithi059 ceph-mon[31664]: pgmap v1159: 97 pgs: 97 active+clean; 960 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 767 op/s 2024-04-02T17:38:44.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:43 smithi164 ceph-mon[36499]: pgmap v1159: 97 pgs: 97 active+clean; 960 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 767 op/s 2024-04-02T17:38:44.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:43 smithi154 ceph-mon[36479]: pgmap v1159: 97 pgs: 97 active+clean; 960 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 767 op/s 2024-04-02T17:38:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:38:46.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:45 smithi059 ceph-mon[31664]: pgmap v1160: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-04-02T17:38:46.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:45 smithi164 ceph-mon[36499]: pgmap v1160: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-04-02T17:38:46.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:45 smithi154 ceph-mon[36479]: pgmap v1160: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 690 op/s 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 454, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 451, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 548144, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 279296, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.143756694, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.064107869, 2024-04-02T17:38:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.053388504, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.019637785, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 450, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.049157886, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000109239 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 548144, 2024-04-02T17:38:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 279296 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 345, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 320, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 216496, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.058882953, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.047991327, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.004449114, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 343, 2024-04-02T17:38:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.041693542, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000121555 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 216496 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3024, 2024-04-02T17:38:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 448, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.001192361, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000048514, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.001051700, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001744088, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000581362 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-04-02T17:38:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:38:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:38:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:38:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:38:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:38:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 150766, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 31, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-04-02T17:38:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2024-04-02T17:38:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2024-04-02T17:38:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 456, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 483930, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 456, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 456, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 483930, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:47.502 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:38:48.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:48 smithi059 ceph-mon[31664]: pgmap v1161: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 542 op/s 2024-04-02T17:38:48.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:48 smithi164 ceph-mon[36499]: pgmap v1161: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 542 op/s 2024-04-02T17:38:48.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:48 smithi154 ceph-mon[36479]: pgmap v1161: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 542 op/s 2024-04-02T17:38:48.424 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-02T17:38:48.424 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-02T17:38:48.424 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-02T17:38:48.425 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-02T17:38:48.425 DEBUG:tasks.check_counter:Getting stats from d 2024-04-02T17:38:48.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-04-02T17:38:48.906 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:38:50.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:50 smithi059 ceph-mon[31664]: pgmap v1162: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 7.0 MiB/s wr, 392 op/s 2024-04-02T17:38:50.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:50 smithi164 ceph-mon[36499]: pgmap v1162: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 7.0 MiB/s wr, 392 op/s 2024-04-02T17:38:50.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:50 smithi154 ceph-mon[36479]: pgmap v1162: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 7.0 MiB/s wr, 392 op/s 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1888, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1890, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 159867499, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 600448, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 9, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-04-02T17:38:51.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.691292343, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.118176270, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.438291742, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.101920586, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1888, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.165361351, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000087585 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2186, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2133, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 106183160, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 814499, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.692734996, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.169103778, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.367351575, 2024-04-02T17:38:51.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.121638803, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2132, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.213538882, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000100158 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 904880, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 285024 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2601, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2947, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 109471338, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 1520600, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.780051795, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.257206790, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.327864744, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.141582269, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2946, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.302317609, 2024-04-02T17:38:51.330 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000102619 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 695072 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.331 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 10356, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.539110430, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000052057 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.332 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-04-02T17:38:51.333 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 38816, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 10161, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 28655, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 24299, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-04-02T17:38:51.334 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-04-02T17:38:51.335 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-04-02T17:38:51.336 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-04-02T17:38:51.337 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:38:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-04-02T17:38:51.339 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 186321, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 139, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 17, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 143884539, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 377109838, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 377108700, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.340 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 186321 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 114, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 38832, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 38718, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 15, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 94, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 79, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 38816, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 39218, 2024-04-02T17:38:51.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 402, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 88616, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215416 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-04-02T17:38:51.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-04-02T17:38:51.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-04-02T17:38:51.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.350 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-04-02T17:38:51.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 2, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 1, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-04-02T17:38:51.354 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 2832, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:38:51.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-04-02T17:38:51.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 248366, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 189, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-04-02T17:38:51.357 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7920, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 167, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 27621194, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 168688, 2024-04-02T17:38:51.358 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 6149, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-04-02T17:38:51.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 6149, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6149, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 24.335710131, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003957669 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6149, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 6150 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 6149, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 6149, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-04-02T17:38:51.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 3322, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 2825, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-04-02T17:38:51.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-04-02T17:38:51.362 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 93, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-04-02T17:38:51.363 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 6, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 11, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 1, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-04-02T17:38:51.364 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:51.365 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 6671, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 374891990, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 6671, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 6671, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 374891990, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.366 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 6149, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 372917499, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 6149, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 372917499, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 6149, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 6149, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 6149, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 6149, 2024-04-02T17:38:51.368 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:51.369 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55aaaef22780": { 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:51.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:51.371 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:38:52.266 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-04-02T17:38:52.266 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-04-02T17:38:52.266 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-04-02T17:38:52.266 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-04-02T17:38:52.267 DEBUG:tasks.check_counter:Getting stats from g 2024-04-02T17:38:52.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-04-02T17:38:52.297 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:52 smithi059 ceph-mon[31664]: pgmap v1163: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-02T17:38:52.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:52 smithi059 ceph-mon[31664]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:52.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:52 smithi164 ceph-mon[36499]: pgmap v1163: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-02T17:38:52.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:52 smithi164 ceph-mon[36499]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:52 smithi154 ceph-mon[36479]: pgmap v1163: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 3.1 MiB/s wr, 226 op/s 2024-04-02T17:38:52.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:52 smithi154 ceph-mon[36479]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T17:38:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:38:54.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:54 smithi059 ceph-mon[31664]: pgmap v1164: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 709 KiB/s wr, 123 op/s 2024-04-02T17:38:54.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:54 smithi164 ceph-mon[36499]: pgmap v1164: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 709 KiB/s wr, 123 op/s 2024-04-02T17:38:54.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:54 smithi154 ceph-mon[36479]: pgmap v1164: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 709 KiB/s wr, 123 op/s 2024-04-02T17:38:54.992 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:38:54.992 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:38:54.992 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 497076, 2024-04-02T17:38:54.992 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 459472, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 121911169, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 232691196, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 11, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 63.636286856, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 27.956205366, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 40.134919102, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 3.148117457, 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:54.993 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 459471, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 52.136616254, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000113470 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 874558, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.263357152, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000301 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.994 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 436000, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 612973, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 158942527, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 594651882, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 22, 2024-04-02T17:38:54.995 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 6, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 128.322555891, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 88.194255943, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 48.375034568, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 3.043478446, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 612971, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 76.342650050, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000124545 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:54.996 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 865573, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.404102531, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000466 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 150138464, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 458636480 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 373612, 2024-04-02T17:38:54.997 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 374328, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 93657078, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 189684727, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 9, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 8, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 38.288778672, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 17.414468356, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 23.740499853, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 1.501999151, 2024-04-02T17:38:54.998 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 374327, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 49.141676167, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000131280 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 682184, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.161034535, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000236 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:54.999 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 932144, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 310080 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 96336, 2024-04-02T17:38:55.000 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 195.018984669, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002024362 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 67631, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 16.312888695, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000241204 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 304125, 2024-04-02T17:38:55.001 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 233841, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 233841, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1205.161989046, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005153766 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 70284, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 44, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 2, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 2, 2024-04-02T17:38:55.002 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 297, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 133, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 98, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 66, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 106, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 550, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 43, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 43, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 6, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 491435, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 231378, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 70280, 2024-04-02T17:38:55.003 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 209, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 5091, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 20481, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 10, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 128, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 11, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 443, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 10, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-04-02T17:38:55.004 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 4, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 234, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 10, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 10, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 24, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 20639, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 132629, 2024-04-02T17:38:55.005 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 68542, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 26118, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 11931, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 175761 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 1019, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 1696, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 916, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 1650, 2024-04-02T17:38:55.006 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 1435, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 56, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 37538, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 37482, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:38:55.007 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 12, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 4, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 177927, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 75046, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 73741, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 104186, 2024-04-02T17:38:55.008 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 1305, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 221, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 93, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 91, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 130, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 9, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 2, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 133506801, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 346045879, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 52055, 2024-04-02T17:38:55.009 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 461.455526116, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.008864768 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 91, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 37864, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 37773, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 47, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 874, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 827, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 319, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 67495, 2024-04-02T17:38:55.010 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 67176, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 43, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 40818, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 40775, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 80040, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223608 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 437301, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 3651, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 304125, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 4498, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 22704, 2024-04-02T17:38:55.011 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 37391, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 234.424099341, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.006269532 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 15828, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 222.974535709, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.014087347 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-04-02T17:38:55.012 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 71572, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 201.239246394, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002811703 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.013 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.014 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.015 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.016 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000887835, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000443917 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 63505, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 318.693618262, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005018401 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6961, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 49.909597684, 2024-04-02T17:38:55.018 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.007169889 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 20, 2024-04-02T17:38:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.307142028, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.015357101 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-04-02T17:38:55.020 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 28, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.544374353, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.019441941 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1016, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.612242545, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003555356 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.021 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-04-02T17:38:55.022 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 37518, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 173.456244895, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.004623280 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 2, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 1, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 20862, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 20862, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 1559, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-04-02T17:38:55.024 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:38:55.025 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:38:55.026 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:38:55.027 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 303144, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 191, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 352, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 4, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-04-02T17:38:55.028 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 7920, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 167, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 545857, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 11995, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-04-02T17:38:55.029 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 94816, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 349786489, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 33, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 94816, 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-04-02T17:38:55.030 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 94816, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 940.034815137, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.009914305 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 94816, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 97175 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 94816, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 1473, 2024-04-02T17:38:55.031 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 93343, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 4, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 138, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 1463, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 53518, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 788, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 2, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 37621, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-04-02T17:38:55.032 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 10, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 276, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-04-02T17:38:55.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-04-02T17:38:55.034 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 93, 2024-04-02T17:38:55.035 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 6, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 12, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 2474, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 881 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-04-02T17:38:55.036 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 6, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 37482, 2024-04-02T17:38:55.037 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 37482, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 1306684, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 237878261, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 1306684, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1306684, 2024-04-02T17:38:55.038 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 237878261, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:55.039 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 94816, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 353572171, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 94816, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 353572171, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-04-02T17:38:55.040 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 94816, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 94816, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 94816, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 94816, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:55.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 37482, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 3785682, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 37482, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 1463, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 3785682, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55ae03dbe500": { 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 177927, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 341851575, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 177927, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 52055, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 341851575, 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:55.043 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:55.044 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:55.044 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:55.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:55.044 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:38:55.918 INFO:tasks.check_counter:Daemon mds.g mds.exported=10 2024-04-02T17:38:55.918 INFO:tasks.check_counter:Daemon mds.g mds.imported=11 2024-04-02T17:38:55.918 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=1019 2024-04-02T17:38:55.918 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=1696 2024-04-02T17:38:55.919 DEBUG:tasks.check_counter:Getting stats from j 2024-04-02T17:38:55.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-04-02T17:38:56.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:56 smithi059 ceph-mon[31664]: pgmap v1165: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 KiB/s wr, 41 op/s 2024-04-02T17:38:56.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:56 smithi164 ceph-mon[36499]: pgmap v1165: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 KiB/s wr, 41 op/s 2024-04-02T17:38:56.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:56 smithi154 ceph-mon[36479]: pgmap v1165: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 21 KiB/s wr, 41 op/s 2024-04-02T17:38:56.432 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T17:38:58.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:38:58 smithi059 ceph-mon[31664]: pgmap v1166: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:38:58.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:38:58 smithi164 ceph-mon[36499]: pgmap v1166: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:38:58.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:38:58 smithi154 ceph-mon[36479]: pgmap v1166: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T17:38:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 0, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 0, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 0, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.003988686, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000713332, 2024-04-02T17:38:58.460 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:38:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 467, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 462, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 553616, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 285504, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.137492108, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.059431738, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.050886219, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.019374305, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 462, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.051124421, 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000110658 2024-04-02T17:38:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 553616, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 285504 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 352, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 320, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 217728, 2024-04-02T17:38:58.463 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.065889962, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.055714054, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.003492318, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 350, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.036551776, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000104433 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:58.464 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 217728 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:38:58.465 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-04-02T17:38:58.466 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:38:58.467 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 150766, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 31, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2024-04-02T17:38:58.468 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-04-02T17:38:58.469 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 465, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 485100, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 465, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 465, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 485100, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-02T17:38:58.470 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-02T17:38:59.283 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-02T17:38:59.283 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-02T17:38:59.283 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-02T17:38:59.283 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-02T17:38:59.283 DEBUG:tasks.check_counter:Getting stats from b 2024-04-02T17:38:59.284 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-04-02T17:38:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:39:00.092 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:00 smithi154 ceph-mon[36479]: pgmap v1167: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-02T17:39:00.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:00 smithi059 ceph-mon[31664]: pgmap v1167: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-02T17:39:00.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:00 smithi164 ceph-mon[36499]: pgmap v1167: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 321280, 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 351194, 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 76456282, 2024-04-02T17:39:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 310614830, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 21.313440349, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 8.259965252, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 12.004207855, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 2.726647138, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 351193, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 39.023199568, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000111116 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 533936, 2024-04-02T17:39:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.098788337, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000185 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 511334, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 594858, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 123575959, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 305942218, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 29.073915632, 2024-04-02T17:39:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 11.568865898, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 18.307286268, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 2.066634120, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 594863, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 47.030714496, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000079061 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 999725, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.160605797, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000160 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 939360, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 294256 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 749458, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 954932, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 240592834, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 729492848, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 276, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 9, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 54.959477238, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 25.460676352, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 34.648054565, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.817744600, 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 954923, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 56.642191043, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000059315 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1454092, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.273639502, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000188 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 171101152, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 534661472 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-02T17:39:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 137568, 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 189.749351068, 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001379313 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 74889, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 9.436704692, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000126009 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 334394, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 264805, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 264805, 2024-04-02T17:39:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1345.766492613, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005082103 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 69589, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 4, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 1, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 3201, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 10, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 18, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 777, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 173, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 594, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 10, 2024-04-02T17:39:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 525, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 1829, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 76, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 76, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 505586, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 253884, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 69561, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 163, 2024-04-02T17:39:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 2934, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 138, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 20, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 1232, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 19, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 188, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 120, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-02T17:39:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 120, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 45, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 4, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 41, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 75, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 35, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 33924, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 137294, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 95950, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 33131, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 15130, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 156965 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 467, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 436, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 239, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 521, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 3570, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 82, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 39723, 2024-04-02T17:39:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 39641, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 4, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 20, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 28, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 209647, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 208357, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 208357, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 1290, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 2836, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 321, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 319, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 319, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 1, 2024-04-02T17:39:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 4, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 451413974, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 453519304, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 72837, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 697.661532552, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.009578394 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 116, 2024-04-02T17:39:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 40831, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 40715, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 40, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 521, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 481, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 788, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 95932, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 95144, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 44, 2024-04-02T17:39:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 44114, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 44070, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 86724, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 231800 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 448961, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 31478, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 334394, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 5455, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 62848, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 40340, 2024-04-02T17:39:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 184.069668924, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004562956 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 20619, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 191.947336152, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.009309245 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 92310, 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 286.295020386, 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.003101451 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 119, 2024-04-02T17:39:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.619972898, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005209856 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-02T17:39:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 10, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.015122602, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001512260 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 62370, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 513.481906863, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.008232834 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 8054, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 45.870095890, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005695318 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 44, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.412446386, 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.032101054 2024-04-02T17:39:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 18, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.617184407, 2024-04-02T17:39:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.034288022 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1224, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 2.144693456, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001752200 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.019145024, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002735003 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-02T17:39:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 39690, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 119.273899625, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.003005137 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-02T17:39:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 21560, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 21560, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1566, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 360, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 360, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-02T17:39:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 286144, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 201, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 528, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 6, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 167, 2024-04-02T17:39:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 959705, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 19494, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 141852, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 482552649, 2024-04-02T17:39:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 32, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 141852, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 141852, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 3315.006921103, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.023369476 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 141852, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 192063 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 141852, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1286, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 140566, 2024-04-02T17:39:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 3049, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 20526, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 1157, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 73991, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 1074, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 10, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 40454, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-02T17:39:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 129, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 41052, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-02T17:39:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-02T17:39:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 93, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 7, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 8392, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 10, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 2219 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 6, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 39641, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 39641, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1582068, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 276867721, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1582068, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1582068, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 276867721, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 141852, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 486556322, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 141852, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 486556322, 2024-04-02T17:39:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 141852, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 141852, 2024-04-02T17:39:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 141852, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 141852, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 39641, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 4003741, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 39641, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1154, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 4003741, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x5587af86c500": { 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 67231, 2024-04-02T17:39:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 209647, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 449325000, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 209647, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 72837, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 449257769, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:01.680 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-02T17:39:02.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:02 smithi059 ceph-mon[31664]: pgmap v1168: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-02T17:39:02.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:02 smithi164 ceph-mon[36499]: pgmap v1168: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-02T17:39:02.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:02 smithi154 ceph-mon[36479]: pgmap v1168: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-02T17:39:02.751 INFO:tasks.check_counter:Daemon mds.b mds.exported=20 2024-04-02T17:39:02.751 INFO:tasks.check_counter:Daemon mds.b mds.imported=19 2024-04-02T17:39:02.751 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=467 2024-04-02T17:39:02.751 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=436 2024-04-02T17:39:02.751 DEBUG:tasks.check_counter:Getting stats from e 2024-04-02T17:39:02.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-04-02T17:39:02.994 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:39:04.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:04 smithi059 ceph-mon[31664]: pgmap v1169: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-02T17:39:04.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:04 smithi164 ceph-mon[36499]: pgmap v1169: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-02T17:39:04.371 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:04 smithi154 ceph-mon[36479]: pgmap v1169: 97 pgs: 97 active+clean; 941 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 10 op/s 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 4155, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 4157, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 166906523, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 1321297, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.399379972, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.095764936, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.176958822, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.100862048, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 4155, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.137839241, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000033174 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:39:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1399, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1747, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 105213724, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 1143626, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 7, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.175674724, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.038406723, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.092343873, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.034033946, 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1746, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.057987598, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000033211 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 700272 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1181, 2024-04-02T17:39:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1128, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 72375484, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 495886, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.157881003, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.034244022, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.089515630, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.027655373, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1127, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.039901484, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000035405 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 905696, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 286960 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 10609, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.253480424, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000023892 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-02T17:39:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 37735, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 10682, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 27053, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 22475, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-04-02T17:39:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 1, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-04-02T17:39:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-02T17:39:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 177937, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 130, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 16, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 133506801, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 346067278, 2024-04-02T17:39:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 346065194, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 177937 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 69, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 37615, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 37546, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 24, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 84, 2024-04-02T17:39:04.681 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 60, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 37735, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 38056, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 321, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 85592, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215416 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-04-02T17:39:04.682 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-02T17:39:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.688 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.689 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:04.690 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.691 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.693 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-04-02T17:39:04.694 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 3111, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:04.695 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:04.696 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:04.697 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 243049, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 138, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 167, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:04.698 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 26762070, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 158242, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 6206, 2024-04-02T17:39:04.699 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 3, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 6206, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6206, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 16.753569001, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002699576 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6206, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 6207 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 6206, 2024-04-02T17:39:04.700 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 6206, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 3319, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 2885, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-02T17:39:04.701 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-02T17:39:04.702 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-02T17:39:04.703 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 93, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 6, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 11, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 1, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:04.704 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.705 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 6731, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 343860168, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 6731, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6731, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 343860168, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 6206, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 341874505, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6206, 2024-04-02T17:39:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 341874505, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:04.708 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 6206, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 6206, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6206, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 6206, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:04.709 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x5581780e06e0": { 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:04.711 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:04.712 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:04.712 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.712 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:04.712 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-02T17:39:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:05 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:39:05.338 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:05 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:39:05.365 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-02T17:39:05.366 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-02T17:39:05.366 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-02T17:39:05.366 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-02T17:39:05.366 DEBUG:tasks.check_counter:Getting stats from h 2024-04-02T17:39:05.366 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-04-02T17:39:05.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:05 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T17:39:05.606 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:39:06.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:06 smithi164 ceph-mon[36499]: pgmap v1170: 97 pgs: 97 active+clean; 942 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-02T17:39:06.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:06 smithi154 ceph-mon[36479]: pgmap v1170: 97 pgs: 97 active+clean; 942 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-02T17:39:06.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:06 smithi059 ceph-mon[31664]: pgmap v1170: 97 pgs: 97 active+clean; 942 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-02T17:39:07.787 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-02T17:39:07.787 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:07.787 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 216201, 2024-04-02T17:39:07.787 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 215721, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 55682376, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 218572152, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 19, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 14.098662130, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 5.386004475, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 7.726346598, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.990141923, 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:07.788 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 215719, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 10.741049146, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000049791 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 337054, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.064958789, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000192 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 736 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 375285, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 377140, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 88809333, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 235474762, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 20.743357696, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 8.179075183, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 13.633668891, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.357992766, 2024-04-02T17:39:07.790 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 377139, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 20.411242241, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000054121 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 678476, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.112619681, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000165 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 777712 2024-04-02T17:39:07.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 727791, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 878670, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 233819614, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 629684477, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 7, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 50.862927986, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 23.463857930, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 33.204144572, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.189371739, 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:07.792 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 878668, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 39.117518570, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000044519 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1391414, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.252481749, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000181 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 156275552, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 476205168 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.793 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 99023, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 159.504571558, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001610783 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.794 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 75512, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 9.100251309, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000120513 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 310361, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 243536, 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-02T17:39:07.795 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 243536, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 985.334783772, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004045951 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 66825, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 118, 2024-04-02T17:39:07.796 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 9, 2024-04-02T17:39:07.797 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 13, 2024-04-02T17:39:07.797 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 1022, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 631, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 281, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 110, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 113, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 770, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 69, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 69, 2024-04-02T17:39:07.798 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 497199, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 233851, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 66821, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 183, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 5431, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 4, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 12, 2024-04-02T17:39:07.799 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 199, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 12, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 431, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:07.800 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 24, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 19060, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 139701, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 78058, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 31691, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 15124, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 169256 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-02T17:39:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 1218, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 1704, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 908, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 1689, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 1474, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 76, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 39794, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 39718, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:07.802 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 13, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 22, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-02T17:39:07.803 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 193142, 2024-04-02T17:39:07.804 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 89937, 2024-04-02T17:39:07.805 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 86031, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 107111, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 3906, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 240, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 112, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 107, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 133, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 9, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 5, 2024-04-02T17:39:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 163246576, 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 386918916, 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 53312, 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 465.212869592, 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.008726231 2024-04-02T17:39:07.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 132, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 40161, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 40029, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 47, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 970, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 923, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 1016, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 78454, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 77438, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 43117, 2024-04-02T17:39:07.808 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 43117, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 82112, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 223608 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 429618, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 1991, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 310361, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 5250, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 14292, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 39582, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 185.423777355, 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004684547 2024-04-02T17:39:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7447, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 106.371745988, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.014283838 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.810 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 86481, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 123.164116642, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001424175 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.813 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:07.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.813 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.814 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.815 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 60120, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 388.796652514, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.006467010 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 8934, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 52.285318521, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005852397 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:07.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 20, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.320217685, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.016010884 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.817 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 22, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.397313380, 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.018059699 2024-04-02T17:39:07.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1156, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 2.111825582, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001826838 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 39774, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 126.463816105, 2024-04-02T17:39:07.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.003179559 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-04-02T17:39:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 20669, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 20669, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1473, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:07.822 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:07.823 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:07.824 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 416155, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 258, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 704, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 8, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 167, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:07.825 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 1209871, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 25703, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 98102, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 391676681, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 33, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 98102, 2024-04-02T17:39:07.826 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 98102, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1040.263306786, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.010603894 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 98102, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 100477 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 98102, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1317, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 96785, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-02T17:39:07.827 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 153, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 1317, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 54629, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 729, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 9, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 39934, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-04-02T17:39:07.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 306, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-02T17:39:07.829 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 93, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-02T17:39:07.830 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 6, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 2604, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 0, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 796 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.831 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 6, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 39718, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 39718, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1319273, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 240091015, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1319273, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1319273, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 240091015, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 98102, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 395688199, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 98102, 2024-04-02T17:39:07.834 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 395688199, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:07.835 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 98102, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 98102, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 98102, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 98102, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 39718, 2024-04-02T17:39:07.836 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 4011518, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 39718, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1317, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 4011518, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x559c4f896500": { 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 148292, 2024-04-02T17:39:07.837 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 193142, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 382724612, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 193142, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 53312, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 382576320, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:07.838 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.839 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:07.839 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-02T17:39:08.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:08 smithi164 ceph-mon[36499]: pgmap v1171: 97 pgs: 97 active+clean; 938 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 441 KiB/s wr, 20 op/s 2024-04-02T17:39:08.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:08 smithi059 ceph-mon[31664]: pgmap v1171: 97 pgs: 97 active+clean; 938 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 441 KiB/s wr, 20 op/s 2024-04-02T17:39:08.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:08 smithi154 ceph-mon[36479]: pgmap v1171: 97 pgs: 97 active+clean; 938 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 441 KiB/s wr, 20 op/s 2024-04-02T17:39:08.528 INFO:tasks.check_counter:Daemon mds.h mds.exported=12 2024-04-02T17:39:08.528 INFO:tasks.check_counter:Daemon mds.h mds.imported=12 2024-04-02T17:39:08.528 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1218 2024-04-02T17:39:08.529 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=1704 2024-04-02T17:39:08.529 DEBUG:tasks.check_counter:Getting stats from k 2024-04-02T17:39:08.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-04-02T17:39:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T17:39:10.149 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:10 smithi154 ceph-mon[36479]: pgmap v1172: 97 pgs: 97 active+clean; 933 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-02T17:39:10.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:10 smithi164 ceph-mon[36499]: pgmap v1172: 97 pgs: 97 active+clean; 933 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 3439, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 3441, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 148710712, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 1093603, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:10.423 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.358271893, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.087166937, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.158092050, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.088906165, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 3439, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.124926016, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000036326 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.424 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1907, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2261, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 130508190, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 1471970, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 9, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.241244243, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.045753197, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.132178202, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.048844279, 2024-04-02T17:39:10.425 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2259, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.072000488, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000031872 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 866112 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.426 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1481, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1426, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 106008260, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 591750, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.188929407, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.041550711, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.104465682, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.034934786, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1426, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.056009613, 2024-04-02T17:39:10.427 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000039277 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 909808, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 289968 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.428 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 10682, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.255938019, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000023959 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-04-02T17:39:10.430 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 39951, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 11692, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 28259, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 23247, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 1, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-04-02T17:39:10.431 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-02T17:39:10.432 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-04-02T17:39:10.433 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-04-02T17:39:10.434 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:10.435 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 193058, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-04-02T17:39:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 133, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 17, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 163246576, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 386770624, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 386768973, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.437 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 193058 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 107, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 39866, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 39759, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 21, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 150, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 129, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 39951, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 40324, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 373, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-04-02T17:39:10.438 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 89032, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215416 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-04-02T17:39:10.439 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.440 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.441 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.444 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.445 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:10.447 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.448 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.449 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.450 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-04-02T17:39:10.451 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 3115, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:10.452 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:10.453 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:10.454 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 245501, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 185, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:10.455 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 167, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 28425170, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 173004, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-02T17:39:10.456 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 6293, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 3, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 6293, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6293, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 15.940590164, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002533066 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6293, 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 6294 2024-04-02T17:39:10.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 6293, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 6293, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 3368, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 2923, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-04-02T17:39:10.458 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-02T17:39:10.459 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-02T17:39:10.460 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 93, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 6, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 11, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 11, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-02T17:39:10.461 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-04-02T17:39:10.462 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 6823, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 384583068, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 6823, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6823, 2024-04-02T17:39:10.463 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 384583068, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:10.464 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 6293, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 382581300, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6293, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 382581300, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-02T17:39:10.465 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 6293, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 6293, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 6293, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 6293, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.467 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x556fcb3446e0": { 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-02T17:39:10.468 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-02T17:39:10.469 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-02T17:39:10.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:10 smithi059 ceph-mon[31664]: pgmap v1172: 97 pgs: 97 active+clean; 933 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-02T17:39:11.287 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-02T17:39:11.287 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-02T17:39:11.287 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-02T17:39:11.287 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-02T17:39:11.287 DEBUG:tasks.check_counter:Getting stats from c 2024-04-02T17:39:11.287 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-04-02T17:39:11.664 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: pgmap v1173: 97 pgs: 97 active+clean; 928 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:39:12.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:12 smithi164 ceph-mon[36499]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: pgmap v1173: 97 pgs: 97 active+clean; 928 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:39:12.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:12 smithi154 ceph-mon[36479]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: pgmap v1173: 97 pgs: 97 active+clean; 928 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 48 op/s 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T17:39:12.450 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:12 smithi059 ceph-mon[31664]: from='mgr.14152 ' entity='mgr.x' 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 468717, 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 414774, 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 118085064, 2024-04-02T17:39:13.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 222601025, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 18, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 25.433511336, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 9.718916083, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 17.512125502, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.722710863, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 414773, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 19.453190628, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000046900 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 808553, 2024-04-02T17:39:13.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.128606279, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000159 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 506329, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 661928, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 165440208, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 564026741, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 14, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-02T17:39:13.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 35.489539248, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 15.207815209, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 23.742431817, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.831767103, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 661927, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 40.340158880, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000060943 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 991882, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.186968924, 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000188 2024-04-02T17:39:13.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 119850592, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 426847664 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 201982, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 198109, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 48311403, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 196430421, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 10, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 17.245729304, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 8.119609317, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 8.072469981, 2024-04-02T17:39:13.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 2.154214710, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 198107, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 8.978251611, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000045320 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 331782, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.061631857, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000185 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 779680 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.980 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 88710, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 155.669450767, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001754812 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:13.981 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 67962, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 9.859872327, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000145079 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 226155, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 225754, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 225754, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1005.279004043, 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004452984 2024-04-02T17:39:13.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 401, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 97, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 7, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 7, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 867, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 545, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 237, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 85, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 88, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 279, 2024-04-02T17:39:13.983 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 1, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 1, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 6, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 391821, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 222657, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 400, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 116, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 4949, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 5, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 27, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 5, 2024-04-02T17:39:13.984 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 280, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:13.985 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 24, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 4548, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 111238, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 61464, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 31620, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 12536, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 159694 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 330, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 2301, 2024-04-02T17:39:13.986 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 1203, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 2203, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 418, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 9, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 36076, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 36067, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-02T17:39:13.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 5, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 14, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 172979, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 70392, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 61451, 2024-04-02T17:39:13.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 111528, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 8941, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 214, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 86, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 76, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 138, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 10, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 10, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 128241948, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 356258664, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-02T17:39:13.989 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 49066, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 487.308909827, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.009931702 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 100, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 36239, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 36139, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 37, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1208, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 1171, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 867, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 67949, 2024-04-02T17:39:13.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 67082, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 1, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 39290, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 39289, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 76224, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223608 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 327980, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 57, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 226155, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 5492, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 4399, 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-02T17:39:13.991 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 35895, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 206.212852139, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005744890 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 12880, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 151.371041582, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.011752410 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 75149, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 125.551211385, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001670697 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-02T17:39:13.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000137719, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000137719 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:13.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.995 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.995 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-02T17:39:13.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.996 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 56922, 2024-04-02T17:39:13.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 345.609316479, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006071629 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7702, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 52.352789293, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006797298 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 20, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.285027127, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.014251356 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:13.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 33, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.721704433, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.021869831 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1096, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.840805931, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001679567 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-02T17:39:14.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 36056, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 121.334117955, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003365157 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:14.002 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-02T17:39:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 22033, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 22033, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1569, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:14.004 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:14.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 820578, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 332, 2024-04-02T17:39:14.006 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 704, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 8, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 964420, 2024-04-02T17:39:14.007 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 19532, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 89206, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 360126198, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 89206, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 89206, 2024-04-02T17:39:14.008 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1041.392942314, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.011674023 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 89206, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 91083 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 89206, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1110, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 88096, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1, 2024-04-02T17:39:14.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 158, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 1110, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 50176, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 599, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 7, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 36247, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 316, 2024-04-02T17:39:14.010 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-02T17:39:14.011 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 6, 2024-04-02T17:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 1775, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 694 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 6, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 36067, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 36067, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:14.014 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1177024, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 211037571, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1177024, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1177024, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 211037571, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 89206, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 363768965, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 89206, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 363768965, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 89206, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 89206, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 89206, 2024-04-02T17:39:14.017 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 89206, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 36067, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 3642767, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:14.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 36067, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1110, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 3642767, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55d24dd84500": { 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:14.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 172979, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 352064360, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 172979, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 49066, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 352064360, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:14.020 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-02T17:39:14.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:14 smithi059 ceph-mon[31664]: pgmap v1174: 97 pgs: 97 active+clean; 918 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-02T17:39:14.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:14 smithi164 ceph-mon[36499]: pgmap v1174: 97 pgs: 97 active+clean; 918 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-02T17:39:14.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:14 smithi154 ceph-mon[36479]: pgmap v1174: 97 pgs: 97 active+clean; 918 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-02T17:39:15.604 INFO:tasks.check_counter:Daemon mds.c mds.exported=5 2024-04-02T17:39:15.604 INFO:tasks.check_counter:Daemon mds.c mds.imported=5 2024-04-02T17:39:15.604 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=330 2024-04-02T17:39:15.604 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=2301 2024-04-02T17:39:15.604 DEBUG:tasks.check_counter:Getting stats from f 2024-04-02T17:39:15.605 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-04-02T17:39:15.861 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:39:16.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:16 smithi164 ceph-mon[36499]: pgmap v1175: 97 pgs: 97 active+clean; 897 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-04-02T17:39:16.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:16 smithi154 ceph-mon[36479]: pgmap v1175: 97 pgs: 97 active+clean; 897 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-04-02T17:39:16.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:16 smithi059 ceph-mon[31664]: pgmap v1175: 97 pgs: 97 active+clean; 897 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-04-02T17:39:18.087 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-02T17:39:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 802, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 804, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 58956124, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 255055, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.093850981, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.017089439, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.047878776, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.022525328, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 802, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.035046783, 2024-04-02T17:39:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000043699 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 320, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 398, 2024-04-02T17:39:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 50384125, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 327971, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.063663341, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.008401308, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.043164909, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.008383363, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 397, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.022793950, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000057415 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 227744 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 521, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 500, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 47233489, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 188747, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-04-02T17:39:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.069324466, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.011638761, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.043562513, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.012729223, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 499, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.027875373, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000055862 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 323424, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 65680 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2566, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.195875860, 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000076335 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-02T17:39:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-02T17:39:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 15785, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 10583, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 5202, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 697, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 2, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-02T17:39:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-02T17:39:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-02T17:39:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 73108, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 26, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 451413974, 2024-04-02T17:39:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 453659096, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 453658497, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 73108 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 24, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 15633, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 15609, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 23, 2024-04-02T17:39:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 65, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 42, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 15785, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 16063, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 278, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 64364, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215392 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-02T17:39:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3102, 2024-04-02T17:39:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-02T17:39:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 241401, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 126, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 11298906, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 69666, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-02T17:39:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 1504, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 26, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 4, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 1504, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1504, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 10.131835710, 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006736592 2024-04-02T17:39:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1504, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1505 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 1504, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1504, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 754, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 747, 2024-04-02T17:39:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 2, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-02T17:39:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 0, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 11, 2024-04-02T17:39:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 11, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-02T17:39:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1639, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 156416410, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1639, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1639, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 156416410, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1504, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 155832756, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1504, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 155832756, 2024-04-02T17:39:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-02T17:39:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1504, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1504, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1504, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1504, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x5576d65986e0": { 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:18.137 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-02T17:39:18.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:18 smithi164 ceph-mon[36499]: pgmap v1176: 97 pgs: 97 active+clean; 893 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-02T17:39:18.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:18 smithi154 ceph-mon[36479]: pgmap v1176: 97 pgs: 97 active+clean; 893 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-02T17:39:18.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:18 smithi059 ceph-mon[31664]: pgmap v1176: 97 pgs: 97 active+clean; 893 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-02T17:39:18.864 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-02T17:39:18.864 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-02T17:39:18.864 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-02T17:39:18.864 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-02T17:39:18.864 DEBUG:tasks.check_counter:Getting stats from i 2024-04-02T17:39:18.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-04-02T17:39:19.110 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:39:20.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:20 smithi164 ceph-mon[36499]: pgmap v1177: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.4 MiB/s wr, 74 op/s 2024-04-02T17:39:20.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:20 smithi154 ceph-mon[36479]: pgmap v1177: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.4 MiB/s wr, 74 op/s 2024-04-02T17:39:20.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:20 smithi059 ceph-mon[31664]: pgmap v1177: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.4 MiB/s wr, 74 op/s 2024-04-02T17:39:20.705 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 475102, 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 443901, 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 116457529, 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 184513753, 2024-04-02T17:39:20.706 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 9, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 25.696339356, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 9.802536901, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 18.048823351, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.550982861, 2024-04-02T17:39:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 443903, 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 20.283962568, 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000045694 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:20.708 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 869072, 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.135751524, 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000156 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 933328, 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 295040 2024-04-02T17:39:20.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 210651, 2024-04-02T17:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 215226, 2024-04-02T17:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 50030080, 2024-04-02T17:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 256648276, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 17, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 17.488844501, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 8.096308896, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 8.055627356, 2024-04-02T17:39:20.714 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 2.284913474, 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 215224, 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 9.440987730, 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000043865 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.715 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:20.716 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 331179, 2024-04-02T17:39:20.716 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.060279782, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000182 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 793984 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 617785, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 800201, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 207233383, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 631984731, 2024-04-02T17:39:20.717 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 18, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 41.428597705, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 17.941772868, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 28.202854568, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 1.842512621, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 800197, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 47.105572930, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000058867 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1174255, 2024-04-02T17:39:20.718 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.222817936, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000189 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 154101264, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 480469280 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 95006, 2024-04-02T17:39:20.719 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 154.485390488, 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001626059 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:20.720 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 73951, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 9.802556953, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000132554 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 303390, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 237477, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 237477, 2024-04-02T17:39:20.721 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 983.055952524, 2024-04-02T17:39:20.722 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004139583 2024-04-02T17:39:20.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.722 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 65913, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 61, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 3, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 4, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 277, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 182, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 81, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 14, 2024-04-02T17:39:20.723 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 17, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 548, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 1, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 1, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 6, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 476510, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 222843, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 65909, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 158, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:20.724 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 4233, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 10, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 137, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 10, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 381, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-02T17:39:20.725 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 12, 2024-04-02T17:39:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 27291, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 142968, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 84439, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 29981, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 13559, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 160016 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 768, 2024-04-02T17:39:20.727 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 1986, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 1055, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 1914, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 1081, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 4, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 38750, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 38746, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:20.728 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 12, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 7, 2024-04-02T17:39:20.729 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 186517, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 83633, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 73741, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 112776, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 9892, 2024-04-02T17:39:20.730 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 229, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 101, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 89, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 140, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 10, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 12, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 143884539, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 377372371, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 51565, 2024-04-02T17:39:20.731 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 486.954941240, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.009443516 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 36, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 39028, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 38992, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 35, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1036, 2024-04-02T17:39:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 1001, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 277, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 77042, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 76765, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 1, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 42549, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 42548, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 79972, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223608 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-02T17:39:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 413752, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 2071, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 303390, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 5593, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 13144, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 38564, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 182.979176508, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004744818 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.734 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4012, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 68.413148330, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.017052130 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 86216, 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 118.420822502, 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001373536 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.736 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000089567, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000089567 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.737 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.738 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-02T17:39:20.740 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.740 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.740 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.741 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.742 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 59900, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 447.947253487, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.007478251 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8361, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 45.577791796, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005451236 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:20.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.744 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 20, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.985513508, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.049275675 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:20.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 21, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.407452519, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.019402500 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.746 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1652, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.931278141, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001169054 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 38730, 2024-04-02T17:39:20.748 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 116.393426166, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003005252 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-02T17:39:20.749 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 2, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 21992, 2024-04-02T17:39:20.750 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 21992, 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1575, 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:20.751 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:20.752 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:20.753 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:20.754 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 292507, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 124, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 440, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 5, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:20.755 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 337419, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 3818, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-02T17:39:20.756 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 95476, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 381458831, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 95476, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 95476, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1030.560100992, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.010793917 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 95476, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 97813 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.757 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 95476, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1410, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 94066, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 142, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 1410, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 52975, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 797, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 3, 2024-04-02T17:39:20.758 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 38904, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 284, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-02T17:39:20.759 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-02T17:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 6, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 2418, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 879 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.761 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 6, 2024-04-02T17:39:20.762 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 6, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 38746, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 38746, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1303534, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 237177304, 2024-04-02T17:39:20.763 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1303534, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1303534, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 237177304, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:20.764 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 95476, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 385372177, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 95476, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 385372177, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 95476, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 95476, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 95476, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 95476, 2024-04-02T17:39:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 38746, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 3913346, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 38746, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:20.767 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1410, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 3913346, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55987dacc500": { 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 186517, 2024-04-02T17:39:20.768 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 373178067, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 186517, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 51565, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 373178067, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:20.769 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-02T17:39:21.470 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2024-04-02T17:39:21.470 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-04-02T17:39:21.471 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=768 2024-04-02T17:39:21.471 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1986 2024-04-02T17:39:21.471 DEBUG:tasks.check_counter:Getting stats from l 2024-04-02T17:39:21.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-04-02T17:39:21.713 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T17:39:22.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:22 smithi154 ceph-mon[36479]: pgmap v1178: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 676 KiB/s wr, 58 op/s 2024-04-02T17:39:22.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:22 smithi059 ceph-mon[31664]: pgmap v1178: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 676 KiB/s wr, 58 op/s 2024-04-02T17:39:22.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:22 smithi164 ceph-mon[36499]: pgmap v1178: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 676 KiB/s wr, 58 op/s 2024-04-02T17:39:23.495 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2693, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2638, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 72900544, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 978454, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.301709154, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.079294424, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.123635104, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.078913838, 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2638, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.117731439, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000044629 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 910896, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 292528 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2599, 2024-04-02T17:39:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2956, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 113907294, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1609871, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.246021835, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.065939487, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.104164264, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.059845391, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2953, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.112746391, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000038180 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 783968 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1130, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1132, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 167805884, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 359352, 2024-04-02T17:39:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 9, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.184002201, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.021288127, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.126274474, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.029255719, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1130, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.043919100, 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000038866 2024-04-02T17:39:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-02T17:39:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 10006, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.260639420, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000026048 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-02T17:39:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 36174, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 8757, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 27417, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 23663, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-02T17:39:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T17:39:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-02T17:39:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-02T17:39:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-02T17:39:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 172979, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 138, 2024-04-02T17:39:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 16, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 128241948, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 356258664, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 356258065, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 172979 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 22, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 36116, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 36094, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 20, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 112, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 92, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 36174, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 36424, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 250, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 85780, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-02T17:39:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-02T17:39:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-02T17:39:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-02T17:39:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-02T17:39:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-02T17:39:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-02T17:39:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-02T17:39:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-02T17:39:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-02T17:39:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-02T17:39:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-02T17:39:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-02T17:39:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-02T17:39:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-02T17:39:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-02T17:39:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-02T17:39:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-02T17:39:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-02T17:39:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-02T17:39:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3106, 2024-04-02T17:39:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-02T17:39:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-02T17:39:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-02T17:39:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-02T17:39:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T17:39:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-02T17:39:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T17:39:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-02T17:39:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T17:39:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T17:39:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T17:39:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T17:39:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T17:39:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-02T17:39:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T17:39:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T17:39:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-02T17:39:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-02T17:39:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-02T17:39:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-02T17:39:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T17:39:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T17:39:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T17:39:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T17:39:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T17:39:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-02T17:39:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-02T17:39:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-02T17:39:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T17:39:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T17:39:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-02T17:39:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-02T17:39:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-02T17:39:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-02T17:39:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T17:39:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-02T17:39:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T17:39:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-02T17:39:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 239700, 2024-04-02T17:39:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 85, 2024-04-02T17:39:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-02T17:39:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-02T17:39:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-02T17:39:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-02T17:39:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-02T17:39:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-02T17:39:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-02T17:39:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-02T17:39:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7920, 2024-04-02T17:39:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 167, 2024-04-02T17:39:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T17:39:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-02T17:39:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-02T17:39:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-02T17:39:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 25543861, 2024-04-02T17:39:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 145928, 2024-04-02T17:39:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-02T17:39:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-02T17:39:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-02T17:39:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-02T17:39:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-02T17:39:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-02T17:39:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-02T17:39:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 5884, 2024-04-02T17:39:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-04-02T17:39:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-04-02T17:39:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 5884, 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5884, 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 16.333731881, 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002775957 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-02T17:39:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5884, 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 5885 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 5884, 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 5884, 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-02T17:39:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 3313, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 2569, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-02T17:39:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-02T17:39:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-02T17:39:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 93, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 6, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 11, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 14, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 3, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-02T17:39:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-02T17:39:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 6418, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 354005870, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 6418, 2024-04-02T17:39:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 6418, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 354005870, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 5884, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 352067320, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5884, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 352067320, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 5884, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 5884, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5884, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5884, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-02T17:39:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x559d5894a6e0": { 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-02T17:39:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-02T17:39:23.567 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-02T17:39:24.252 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-02T17:39:24.252 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-02T17:39:24.252 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-02T17:39:24.253 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-02T17:39:24.253 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-02T17:39:24.264 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T17:39:24.264 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:39:24.264 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T17:39:24.334 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:39:24.334 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T17:39:24.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:39:24.398 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:24 smithi154 ceph-mon[36479]: pgmap v1179: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 439 KiB/s wr, 45 op/s 2024-04-02T17:39:24.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:24 smithi059 ceph-mon[31664]: pgmap v1179: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 439 KiB/s wr, 45 op/s 2024-04-02T17:39:24.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:24 smithi164 ceph-mon[36499]: pgmap v1179: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 439 KiB/s wr, 45 op/s 2024-04-02T17:39:24.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 -- 172.21.15.59:0/3089547078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 msgr2=0x7f70ac100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 --2- 172.21.15.59:0/3089547078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac100e50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f709c009900 tx=0x7f709c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 -- 172.21.15.59:0/3089547078 shutdown_connections 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 --2- 172.21.15.59:0/3089547078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac101c20 0x7f70ac1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 --2- 172.21.15.59:0/3089547078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac1025e0 0x7f70ac108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 --2- 172.21.15.59:0/3089547078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac100e50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 -- 172.21.15.59:0/3089547078 >> 172.21.15.59:0/3089547078 conn(0x7f70ac0fbfa0 msgr2=0x7f70ac0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 -- 172.21.15.59:0/3089547078 shutdown_connections 2024-04-02T17:39:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.632+0000 7f70b1000640 1 -- 172.21.15.59:0/3089547078 wait complete. 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.633+0000 7f70b1000640 1 Processor -- start 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 -- start start 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 0x7f70ac1099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac1025e0 0x7f70ac109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac10d830 con 0x7f70ac1025e0 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac10d6b0 con 0x7f70ac101c20 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70b1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac10d9b0 con 0x7f70ac100a30 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70a9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 0x7f70ac1099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70a9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51504/0 (socket says 172.21.15.59:51504) 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70a9e00640 1 -- 172.21.15.59:0/90403678 learned_addr learned my addr 172.21.15.59:0/90403678 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.634+0000 7f70aa800640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac1025e0 0x7f70ac109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 -- 172.21.15.59:0/90403678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 msgr2=0x7f70ac1099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 0x7f70ac1099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 -- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac1025e0 msgr2=0x7f70ac109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac1025e0 0x7f70ac109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 -- 172.21.15.59:0/90403678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac10a7c0 con 0x7f70ac100a30 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a3e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 0x7f70ac1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a9e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c043020 con 0x7f70ac100a30 2024-04-02T17:39:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.635+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c0054e0 con 0x7f70ac100a30 2024-04-02T17:39:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.636+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac1a1f40 con 0x7f70ac100a30 2024-04-02T17:39:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.636+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c003c40 con 0x7f70ac100a30 2024-04-02T17:39:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.636+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a23a0 con 0x7f70ac100a30 2024-04-02T17:39:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.637+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac107d50 con 0x7f70ac100a30 2024-04-02T17:39:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.639+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f709c046020 con 0x7f70ac100a30 2024-04-02T17:39:24.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.640+0000 7f70a2a00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 0x7f70900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:24.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.640+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f709c0f2a50 con 0x7f70ac100a30 2024-04-02T17:39:24.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.641+0000 7f70a3e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 0x7f70900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:24.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.641+0000 7f70a3e00640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 0x7f70900a60f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70ac10b120 tx=0x7f7094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:24.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.643+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f709c04d020 con 0x7f70ac100a30 2024-04-02T17:39:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.854+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70ac02d540 con 0x7f70ac100a30 2024-04-02T17:39:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.856+0000 7f70a2a00640 1 -- 172.21.15.59:0/90403678 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22857 (secure 0 0 0) 0x7f709c0bf370 con 0x7f70ac100a30 2024-04-02T17:39:24.855 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:39:24.855 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:32:50.136490+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 msgr2=0x7f70900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 0x7f70900a60f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70ac10b120 tx=0x7f7094005f90 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 msgr2=0x7f70ac10b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c02c650 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 shutdown_connections 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f70900a3c30 0x7f70900a60f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac1025e0 0x7f70ac109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ac101c20 0x7f70ac1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 --2- 172.21.15.59:0/90403678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70ac100a30 0x7f70ac10b3b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:24.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 >> 172.21.15.59:0/90403678 conn(0x7f70ac0fbfa0 msgr2=0x7f70ac104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 shutdown_connections 2024-04-02T17:39:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:24.859+0000 7f70b1000640 1 -- 172.21.15.59:0/90403678 wait complete. 2024-04-02T17:39:24.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T17:05:29.509926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T17:10:32.636166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T17:10:33.468395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T17:39:24.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 -- 172.21.15.59:0/1446457590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0103d10 msgr2=0x7f62e0104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 --2- 172.21.15.59:0/1446457590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0103d10 0x7f62e0104130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02bc40 comp rx=0 tx=0).stop 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 -- 172.21.15.59:0/1446457590 shutdown_connections 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 --2- 172.21.15.59:0/1446457590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0104f00 0x7f62e0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 --2- 172.21.15.59:0/1446457590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e01058c0 0x7f62e010c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 --2- 172.21.15.59:0/1446457590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0103d10 0x7f62e0104130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 -- 172.21.15.59:0/1446457590 >> 172.21.15.59:0/1446457590 conn(0x7f62e00ff2c0 msgr2=0x7f62e01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:25.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 -- 172.21.15.59:0/1446457590 shutdown_connections 2024-04-02T17:39:25.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.116+0000 7f62e7400640 1 -- 172.21.15.59:0/1446457590 wait complete. 2024-04-02T17:39:25.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.117+0000 7f62e7400640 1 Processor -- start 2024-04-02T17:39:25.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.117+0000 7f62e7400640 1 -- start start 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103d10 0x7f62e019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 0x7f62e019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 0x7f62e01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010f9e0 con 0x7f62e01058c0 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010f860 con 0x7f62e0104f00 2024-04-02T17:39:25.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e7400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62e010fb60 con 0x7f62e0103d10 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 0x7f62e01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 0x7f62e019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 0x7f62e01a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33662/0 (socket says 172.21.15.59:33662) 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e5400640 1 -- 172.21.15.59:0/94841509 learned_addr learned my addr 172.21.15.59:0/94841509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.118+0000 7f62e4a00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103d10 0x7f62e019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 -- 172.21.15.59:0/94841509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103d10 msgr2=0x7f62e019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103d10 0x7f62e019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 -- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 msgr2=0x7f62e01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 0x7f62e01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 -- 172.21.15.59:0/94841509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e01a1c40 con 0x7f62e0104f00 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dfe00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 0x7f62e019a560 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.119+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ccc0 con 0x7f62e0104f00 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.120+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d4010910 con 0x7f62e0104f00 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.120+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e01a1f30 con 0x7f62e0104f00 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.120+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4018960 con 0x7f62e0104f00 2024-04-02T17:39:25.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.120+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a23f0 con 0x7f62e0104f00 2024-04-02T17:39:25.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.121+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e010b030 con 0x7f62e0104f00 2024-04-02T17:39:25.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.123+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62d4010ac0 con 0x7f62e0104f00 2024-04-02T17:39:25.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.124+0000 7f62dd600640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.124+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f62d40d07f0 con 0x7f62e0104f00 2024-04-02T17:39:25.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.125+0000 7f62e4a00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.125+0000 7f62e4a00640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 0x7f62c80a6070 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c180 tx=0x7f62cc02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:25.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.126+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f62d409cac0 con 0x7f62e0104f00 2024-04-02T17:39:25.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:25 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/90403678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:25.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.335+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62e002d5c0 con 0x7f62e0104f00 2024-04-02T17:39:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.336+0000 7f62dd600640 1 -- 172.21.15.59:0/94841509 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f62d40a0dc0 con 0x7f62e0104f00 2024-04-02T17:39:25.335 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-02T17:39:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.338+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 msgr2=0x7f62c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.338+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 0x7f62c80a6070 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c180 tx=0x7f62cc02cf90 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.338+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 msgr2=0x7f62e019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.338+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 0x7f62e019a560 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 shutdown_connections 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e01058c0 0x7f62e01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e0104f00 0x7f62e019a560 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 --2- 172.21.15.59:0/94841509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103d10 0x7f62e019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 >> 172.21.15.59:0/94841509 conn(0x7f62e00ff2c0 msgr2=0x7f62e0108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 shutdown_connections 2024-04-02T17:39:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.339+0000 7f62e7400640 1 -- 172.21.15.59:0/94841509 wait complete. 2024-04-02T17:39:25.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T17:39:25.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:25 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/90403678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:25.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:25 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/90403678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 -- 172.21.15.59:0/4071004503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4100250 msgr2=0x7f7fc4102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/4071004503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4100250 0x7f7fc4102680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009900 tx=0x7f7fb802bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 -- 172.21.15.59:0/4071004503 shutdown_connections 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/4071004503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4102bc0 0x7f7fc4105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/4071004503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 0x7f7fc41059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/4071004503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4100250 0x7f7fc4102680 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 -- 172.21.15.59:0/4071004503 >> 172.21.15.59:0/4071004503 conn(0x7f7fc40fbeb0 msgr2=0x7f7fc40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 -- 172.21.15.59:0/4071004503 shutdown_connections 2024-04-02T17:39:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.594+0000 7f7fc9e00640 1 -- 172.21.15.59:0/4071004503 wait complete. 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 Processor -- start 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 -- start start 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 0x7f7fc419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 0x7f7fc41a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113c80 con 0x7f7fc4100250 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113b00 con 0x7f7fc4105570 2024-04-02T17:39:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.596+0000 7f7fc9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113e00 con 0x7f7fc4102bc0 2024-04-02T17:39:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 0x7f7fc419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33680/0 (socket says 172.21.15.59:33680) 2024-04-02T17:39:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc8a00640 1 -- 172.21.15.59:0/2056235470 learned_addr learned my addr 172.21.15.59:0/2056235470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc9400640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 0x7f7fc41a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2056235470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 msgr2=0x7f7fc41a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 0x7f7fc41a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 msgr2=0x7f7fc419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2056235470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc41a5eb0 con 0x7f7fc4102bc0 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.597+0000 7f7fc8a00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:39:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 0x7f7fc419e7d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e9f0 tx=0x7f7fb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000cd80 con 0x7f7fc4102bc0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc41a61a0 con 0x7f7fc4102bc0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb00035f0 con 0x7f7fc4102bc0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0010670 con 0x7f7fc4102bc0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.598+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a6740 con 0x7f7fc4102bc0 2024-04-02T17:39:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.599+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc40fd9a0 con 0x7f7fc4102bc0 2024-04-02T17:39:25.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.602+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fb00037a0 con 0x7f7fc4102bc0 2024-04-02T17:39:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.603+0000 7f7fc1600640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 0x7f7fac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.604+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f7fb00d0910 con 0x7f7fc4102bc0 2024-04-02T17:39:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.604+0000 7f7fc8a00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 0x7f7fac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.605+0000 7f7fc8a00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 0x7f7fac0a6070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.605+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7fb009d230 con 0x7f7fc4102bc0 2024-04-02T17:39:25.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.816+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7fc402d370 con 0x7f7fc4102bc0 2024-04-02T17:39:25.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.817+0000 7f7fc1600640 1 -- 172.21.15.59:0/2056235470 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7fb009cb30 con 0x7f7fc4102bc0 2024-04-02T17:39:25.815 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-02T17:39:25.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.819+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 msgr2=0x7f7fac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.819+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 0x7f7fac0a6070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb802c880 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.819+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 msgr2=0x7f7fc419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.819+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 0x7f7fc419e7d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e9f0 tx=0x7f7fb000eeb0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 shutdown_connections 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4105570 0x7f7fc41a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4102bc0 0x7f7fc419e7d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f7fac0a3bb0 0x7f7fac0a6070 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 --2- 172.21.15.59:0/2056235470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4100250 0x7f7fc419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 >> 172.21.15.59:0/2056235470 conn(0x7f7fc40fbeb0 msgr2=0x7f7fc4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 shutdown_connections 2024-04-02T17:39:25.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:25.820+0000 7f7fc9e00640 1 -- 172.21.15.59:0/2056235470 wait complete. 2024-04-02T17:39:25.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T17:39:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 -- 172.21.15.59:0/3296358896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc104e10 msgr2=0x7f2cfc105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 --2- 172.21.15.59:0/3296358896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc104e10 0x7f2cfc105290 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009900 tx=0x7f2ce802bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 -- 172.21.15.59:0/3296358896 shutdown_connections 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 --2- 172.21.15.59:0/3296358896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1057d0 0x7f2cfc10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 --2- 172.21.15.59:0/3296358896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc104e10 0x7f2cfc105290 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 --2- 172.21.15.59:0/3296358896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 -- 172.21.15.59:0/3296358896 >> 172.21.15.59:0/3296358896 conn(0x7f2cfc0ff1b0 msgr2=0x7f2cfc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 -- 172.21.15.59:0/3296358896 shutdown_connections 2024-04-02T17:39:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.072+0000 7f2d02400640 1 -- 172.21.15.59:0/3296358896 wait complete. 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.073+0000 7f2d02400640 1 Processor -- start 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.073+0000 7f2d02400640 1 -- start start 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 0x7f2cfc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc1057d0 0x7f2cfc1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10f8f0 con 0x7f2cfc103c20 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10f770 con 0x7f2cfc104e10 2024-04-02T17:39:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d02400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10fa70 con 0x7f2cfc1057d0 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d01000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d01000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33696/0 (socket says 172.21.15.59:33696) 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d01000640 1 -- 172.21.15.59:0/796470509 learned_addr learned my addr 172.21.15.59:0/796470509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2cfbe00640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 0x7f2cfc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.074+0000 7f2d01a00640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc1057d0 0x7f2cfc1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 -- 172.21.15.59:0/796470509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc1057d0 msgr2=0x7f2cfc1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc1057d0 0x7f2cfc1a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 -- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 msgr2=0x7f2cfc199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 -- 172.21.15.59:0/796470509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a1b20 con 0x7f2cfc104e10 2024-04-02T17:39:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cfbe00640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 0x7f2cfc19a440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8036040 tx=0x7f2ce802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8043020 con 0x7f2cfc104e10 2024-04-02T17:39:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a1db0 con 0x7f2cfc104e10 2024-04-02T17:39:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce80054e0 con 0x7f2cfc104e10 2024-04-02T17:39:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8003c30 con 0x7f2cfc104e10 2024-04-02T17:39:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.075+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a2280 con 0x7f2cfc104e10 2024-04-02T17:39:26.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.077+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc10aee0 con 0x7f2cfc104e10 2024-04-02T17:39:26.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.079+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ce8046020 con 0x7f2cfc104e10 2024-04-02T17:39:26.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.080+0000 7f2cf9600640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 0x7f2ce40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.080+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2ce80f2a30 con 0x7f2cfc104e10 2024-04-02T17:39:26.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.080+0000 7f2d01000640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 0x7f2ce40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.081+0000 7f2d01000640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 0x7f2ce40a6070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00058c0 tx=0x7f2cf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:26.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.083+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2ce804d020 con 0x7f2cfc104e10 2024-04-02T17:39:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.297+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2cfc02d370 con 0x7f2cfc104e10 2024-04-02T17:39:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.298+0000 7f2cf9600640 1 -- 172.21.15.59:0/796470509 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f2ce80bf350 con 0x7f2cfc104e10 2024-04-02T17:39:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-02T17:39:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 msgr2=0x7f2ce40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 0x7f2ce40a6070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00058c0 tx=0x7f2cf0005f90 comp rx=0 tx=0).stop 2024-04-02T17:39:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 msgr2=0x7f2cfc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 0x7f2cfc19a440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8036040 tx=0x7f2ce802c650 comp rx=0 tx=0).stop 2024-04-02T17:39:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 shutdown_connections 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cfc1057d0 0x7f2cfc1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc104e10 0x7f2cfc19a440 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f2ce40a3bb0 0x7f2ce40a6070 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 --2- 172.21.15.59:0/796470509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.301+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 >> 172.21.15.59:0/796470509 conn(0x7f2cfc0ff1b0 msgr2=0x7f2cfc1080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.302+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 shutdown_connections 2024-04-02T17:39:26.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.302+0000 7f2d02400640 1 -- 172.21.15.59:0/796470509 wait complete. 2024-04-02T17:39:26.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T17:39:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:26 smithi154 ceph-mon[36479]: pgmap v1180: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-04-02T17:39:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:26 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/94841509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:26 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2056235470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:26 smithi059 ceph-mon[31664]: pgmap v1180: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-04-02T17:39:26.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:26 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/94841509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:26 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2056235470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 -- 172.21.15.59:0/129127396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c1035b0 msgr2=0x7fab0c109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 --2- 172.21.15.59:0/129127396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c1035b0 0x7fab0c109e40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab04009900 tx=0x7fab0402bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:26.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 -- 172.21.15.59:0/129127396 shutdown_connections 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 --2- 172.21.15.59:0/129127396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c102bf0 0x7fab0c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 --2- 172.21.15.59:0/129127396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c1035b0 0x7fab0c109e40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 --2- 172.21.15.59:0/129127396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c101a00 0x7fab0c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 -- 172.21.15.59:0/129127396 >> 172.21.15.59:0/129127396 conn(0x7fab0c0fcf70 msgr2=0x7fab0c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 -- 172.21.15.59:0/129127396 shutdown_connections 2024-04-02T17:39:26.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.552+0000 7fab12e00640 1 -- 172.21.15.59:0/129127396 wait complete. 2024-04-02T17:39:26.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.553+0000 7fab12e00640 1 Processor -- start 2024-04-02T17:39:26.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 -- start start 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 0x7fab0c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c1035b0 0x7fab0c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10d680 con 0x7fab0c101a00 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10d500 con 0x7fab0c1035b0 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10d800 con 0x7fab0c102bf0 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab11000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab11a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 0x7fab0c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab11000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51560/0 (socket says 172.21.15.59:51560) 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab11000640 1 -- 172.21.15.59:0/3398190728 learned_addr learned my addr 172.21.15.59:0/3398190728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.554+0000 7fab12400640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c1035b0 0x7fab0c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 -- 172.21.15.59:0/3398190728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c1035b0 msgr2=0x7fab0c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c1035b0 0x7fab0c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 -- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 msgr2=0x7fab0c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 0x7fab0c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 -- 172.21.15.59:0/3398190728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c1a1b70 con 0x7fab0c102bf0 2024-04-02T17:39:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11a00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 0x7fab0c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:39:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab11000640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9f0 tx=0x7faaf800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf800cd80 con 0x7fab0c102bf0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c1a1e60 con 0x7fab0c102bf0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf8010910 con 0x7fab0c102bf0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.555+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80188f0 con 0x7fab0c102bf0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.556+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a2370 con 0x7fab0c102bf0 2024-04-02T17:39:26.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.557+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c1a25a0 con 0x7fab0c102bf0 2024-04-02T17:39:26.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.559+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faaf8010ac0 con 0x7fab0c102bf0 2024-04-02T17:39:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.560+0000 7fab0a000640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.560+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7faaf80d0ac0 con 0x7fab0c102bf0 2024-04-02T17:39:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.561+0000 7fab11a00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.561+0000 7fab11a00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab0c102a50 tx=0x7fab00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.562+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faaf809cd90 con 0x7fab0c102bf0 2024-04-02T17:39:26.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:26 smithi164 ceph-mon[36499]: pgmap v1180: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-04-02T17:39:26.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:26 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/94841509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:26 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2056235470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T17:39:26.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.782+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab0c1a2930 con 0x7fab0c102bf0 2024-04-02T17:39:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.783+0000 7fab0a000640 1 -- 172.21.15.59:0/3398190728 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+592 (secure 0 0 0) 0x7faaf80a1090 con 0x7fab0c102bf0 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2274685378 2024-04-03T17:02:49.350524+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2274685378 2024-04-03T17:02:49.350524+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2569029589 2024-04-03T17:02:15.389808+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2590435245 2024-04-03T17:02:49.350524+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2036014369 2024-04-03T17:02:49.350524+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2195394001 2024-04-03T17:02:15.389808+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2817128980 2024-04-03T17:02:49.350524+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/3477889503 2024-04-03T17:02:15.389808+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/3477889503 2024-04-03T17:02:15.389808+0000 2024-04-02T17:39:26.782 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3686798564 2024-04-03T17:02:15.389808+0000 2024-04-02T17:39:26.783 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-04-02T17:39:26.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 msgr2=0x7faaf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab0c102a50 tx=0x7fab00009500 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 msgr2=0x7fab0c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9f0 tx=0x7faaf800eeb0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 shutdown_connections 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab0c1035b0 0x7fab0c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab0c102bf0 0x7fab0c19a490 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7faaf40a3c30 0x7faaf40a60f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 --2- 172.21.15.59:0/3398190728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0c101a00 0x7fab0c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 >> 172.21.15.59:0/3398190728 conn(0x7fab0c0fcf70 msgr2=0x7fab0c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 shutdown_connections 2024-04-02T17:39:26.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:26.786+0000 7fab12e00640 1 -- 172.21.15.59:0/3398190728 wait complete. 2024-04-02T17:39:26.804 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T17:39:26.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:26.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-02T17:39:27.076 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T17:39:27.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.077 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T17:39:27.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.171 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-02T17:39:27.259 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T17:39:27.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T17:39:27.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:27 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/796470509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:39:27.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:27 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3398190728' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T17:39:27.384 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:27 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/796470509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:39:27.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:27 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3398190728' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T17:39:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:27 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/796470509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T17:39:27.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:27 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3398190728' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T17:39:27.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-02T17:39:27.600 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T17:39:27.601 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-02T17:39:27.601 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T17:39:27.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c66b7c2f3bbcba0baf1fb77c8dfcedf75c53c04175f36c83ca835ca0eca22e03/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/c66b7c2f3bbcba0baf1fb77c8dfcedf75c53c04175f36c83ca835ca0eca22e03/diff,workdir=/var/lib/containers/storage/overlay/c66b7c2f3bbcba0baf1fb77c8dfcedf75c53c04175f36c83ca835ca0eca22e03/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/653d9c767da1f59c6cb338b8dbe1dcd3aee028762e2f50fa9d4d9c738d818f18/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/653d9c767da1f59c6cb338b8dbe1dcd3aee028762e2f50fa9d4d9c738d818f18/diff,workdir=/var/lib/containers/storage/overlay/653d9c767da1f59c6cb338b8dbe1dcd3aee028762e2f50fa9d4d9c738d818f18/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/d6667d06782efacb1439fffa6028db3fbd51ad864e44646aae338c1bc2d59b05/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/d6667d06782efacb1439fffa6028db3fbd51ad864e44646aae338c1bc2d59b05/diff,workdir=/var/lib/containers/storage/overlay/d6667d06782efacb1439fffa6028db3fbd51ad864e44646aae338c1bc2d59b05/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6c57b98b92ba2f1984f57892a39a09b99a1b4f89588648f0211df968c29a3e51/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/6c57b98b92ba2f1984f57892a39a09b99a1b4f89588648f0211df968c29a3e51/diff,workdir=/var/lib/containers/storage/overlay/6c57b98b92ba2f1984f57892a39a09b99a1b4f89588648f0211df968c29a3e51/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7607d6daa04135f6fce58f12756d1dbfd184d6f4a9deed92774d0c92001e47ea/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/7607d6daa04135f6fce58f12756d1dbfd184d6f4a9deed92774d0c92001e47ea/diff,workdir=/var/lib/containers/storage/overlay/7607d6daa04135f6fce58f12756d1dbfd184d6f4a9deed92774d0c92001e47ea/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dab6cd894ca5068f54157106804e7325291379525f400e27b489f620b570c00b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/dab6cd894ca5068f54157106804e7325291379525f400e27b489f620b570c00b/diff,workdir=/var/lib/containers/storage/overlay/dab6cd894ca5068f54157106804e7325291379525f400e27b489f620b570c00b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9b97a57bf1f7530285f08e31505f00b1458fedec55ff30e1618944bb3948353e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/9b97a57bf1f7530285f08e31505f00b1458fedec55ff30e1618944bb3948353e/diff,workdir=/var/lib/containers/storage/overlay/9b97a57bf1f7530285f08e31505f00b1458fedec55ff30e1618944bb3948353e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2c6eb644b38030968417bd088c4b42914e7142b3689f13367c0f16e41a1b1927/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/2c6eb644b38030968417bd088c4b42914e7142b3689f13367c0f16e41a1b1927/diff,workdir=/var/lib/containers/storage/overlay/2c6eb644b38030968417bd088c4b42914e7142b3689f13367c0f16e41a1b1927/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a751d7335287ad5da533b819879a0f29c114743971c8f8ea22d82ada3377db04/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/a751d7335287ad5da533b819879a0f29c114743971c8f8ea22d82ada3377db04/diff,workdir=/var/lib/containers/storage/overlay/a751d7335287ad5da533b819879a0f29c114743971c8f8ea22d82ada3377db04/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a73df6a4b0cf499457ae3df380b42e27182e47e735528d529a04edb3efb5fbeb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/SUH7PUUNLH64QIXTUOYRQLLIDL:/var/lib/containers/storage/overlay/l/GW77LPYNJXIVFXGN6EBYL7AEF5,upperdir=/var/lib/containers/storage/overlay/a73df6a4b0cf499457ae3df380b42e27182e47e735528d529a04edb3efb5fbeb/diff,workdir=/var/lib/containers/storage/overlay/a73df6a4b0cf499457ae3df380b42e27182e47e735528d529a04edb3efb5fbeb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 2024-04-02T17:39:27.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.682 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-02T17:39:27.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T17:39:27.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-02T17:39:28.031 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2024-04-02T17:39:28.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T17:39:28.036 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T17:39:28.047 INFO:tasks.cephadm:Teardown begin 2024-04-02T17:39:28.047 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T17:39:28.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:39:28.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1113536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0104f30 msgr2=0x7fa4a01053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1113536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0104f30 0x7fa4a01053b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1113536735 shutdown_connections 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1113536735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a01058f0 0x7fa4a010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1113536735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0104f30 0x7fa4a01053b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1113536735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0103d40 0x7fa4a0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1113536735 >> 172.21.15.59:0/1113536735 conn(0x7fa4a00ff2b0 msgr2=0x7fa4a01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1113536735 shutdown_connections 2024-04-02T17:39:28.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.339+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1113536735 wait complete. 2024-04-02T17:39:28.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 Processor -- start 2024-04-02T17:39:28.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 -- start start 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 0x7fa4a019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a01058f0 0x7fa4a01a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010f9f0 con 0x7fa4a01058f0 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010f870 con 0x7fa4a0103d40 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.340+0000 7fa4a5e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010fb70 con 0x7fa4a0104f30 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 0x7fa4a019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a01058f0 0x7fa4a01a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51574/0 (socket says 172.21.15.59:51574) 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 -- 172.21.15.59:0/1915659358 learned_addr learned my addr 172.21.15.59:0/1915659358 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49ea00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 0x7fa4a019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50708/0 (socket says 172.21.15.59:50708) 2024-04-02T17:39:28.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 -- 172.21.15.59:0/1915659358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 msgr2=0x7fa4a019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 0x7fa4a019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 -- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a01058f0 msgr2=0x7fa4a01a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a01058f0 0x7fa4a01a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 -- 172.21.15.59:0/1915659358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a01a1c60 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.341+0000 7fa49e000640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005ab0 tx=0x7fa48c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.342+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c043020 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.342+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c0054e0 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.342+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a01a1ef0 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.342+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003ce0 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.342+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a23b0 con 0x7fa4a0104f30 2024-04-02T17:39:28.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.343+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a010b060 con 0x7fa4a0104f30 2024-04-02T17:39:28.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.345+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa48c046020 con 0x7fa4a0104f30 2024-04-02T17:39:28.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.347+0000 7fa4a4a00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 0x7fa4840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.347+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa48c0f2540 con 0x7fa4a0104f30 2024-04-02T17:39:28.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.347+0000 7fa49ea00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 0x7fa4840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.348+0000 7fa49ea00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 0x7fa4840a6070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa49000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:28.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.351+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa48c04d020 con 0x7fa4a0104f30 2024-04-02T17:39:28.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:28 smithi154 ceph-mon[36479]: pgmap v1181: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-02T17:39:28.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:28 smithi059 ceph-mon[31664]: pgmap v1181: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-02T17:39:28.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:28 smithi164 ceph-mon[36499]: pgmap v1181: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-02T17:39:28.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.590+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4a00627a0 con 0x7fa4a0104f30 2024-04-02T17:39:28.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.591+0000 7fa4a4a00640 1 -- 172.21.15.59:0/1915659358 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11959 (secure 0 0 0) 0x7fa48c0bee60 con 0x7fa4a0104f30 2024-04-02T17:39:28.590 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:39:28.590 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:38:50.102339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":68,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27440":{"gid":27440,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1035281632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1035281632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1035281632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24560,24547,24529,24530]},"id":1}]} 2024-04-02T17:39:28.590 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 77 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 msgr2=0x7fa4840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 0x7fa4840a6070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa49000b410 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 msgr2=0x7fa4a019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005ab0 tx=0x7fa48c02c6c0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 shutdown_connections 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7fa4840a3bb0 0x7fa4840a6070 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a01058f0 0x7fa4a01a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4a0104f30 0x7fa4a019a580 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 --2- 172.21.15.59:0/1915659358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a0103d40 0x7fa4a019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 >> 172.21.15.59:0/1915659358 conn(0x7fa4a00ff2b0 msgr2=0x7fa4a0108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 shutdown_connections 2024-04-02T17:39:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.594+0000 7fa4a5e00640 1 -- 172.21.15.59:0/1915659358 wait complete. 2024-04-02T17:39:28.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T17:39:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 -- 172.21.15.59:0/835178634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc1090e0 msgr2=0x7f31bc109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 --2- 172.21.15.59:0/835178634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc1090e0 0x7f31bc109560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009900 tx=0x7f31a802bc40 comp rx=0 tx=0).stop 2024-04-02T17:39:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 -- 172.21.15.59:0/835178634 shutdown_connections 2024-04-02T17:39:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 --2- 172.21.15.59:0/835178634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc1090e0 0x7f31bc109560 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 --2- 172.21.15.59:0/835178634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc109aa0 0x7f31bc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 --2- 172.21.15.59:0/835178634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc107f10 0x7f31bc108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 -- 172.21.15.59:0/835178634 >> 172.21.15.59:0/835178634 conn(0x7f31bc076f10 msgr2=0x7f31bc079330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 -- 172.21.15.59:0/835178634 shutdown_connections 2024-04-02T17:39:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.826+0000 7f31bbe00640 1 -- 172.21.15.59:0/835178634 wait complete. 2024-04-02T17:39:28.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.827+0000 7f31bbe00640 1 Processor -- start 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.827+0000 7f31bbe00640 1 -- start start 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc1090e0 0x7f31bc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc109aa0 0x7f31bc1a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc113c30 con 0x7f31bc109aa0 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc113ab0 con 0x7f31bc1090e0 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc113db0 con 0x7f31bc107f10 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc109aa0 0x7f31bc1a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31ba000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc1090e0 0x7f31bc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.59:51596/0 (socket says 172.21.15.59:51596) 2024-04-02T17:39:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 -- 172.21.15.59:0/596838355 learned_addr learned my addr 172.21.15.59:0/596838355 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 -- 172.21.15.59:0/596838355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc1090e0 msgr2=0x7f31bc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc1090e0 0x7f31bc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 -- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc109aa0 msgr2=0x7f31bc1a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc109aa0 0x7f31bc1a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.828+0000 7f31baa00640 1 -- 172.21.15.59:0/596838355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc1a5e60 con 0x7f31bc107f10 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31baa00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00ebc0 tx=0x7f31ac00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac003b70 con 0x7f31bc107f10 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc1a6150 con 0x7f31bc107f10 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac003d10 con 0x7f31bc107f10 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac019980 con 0x7f31bc107f10 2024-04-02T17:39:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.829+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1a6670 con 0x7f31bc107f10 2024-04-02T17:39:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.830+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc10f320 con 0x7f31bc107f10 2024-04-02T17:39:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.832+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f31ac019b20 con 0x7f31bc107f10 2024-04-02T17:39:28.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.833+0000 7f31c0a00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 0x7f31a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:28.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.833+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f31ac0d0cc0 con 0x7f31bc107f10 2024-04-02T17:39:28.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.834+0000 7f31ba000640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 0x7f31a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:28.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.834+0000 7f31ba000640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 0x7f31a00a6070 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f31a802c250 tx=0x7f31a802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:28.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:28.838+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f31ac0d7080 con 0x7f31bc107f10 2024-04-02T17:39:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.077+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31bc02d370 con 0x7f31bc107f10 2024-04-02T17:39:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.079+0000 7f31c0a00640 1 -- 172.21.15.59:0/596838355 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11959 (secure 0 0 0) 0x7f31ac09d5e0 con 0x7f31bc107f10 2024-04-02T17:39:29.078 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:39:29.078 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/346805939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":346805939},{"type":"v1","addr":"172.21.15.59:6835","nonce":346805939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/195975414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":195975414},{"type":"v1","addr":"172.21.15.59:6841","nonce":195975414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T17:10:34.529898+0000","modified":"2024-04-02T17:38:50.102339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24584,"mds_1":24560,"mds_2":24547,"mds_3":24530,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24454":{"gid":24454,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/3588884595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3588884595},{"type":"v1","addr":"172.21.15.164:6833","nonce":3588884595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6833/1241853080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1241853080},{"type":"v1","addr":"172.21.15.154:6833","nonce":1241853080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6837/448006801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":448006801},{"type":"v1","addr":"172.21.15.59:6837","nonce":448006801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/1503733829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1503733829},{"type":"v1","addr":"172.21.15.154:6835","nonce":1503733829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.164:6837/316018252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":316018252},{"type":"v1","addr":"172.21.15.164:6837","nonce":316018252}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6837/2695634895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2695634895},{"type":"v1","addr":"172.21.15.154:6837","nonce":2695634895}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/1157843957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1157843957},{"type":"v1","addr":"172.21.15.164:6839","nonce":1157843957}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":68,"addr":"172.21.15.59:6839/377916146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":377916146},{"type":"v1","addr":"172.21.15.59:6839","nonce":377916146}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6839/3218637046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3218637046},{"type":"v1","addr":"172.21.15.154:6839","nonce":3218637046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27440":{"gid":27440,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1035281632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1035281632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1035281632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584,24560,24547,24529,24530]},"id":1}]} 2024-04-02T17:39:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 77 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 msgr2=0x7f31a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 0x7f31a00a6070 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f31a802c250 tx=0x7f31a802cf90 comp rx=0 tx=0).stop 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 msgr2=0x7f31bc19e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00ebc0 tx=0x7f31ac00c6a0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 shutdown_connections 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f31a00a3bb0 0x7f31a00a6070 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31bc109aa0 0x7f31bc1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31bc1090e0 0x7f31bc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 --2- 172.21.15.59:0/596838355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31bc107f10 0x7f31bc19e240 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.081+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 >> 172.21.15.59:0/596838355 conn(0x7f31bc076f10 msgr2=0x7f31bc10c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:29.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.082+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 shutdown_connections 2024-04-02T17:39:29.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.082+0000 7f31bbe00640 1 -- 172.21.15.59:0/596838355 wait complete. 2024-04-02T17:39:29.101 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T17:39:29.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.302+0000 7f43f7e00640 1 -- 172.21.15.59:0/413495010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 msgr2=0x7f43f810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.302+0000 7f43f7e00640 1 --2- 172.21.15.59:0/413495010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f810bf90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009be0 tx=0x7f43f002bef0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 -- 172.21.15.59:0/413495010 shutdown_connections 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 --2- 172.21.15.59:0/413495010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f810bf90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 --2- 172.21.15.59:0/413495010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8104d40 0x7f43f81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 --2- 172.21.15.59:0/413495010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8103b50 0x7f43f8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 -- 172.21.15.59:0/413495010 >> 172.21.15.59:0/413495010 conn(0x7f43f80ff0e0 msgr2=0x7f43f8101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 -- 172.21.15.59:0/413495010 shutdown_connections 2024-04-02T17:39:29.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.303+0000 7f43f7e00640 1 -- 172.21.15.59:0/413495010 wait complete. 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 Processor -- start 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 -- start start 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8103b50 0x7f43f8199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f81a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43f810f820 con 0x7f43f8104d40 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43f810f6a0 con 0x7f43f8103b50 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43f810f9a0 con 0x7f43f8105700 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33768/0 (socket says 172.21.15.59:33768) 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f6000640 1 -- 172.21.15.59:0/980628203 learned_addr learned my addr 172.21.15.59:0/980628203 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.304+0000 7f43f6a00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8103b50 0x7f43f8199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f7400640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f81a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 -- 172.21.15.59:0/980628203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 msgr2=0x7f43f81a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f81a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 -- 172.21.15.59:0/980628203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8103b50 msgr2=0x7f43f8199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8103b50 0x7f43f8199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 -- 172.21.15.59:0/980628203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f81a1a30 con 0x7f43f8104d40 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f7400640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f81a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f6000640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e9f0 tx=0x7f43e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:29.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e400ccc0 con 0x7f43f8104d40 2024-04-02T17:39:29.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f81a1d20 con 0x7f43f8104d40 2024-04-02T17:39:29.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.305+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e400ce60 con 0x7f43f8104d40 2024-04-02T17:39:29.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.306+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4018630 con 0x7f43f8104d40 2024-04-02T17:39:29.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.306+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f81a2210 con 0x7f43f8104d40 2024-04-02T17:39:29.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.307+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f810ae10 con 0x7f43f8104d40 2024-04-02T17:39:29.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.310+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f43e40046c0 con 0x7f43f8104d40 2024-04-02T17:39:29.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.312+0000 7f43ef400640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 0x7f43dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.312+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f43e40d10d0 con 0x7f43f8104d40 2024-04-02T17:39:29.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.313+0000 7f43f6a00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 0x7f43dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.313+0000 7f43f6a00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 0x7f43dc0a6070 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43f8104ba0 tx=0x7f43e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:29.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.313+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f43e409d9f0 con 0x7f43f8104d40 2024-04-02T17:39:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:29 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/1915659358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:29 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/596838355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:29 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/1915659358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:29 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/596838355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.512+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43f802d540 con 0x7f43f8104d40 2024-04-02T17:39:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.516+0000 7f43ef400640 1 -- 172.21.15.59:0/980628203 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22857 (secure 0 0 0) 0x7f43e40ac960 con 0x7f43f8104d40 2024-04-02T17:39:29.515 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-02T17:39:29.515 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"8286be62-f112-11ee-b647-cb9ed24678a4","created":"2024-04-02T17:01:33.642161+0000","modified":"2024-04-02T17:32:50.136490+0000","last_up_change":"2024-04-02T17:09:36.504122+0000","last_in_change":"2024-04-02T17:09:16.118770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T17:05:29.509926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T17:10:32.636166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T17:10:33.468395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cb8100a6-a589-4808-b2c2-870b291c27de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6803","nonce":1605463711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6805","nonce":1605463711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6809","nonce":1605463711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1605463711},{"type":"v1","addr":"172.21.15.59:6807","nonce":1605463711}]},"public_addr":"172.21.15.59:6803/1605463711","cluster_addr":"172.21.15.59:6805/1605463711","heartbeat_back_addr":"172.21.15.59:6809/1605463711","heartbeat_front_addr":"172.21.15.59:6807/1605463711","state":["exists","up"]},{"osd":1,"uuid":"e9e6de87-ec4b-49a8-b0b6-a5ea6541ce9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6801","nonce":212515491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6803","nonce":212515491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6807","nonce":212515491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":212515491},{"type":"v1","addr":"172.21.15.154:6805","nonce":212515491}]},"public_addr":"172.21.15.154:6801/212515491","cluster_addr":"172.21.15.154:6803/212515491","heartbeat_back_addr":"172.21.15.154:6807/212515491","heartbeat_front_addr":"172.21.15.154:6805/212515491","state":["exists","up"]},{"osd":2,"uuid":"8fde8bb0-3569-424d-8271-a0f77a83f346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6801","nonce":4028786479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6803","nonce":4028786479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6807","nonce":4028786479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4028786479},{"type":"v1","addr":"172.21.15.164:6805","nonce":4028786479}]},"public_addr":"172.21.15.164:6801/4028786479","cluster_addr":"172.21.15.164:6803/4028786479","heartbeat_back_addr":"172.21.15.164:6807/4028786479","heartbeat_front_addr":"172.21.15.164:6805/4028786479","state":["exists","up"]},{"osd":3,"uuid":"73897ded-e965-4239-b88f-776b7af2eb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6811","nonce":3330665578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6813","nonce":3330665578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6817","nonce":3330665578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3330665578},{"type":"v1","addr":"172.21.15.59:6815","nonce":3330665578}]},"public_addr":"172.21.15.59:6811/3330665578","cluster_addr":"172.21.15.59:6813/3330665578","heartbeat_back_addr":"172.21.15.59:6817/3330665578","heartbeat_front_addr":"172.21.15.59:6815/3330665578","state":["exists","up"]},{"osd":4,"uuid":"e81e729a-401f-4f5c-9749-6f88a4c93261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6809","nonce":2249512865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6811","nonce":2249512865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6815","nonce":2249512865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2249512865},{"type":"v1","addr":"172.21.15.154:6813","nonce":2249512865}]},"public_addr":"172.21.15.154:6809/2249512865","cluster_addr":"172.21.15.154:6811/2249512865","heartbeat_back_addr":"172.21.15.154:6815/2249512865","heartbeat_front_addr":"172.21.15.154:6813/2249512865","state":["exists","up"]},{"osd":5,"uuid":"287aef0f-1e4b-4121-8e66-fad24be440d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6809","nonce":3065900784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6811","nonce":3065900784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6815","nonce":3065900784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3065900784},{"type":"v1","addr":"172.21.15.164:6813","nonce":3065900784}]},"public_addr":"172.21.15.164:6809/3065900784","cluster_addr":"172.21.15.164:6811/3065900784","heartbeat_back_addr":"172.21.15.164:6815/3065900784","heartbeat_front_addr":"172.21.15.164:6813/3065900784","state":["exists","up"]},{"osd":6,"uuid":"005113f3-2b16-467c-be8c-a79457eaab2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6819","nonce":2899017867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6821","nonce":2899017867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6825","nonce":2899017867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2899017867},{"type":"v1","addr":"172.21.15.59:6823","nonce":2899017867}]},"public_addr":"172.21.15.59:6819/2899017867","cluster_addr":"172.21.15.59:6821/2899017867","heartbeat_back_addr":"172.21.15.59:6825/2899017867","heartbeat_front_addr":"172.21.15.59:6823/2899017867","state":["exists","up"]},{"osd":7,"uuid":"d5a328eb-01df-4384-ba8a-ec41925acd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6817","nonce":3912867880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6819","nonce":3912867880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6823","nonce":3912867880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3912867880},{"type":"v1","addr":"172.21.15.154:6821","nonce":3912867880}]},"public_addr":"172.21.15.154:6817/3912867880","cluster_addr":"172.21.15.154:6819/3912867880","heartbeat_back_addr":"172.21.15.154:6823/3912867880","heartbeat_front_addr":"172.21.15.154:6821/3912867880","state":["exists","up"]},{"osd":8,"uuid":"77628eb3-b084-412c-9e27-2c9d9bdc3aeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6817","nonce":1918014378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6819","nonce":1918014378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6823","nonce":1918014378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1918014378},{"type":"v1","addr":"172.21.15.164:6821","nonce":1918014378}]},"public_addr":"172.21.15.164:6817/1918014378","cluster_addr":"172.21.15.164:6819/1918014378","heartbeat_back_addr":"172.21.15.164:6823/1918014378","heartbeat_front_addr":"172.21.15.164:6821/1918014378","state":["exists","up"]},{"osd":9,"uuid":"bbeabc6e-28cb-4639-9ef1-5982c75d2e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6827","nonce":980178556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6829","nonce":980178556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6833","nonce":980178556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":980178556},{"type":"v1","addr":"172.21.15.59:6831","nonce":980178556}]},"public_addr":"172.21.15.59:6827/980178556","cluster_addr":"172.21.15.59:6829/980178556","heartbeat_back_addr":"172.21.15.59:6833/980178556","heartbeat_front_addr":"172.21.15.59:6831/980178556","state":["exists","up"]},{"osd":10,"uuid":"7079db2d-8f1f-4aea-b0dd-b7171210f38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6825","nonce":2125299053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6827","nonce":2125299053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6831","nonce":2125299053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2125299053},{"type":"v1","addr":"172.21.15.154:6829","nonce":2125299053}]},"public_addr":"172.21.15.154:6825/2125299053","cluster_addr":"172.21.15.154:6827/2125299053","heartbeat_back_addr":"172.21.15.154:6831/2125299053","heartbeat_front_addr":"172.21.15.154:6829/2125299053","state":["exists","up"]},{"osd":11,"uuid":"360f9d1e-4ae2-4299-92cb-5f8cb4a1e3a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6825","nonce":589108001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6827","nonce":589108001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6831","nonce":589108001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":589108001},{"type":"v1","addr":"172.21.15.164:6829","nonce":589108001}]},"public_addr":"172.21.15.164:6825/589108001","cluster_addr":"172.21.15.164:6827/589108001","heartbeat_back_addr":"172.21.15.164:6831/589108001","heartbeat_front_addr":"172.21.15.164:6829/589108001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:04:38.006564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:02.599464+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:26.642119+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:05:57.903706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:23.073815+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:06:47.509681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:20.445107+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:07:46.033332+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:10.597011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:08:43.837318+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:09.321171+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T17:09:33.951521+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/2274685378":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2569029589":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2590435245":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2036014369":"2024-04-03T17:02:49.350524+0000","172.21.15.59:0/2195394001":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/2817128980":"2024-04-03T17:02:49.350524+0000","172.21.15.59:6800/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:6801/3477889503":"2024-04-03T17:02:15.389808+0000","172.21.15.59:0/3686798564":"2024-04-03T17:02:15.389808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T17:39:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 msgr2=0x7f43dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 0x7f43dc0a6070 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43f8104ba0 tx=0x7f43e8009260 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 msgr2=0x7f43f819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e9f0 tx=0x7f43e400edf0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 shutdown_connections 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43dc0a3bb0 0x7f43dc0a6070 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f8104d40 0x7f43f819a350 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43f8105700 0x7f43f81a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 --2- 172.21.15.59:0/980628203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43f8103b50 0x7f43f8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.518+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 >> 172.21.15.59:0/980628203 conn(0x7f43f80ff0e0 msgr2=0x7f43f8107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.519+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 shutdown_connections 2024-04-02T17:39:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.519+0000 7f43f7e00640 1 -- 172.21.15.59:0/980628203 wait complete. 2024-04-02T17:39:29.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T17:39:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:29 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/1915659358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:29 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/596838355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T17:39:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 -- 172.21.15.59:0/3941209961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54681077e0 msgr2=0x7f5468109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 --2- 172.21.15.59:0/3941209961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54681077e0 0x7f5468109c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5460009900 tx=0x7f546002bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 -- 172.21.15.59:0/3941209961 shutdown_connections 2024-04-02T17:39:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 --2- 172.21.15.59:0/3941209961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54681077e0 0x7f5468109c30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 --2- 172.21.15.59:0/3941209961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5468104eb0 0x7f54681072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 --2- 172.21.15.59:0/3941209961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468102580 0x7f5468104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 -- 172.21.15.59:0/3941209961 >> 172.21.15.59:0/3941209961 conn(0x7f54680fbf90 msgr2=0x7f54680fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:29.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 -- 172.21.15.59:0/3941209961 shutdown_connections 2024-04-02T17:39:29.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.741+0000 7f546ca00640 1 -- 172.21.15.59:0/3941209961 wait complete. 2024-04-02T17:39:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.742+0000 7f546ca00640 1 Processor -- start 2024-04-02T17:39:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.742+0000 7f546ca00640 1 -- start start 2024-04-02T17:39:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 0x7f5468100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54681077e0 0x7f5468100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d870 con 0x7f5468102580 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d6f0 con 0x7f5468104eb0 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f546ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f546810d9f0 con 0x7f54681077e0 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f5465e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f5465e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33776/0 (socket says 172.21.15.59:33776) 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f5465e00640 1 -- 172.21.15.59:0/708657128 learned_addr learned my addr 172.21.15.59:0/708657128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f545d400640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 0x7f5468100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.743+0000 7f5466800640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54681077e0 0x7f5468100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 -- 172.21.15.59:0/708657128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54681077e0 msgr2=0x7f5468100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54681077e0 0x7f5468100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 -- 172.21.15.59:0/708657128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 msgr2=0x7f5468100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 0x7f5468100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 -- 172.21.15.59:0/708657128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468101300 con 0x7f5468102580 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f545d400640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 0x7f5468100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T17:39:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f5465e00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f545800ba50 tx=0x7f545800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458019070 con 0x7f5468102580 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.744+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5458014440 con 0x7f5468102580 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.745+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54681a1fa0 con 0x7f5468102580 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.745+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580075d0 con 0x7f5468102580 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.745+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681a2400 con 0x7f5468102580 2024-04-02T17:39:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.746+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54680688f0 con 0x7f5468102580 2024-04-02T17:39:29.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.749+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f545801c070 con 0x7f5468102580 2024-04-02T17:39:29.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.751+0000 7f545f400640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 0x7f544c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:29.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.751+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f54580cfea0 con 0x7f5468102580 2024-04-02T17:39:29.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.751+0000 7f545d400640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 0x7f544c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:29.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.752+0000 7f545d400640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 0x7f544c0a6070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5468101c60 tx=0x7f5450009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:29.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:29.752+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f545809c1d0 con 0x7f5468102580 2024-04-02T17:39:30.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.002+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f54680fda80 con 0x7f5468102580 2024-04-02T17:39:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.007+0000 7f545f400640 1 -- 172.21.15.59:0/708657128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f54580a04d0 con 0x7f5468102580 2024-04-02T17:39:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T17:39:30.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.012+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 msgr2=0x7f544c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:30.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.012+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 0x7f544c0a6070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5468101c60 tx=0x7f5450009110 comp rx=0 tx=0).stop 2024-04-02T17:39:30.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.012+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 msgr2=0x7f5468101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:30.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.012+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f545800ba50 tx=0x7f545800be50 comp rx=0 tx=0).stop 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 shutdown_connections 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54681077e0 0x7f5468100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5468104eb0 0x7f5468100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f544c0a3bb0 0x7f544c0a6070 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 --2- 172.21.15.59:0/708657128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5468102580 0x7f5468101ef0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 >> 172.21.15.59:0/708657128 conn(0x7f54680fbf90 msgr2=0x7f5468102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 shutdown_connections 2024-04-02T17:39:30.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.013+0000 7f546ca00640 1 -- 172.21.15.59:0/708657128 wait complete. 2024-04-02T17:39:30.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:39:30.004+0000 7fe5dea00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T17:39:30.040 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.247+0000 7f5446600640 1 -- 172.21.15.59:0/354242267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100cf0 msgr2=0x7f5440101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.247+0000 7f5446600640 1 --2- 172.21.15.59:0/354242267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100cf0 0x7f5440101150 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f542c009900 tx=0x7f542c02bc30 comp rx=0 tx=0).stop 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 -- 172.21.15.59:0/354242267 shutdown_connections 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 --2- 172.21.15.59:0/354242267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100cf0 0x7f5440101150 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 --2- 172.21.15.59:0/354242267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440101690 0x7f5440101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 --2- 172.21.15.59:0/354242267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440100390 0x7f54401007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 -- 172.21.15.59:0/354242267 >> 172.21.15.59:0/354242267 conn(0x7f54400fbf90 msgr2=0x7f54400fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 -- 172.21.15.59:0/354242267 shutdown_connections 2024-04-02T17:39:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.248+0000 7f5446600640 1 -- 172.21.15.59:0/354242267 wait complete. 2024-04-02T17:39:30.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.249+0000 7f5446600640 1 Processor -- start 2024-04-02T17:39:30.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.249+0000 7f5446600640 1 -- start start 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100390 0x7f544019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440101690 0x7f54401a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f544010f9d0 con 0x7f5440100390 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f544010f850 con 0x7f5440100cf0 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f5446600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f544010fb50 con 0x7f5440101690 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100390 0x7f544019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:50816/0 (socket says 172.21.15.59:50816) 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 -- 172.21.15.59:0/3154148389 learned_addr learned my addr 172.21.15.59:0/3154148389 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543fe00640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440101690 0x7f54401a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 -- 172.21.15.59:0/3154148389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440101690 msgr2=0x7f54401a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440101690 0x7f54401a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 -- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100390 msgr2=0x7f544019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100390 0x7f544019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.250+0000 7f543ea00640 1 -- 172.21.15.59:0/3154148389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a1cf0 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f543ea00640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f542c02c8e0 tx=0x7f542c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c043020 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c0054e0 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a1f80 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c003b50 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.251+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a2440 con 0x7f5440100cf0 2024-04-02T17:39:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.252+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544010b2b0 con 0x7f5440100cf0 2024-04-02T17:39:30.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.254+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f542c046020 con 0x7f5440100cf0 2024-04-02T17:39:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.255+0000 7f5437e00640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.256+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8765+0+0 (secure 0 0 0) 0x7f542c0f2630 con 0x7f5440100cf0 2024-04-02T17:39:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.256+0000 7f543f400640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:30.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.257+0000 7f543f400640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5430009bd0 tx=0x7f5430005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.259+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f542c04d020 con 0x7f5440100cf0 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: pgmap v1182: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/980628203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T17:39:30.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mon[31664]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mds[71141]: -- 172.21.15.59:0/2785177764 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a43e51a40 con 0x556a43e24800 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: starting mds.g at 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-mds[70495]: -- 172.21.15.59:0/1529624454 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b5664fa40 con 0x560b56622c00 2024-04-02T17:39:30.299 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:39:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: starting mds.d at 2024-04-02T17:39:30.399 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.399 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.399 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mds[66184]: -- 172.21.15.154:0/2140219270 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566e5d9fa40 con 0x5566e5d72c00 2024-04-02T17:39:30.399 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: starting mds.e at 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mds[66821]: -- 172.21.15.154:0/2286918873 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ecfb1a3a40 con 0x55ecfb176800 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: starting mds.b at 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: pgmap v1182: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-02T17:39:30.400 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/980628203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mon[36479]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.401 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mds[66494]: -- 172.21.15.154:0/2624671585 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56545d605a40 con 0x56545d5d8800 2024-04-02T17:39:30.402 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: starting mds.h at 2024-04-02T17:39:30.402 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k[65856]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.402 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k[65856]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.402 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-mds[65860]: -- 172.21.15.154:0/2610912249 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0a40fba40 con 0x55d0a40ce800 2024-04-02T17:39:30.402 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:39:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k[65856]: starting mds.k at 2024-04-02T17:39:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:30.504+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5440062700 con 0x7f5440100cf0 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: pgmap v1182: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/980628203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/708657128' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T17:39:30.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mon[36499]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mds[66927]: -- 172.21.15.164:0/2662875106 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561dbbf7ba40 con 0x561dbbf4ec00 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: starting mds.f at 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mds[66613]: -- 172.21.15.164:0/465964212 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55832cc37a40 con 0x55832cc0ac00 2024-04-02T17:39:30.573 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: starting mds.l at 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mds[67235]: -- 172.21.15.164:0/2723775164 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3498f3a40 con 0x55b3498c6c00 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: starting mds.c at 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i[67538]: ignoring --setuser ceph since I am not root 2024-04-02T17:39:30.574 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i[67538]: ignoring --setgroup ceph since I am not root 2024-04-02T17:39:30.575 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-mds[67542]: -- 172.21.15.164:0/787712911 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f573ae1a40 con 0x55f573ab4c00 2024-04-02T17:39:30.575 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:39:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i[67538]: starting mds.i at 2024-04-02T17:39:31.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.015+0000 7f5437e00640 1 -- 172.21.15.59:0/3154148389 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7f542c0beda0 con 0x7f5440100cf0 2024-04-02T17:39:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.019+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 msgr2=0x7f54240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.019+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5430009bd0 tx=0x7f5430005f90 comp rx=0 tx=0).stop 2024-04-02T17:39:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.019+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 msgr2=0x7f544019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.019+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f542c02c8e0 tx=0x7f542c004970 comp rx=0 tx=0).stop 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 shutdown_connections 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5440101690 0x7f54401a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5440100cf0 0x7f544019a610 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f54240a3bb0 0x7f54240a6070 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 --2- 172.21.15.59:0/3154148389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5440100390 0x7f544019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.020+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 >> 172.21.15.59:0/3154148389 conn(0x7f54400fbf90 msgr2=0x7f54400fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.021+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 shutdown_connections 2024-04-02T17:39:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.021+0000 7f5446600640 1 -- 172.21.15.59:0/3154148389 wait complete. 2024-04-02T17:39:31.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T17:39:31.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.285+0000 7f9b3d600640 1 -- 172.21.15.59:0/331066419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b381077e0 msgr2=0x7f9b38109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.285+0000 7f9b3d600640 1 --2- 172.21.15.59:0/331066419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b381077e0 0x7f9b38109c30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009be0 tx=0x7f9b3002bef0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 -- 172.21.15.59:0/331066419 shutdown_connections 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 --2- 172.21.15.59:0/331066419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b381077e0 0x7f9b38109c30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 --2- 172.21.15.59:0/331066419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b38104eb0 0x7f9b381072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 --2- 172.21.15.59:0/331066419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38102580 0x7f9b38104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 -- 172.21.15.59:0/331066419 >> 172.21.15.59:0/331066419 conn(0x7f9b380fbf90 msgr2=0x7f9b380fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 -- 172.21.15.59:0/331066419 shutdown_connections 2024-04-02T17:39:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.286+0000 7f9b3d600640 1 -- 172.21.15.59:0/331066419 wait complete. 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.287+0000 7f9b3d600640 1 Processor -- start 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.287+0000 7f9b3d600640 1 -- start start 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.287+0000 7f9b3d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102580 0x7f9b38199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.287+0000 7f9b3d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.287+0000 7f9b3d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b381077e0 0x7f9b381a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b3d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d780 con 0x7f9b38104eb0 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b3d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d600 con 0x7f9b38102580 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b3d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810d900 con 0x7f9b381077e0 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59400/0 (socket says 172.21.15.59:59400) 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102580 0x7f9b38199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 -- 172.21.15.59:0/2692872053 learned_addr learned my addr 172.21.15.59:0/2692872053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b37400640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b381077e0 0x7f9b381a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 -- 172.21.15.59:0/2692872053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b381077e0 msgr2=0x7f9b381a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b381077e0 0x7f9b381a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 -- 172.21.15.59:0/2692872053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102580 msgr2=0x7f9b38199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102580 0x7f9b38199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.288+0000 7f9b36000640 1 -- 172.21.15.59:0/2692872053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a1bf0 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b36000640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9a0 tx=0x7f9b2400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cd80 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b240035f0 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a1ee0 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24010670 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.289+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a23a0 con 0x7f9b38104eb0 2024-04-02T17:39:31.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.290+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b381092d0 con 0x7f9b38104eb0 2024-04-02T17:39:31.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.293+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b240037a0 con 0x7f9b38104eb0 2024-04-02T17:39:31.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.294+0000 7f9b2f400640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:31.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.295+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8765+0+0 (secure 0 0 0) 0x7f9b240d0940 con 0x7f9b38104eb0 2024-04-02T17:39:31.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.295+0000 7f9b36a00640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:31.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.296+0000 7f9b36a00640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9b280095f0 tx=0x7f9b28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:31.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.296+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b2409d140 con 0x7f9b38104eb0 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3154148389' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: Cluster is now healthy 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6833/1669415395,v1:172.21.15.154:6835/1669415395] up:boot 2024-04-02T17:39:31.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6836/3168500829,v1:172.21.15.154:6839/3168500829] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6837/1754376634,v1:172.21.15.154:6838/1754376634] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6836/4289419988,v1:172.21.15.164:6838/4289419988] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6837/585830967,v1:172.21.15.164:6839/585830967] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6832/2410775278,v1:172.21.15.164:6834/2410775278] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.164:6833/194290033,v1:172.21.15.164:6835/194290033] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.154:6832/994630516,v1:172.21.15.154:6834/994630516] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6836/1243828102,v1:172.21.15.59:6837/1243828102] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: mds.? [v2:172.21.15.59:6838/390002354,v1:172.21.15.59:6839/390002354] up:boot 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: fsmap 12 up:standby 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:39:31.299 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:39:31.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:39:31.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:39:31.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:39:31.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:31 smithi059 ceph-mon[31664]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:39:31.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:31.506+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9b3802d370 con 0x7f9b38104eb0 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3154148389' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: Cluster is now healthy 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6833/1669415395,v1:172.21.15.154:6835/1669415395] up:boot 2024-04-02T17:39:31.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6836/3168500829,v1:172.21.15.154:6839/3168500829] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6837/1754376634,v1:172.21.15.154:6838/1754376634] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6836/4289419988,v1:172.21.15.164:6838/4289419988] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6837/585830967,v1:172.21.15.164:6839/585830967] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6832/2410775278,v1:172.21.15.164:6834/2410775278] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.164:6833/194290033,v1:172.21.15.164:6835/194290033] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.154:6832/994630516,v1:172.21.15.154:6834/994630516] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6836/1243828102,v1:172.21.15.59:6837/1243828102] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: mds.? [v2:172.21.15.59:6838/390002354,v1:172.21.15.59:6839/390002354] up:boot 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: fsmap 12 up:standby 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:39:31.573 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:39:31.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:39:31.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:39:31.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:39:31.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:39:31.574 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:31 smithi164 ceph-mon[36499]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:39:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3154148389' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T17:39:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T17:39:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T17:39:31.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: Cluster is now healthy 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6833/1669415395,v1:172.21.15.154:6835/1669415395] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6836/3168500829,v1:172.21.15.154:6839/3168500829] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6837/1754376634,v1:172.21.15.154:6838/1754376634] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6836/4289419988,v1:172.21.15.164:6838/4289419988] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6837/585830967,v1:172.21.15.164:6839/585830967] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6832/2410775278,v1:172.21.15.164:6834/2410775278] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.164:6833/194290033,v1:172.21.15.164:6835/194290033] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.154:6832/994630516,v1:172.21.15.154:6834/994630516] up:boot 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T17:39:31.650 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6836/1243828102,v1:172.21.15.59:6837/1243828102] up:boot 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: mds.? [v2:172.21.15.59:6838/390002354,v1:172.21.15.59:6839/390002354] up:boot 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: fsmap 12 up:standby 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T17:39:31.651 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T17:39:31.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T17:39:31.652 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:31 smithi154 ceph-mon[36479]: from='mgr.14152 172.21.15.59:0/431089069' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T17:39:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.164+0000 7f9b2f400640 1 -- 172.21.15.59:0/2692872053 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f9b2409ca40 con 0x7f9b38104eb0 2024-04-02T17:39:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-02T17:39:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 msgr2=0x7f9b1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9b280095f0 tx=0x7f9b28005f90 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 msgr2=0x7f9b3819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9a0 tx=0x7f9b2400ee60 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 shutdown_connections 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f9b1c0a3bb0 0x7f9b1c0a6070 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b38104eb0 0x7f9b3819a510 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b381077e0 0x7f9b381a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 --2- 172.21.15.59:0/2692872053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b38102580 0x7f9b38199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.168+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 >> 172.21.15.59:0/2692872053 conn(0x7f9b380fbf90 msgr2=0x7f9b38101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:32.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.170+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 shutdown_connections 2024-04-02T17:39:32.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.171+0000 7f9b3d600640 1 -- 172.21.15.59:0/2692872053 wait complete. 2024-04-02T17:39:32.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T17:39:32.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 -- 172.21.15.59:0/2229922499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 msgr2=0x7f4370100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:32.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 --2- 172.21.15.59:0/2229922499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370100f50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f435c009860 tx=0x7f435c02bb90 comp rx=0 tx=0).stop 2024-04-02T17:39:32.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 -- 172.21.15.59:0/2229922499 shutdown_connections 2024-04-02T17:39:32.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 --2- 172.21.15.59:0/2229922499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4370101490 0x7f4370105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 --2- 172.21.15.59:0/2229922499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370100f50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 --2- 172.21.15.59:0/2229922499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43700ff9c0 0x7f43700ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.454+0000 7f4375600640 1 -- 172.21.15.59:0/2229922499 >> 172.21.15.59:0/2229922499 conn(0x7f43700fb2c0 msgr2=0x7f43700fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:32.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.455+0000 7f4375600640 1 -- 172.21.15.59:0/2229922499 shutdown_connections 2024-04-02T17:39:32.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.456+0000 7f4375600640 1 -- 172.21.15.59:0/2229922499 wait complete. 2024-04-02T17:39:32.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.457+0000 7f4375600640 1 Processor -- start 2024-04-02T17:39:32.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.457+0000 7f4375600640 1 -- start start 2024-04-02T17:39:32.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.458+0000 7f4375600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:32.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.458+0000 7f4375600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:32.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.458+0000 7f4375600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4370101490 0x7f437010ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:32.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.458+0000 7f436ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f436f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4370101490 0x7f437010ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f436ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.59:56512/0 (socket says 172.21.15.59:56512) 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f4375600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4370113da0 con 0x7f4370101490 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f4375600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4370113c20 con 0x7f43700ff9c0 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f4375600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4370113f20 con 0x7f4370100af0 2024-04-02T17:39:32.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f436ea00640 1 -- 172.21.15.59:0/3754604897 learned_addr learned my addr 172.21.15.59:0/3754604897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.459+0000 7f436e000640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 -- 172.21.15.59:0/3754604897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 msgr2=0x7f4370107f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 -- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4370101490 msgr2=0x7f437010ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4370101490 0x7f437010ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 -- 172.21.15.59:0/3754604897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437010f590 con 0x7f43700ff9c0 2024-04-02T17:39:32.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.460+0000 7f436ea00640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f436000d950 tx=0x7f436000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:32.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.461+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360017030 con 0x7f43700ff9c0 2024-04-02T17:39:32.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.461+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360004020 con 0x7f43700ff9c0 2024-04-02T17:39:32.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.461+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360003d80 con 0x7f43700ff9c0 2024-04-02T17:39:32.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.462+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437010f880 con 0x7f43700ff9c0 2024-04-02T17:39:32.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.462+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701af090 con 0x7f43700ff9c0 2024-04-02T17:39:32.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.464+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370068d40 con 0x7f43700ff9c0 2024-04-02T17:39:32.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.468+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f436001a070 con 0x7f43700ff9c0 2024-04-02T17:39:32.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.471+0000 7f4367400640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 0x7f43540a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T17:39:32.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.471+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8182+0+0 (secure 0 0 0) 0x7f43600cf4f0 con 0x7f43700ff9c0 2024-04-02T17:39:32.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.472+0000 7f436e000640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 0x7f43540a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T17:39:32.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.474+0000 7f436e000640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 0x7f43540a6240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f435c034a60 tx=0x7f435c034b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T17:39:32.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.474+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f436009bfa0 con 0x7f43700ff9c0 2024-04-02T17:39:32.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:32 smithi059 ceph-mon[31664]: pgmap v1184: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:39:32.547 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:32 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:32.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:32 smithi164 ceph-mon[36499]: pgmap v1184: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:39:32.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:32 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:32.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:32 smithi154 ceph-mon[36479]: pgmap v1184: 97 pgs: 97 active+clean; 890 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-02T17:39:32.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:32 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:32.729+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4370104b40 con 0x7f43700ff9c0 2024-04-02T17:39:33.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.195+0000 7f4367400640 1 -- 172.21.15.59:0/3754604897 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7f436009b8a0 con 0x7f43700ff9c0 2024-04-02T17:39:33.194 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 msgr2=0x7f43540a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 0x7f43540a6240 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f435c034a60 tx=0x7f435c034b30 comp rx=0 tx=0).stop 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 msgr2=0x7f43701079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f436000d950 tx=0x7f436000de10 comp rx=0 tx=0).stop 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 shutdown_connections 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:6800/1634390325,v1:172.21.15.59:6801/1634390325] conn(0x7f43540a3d80 0x7f43540a6240 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:33.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4370101490 0x7f437010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4370100af0 0x7f4370107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 --2- 172.21.15.59:0/3754604897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43700ff9c0 0x7f43701079c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T17:39:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.198+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 >> 172.21.15.59:0/3754604897 conn(0x7f43700fb2c0 msgr2=0x7f4370101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T17:39:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.199+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 shutdown_connections 2024-04-02T17:39:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-02T17:39:33.199+0000 7f4375600640 1 -- 172.21.15.59:0/3754604897 wait complete. 2024-04-02T17:39:33.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:39:33.303 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:39:33.335 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:39:33.365 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T17:39:33.365 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T17:39:33.419 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T17:39:33.433 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T17:39:33.449 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T17:39:33.449 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-02T17:39:33.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T17:39:33.522 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T17:39:33.522 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-mon[31664]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T17:39:33.522 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-mon[31664]: from='client.? 172.21.15.59:0/3754604897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.522 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-mon[31664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:33 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T17:39:33.571 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:33 smithi164 ceph-mon[36499]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T17:39:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:33 smithi164 ceph-mon[36499]: from='client.? 172.21.15.59:0/3754604897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.572 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:33 smithi164 ceph-mon[36499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:33 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/2692872053' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T17:39:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:33 smithi154 ceph-mon[36479]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T17:39:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:33 smithi154 ceph-mon[36479]: from='client.? 172.21.15.59:0/3754604897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:33 smithi154 ceph-mon[36479]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T17:39:33.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 systemd[1]: Stopping Ceph mon.a for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:34.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:39:33.977+0000 7fe5e5800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:34.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a[31660]: 2024-04-02T17:39:33.978+0000 7fe5e5800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T17:39:34.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:34 smithi059 podman[100576]: 2024-04-02 17:39:34.182749603 +0000 UTC m=+0.338182577 container died b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-02T17:39:34.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:34 smithi059 podman[100576]: 2024-04-02 17:39:34.616435542 +0000 UTC m=+0.771868520 container cleanup b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:39:34.798 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:34 smithi059 bash[100576]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T17:39:35.275 INFO:journalctl@ceph.mon.a.smithi059.stdout:Apr 02 17:39:34 smithi059 podman[100589]: 2024-04-02 17:39:34.892680256 +0000 UTC m=+0.694193791 container remove b4e1aa018a5e5f2da5144639a68d98ca1fc4b7d9086001c36ebe5e48c027bcec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:39:35.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.a.service' 2024-04-02T17:39:35.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:35.450 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-02T17:39:35.451 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-02T17:39:35.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.b 2024-04-02T17:39:35.821 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:35 smithi154 systemd[1]: Stopping Ceph mon.b for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:36.147 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-b[36475]: 2024-04-02T17:39:35.822+0000 7f49a6ae4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:36.147 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-b[36475]: 2024-04-02T17:39:35.822+0000 7f49a6ae4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-02T17:39:36.148 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:35 smithi154 podman[75971]: 2024-04-02 17:39:35.883647383 +0000 UTC m=+0.184133152 container died 14d487407afd3893ce7bdfd0a5bf3e49c47371f0a5d0a54e4ca7c424a69f5b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T17:39:36.398 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:36 smithi154 podman[75971]: 2024-04-02 17:39:36.231017289 +0000 UTC m=+0.531503047 container cleanup 14d487407afd3893ce7bdfd0a5bf3e49c47371f0a5d0a54e4ca7c424a69f5b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-b, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T17:39:36.399 INFO:journalctl@ceph.mon.b.smithi154.stdout:Apr 02 17:39:36 smithi154 bash[75971]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-b 2024-04-02T17:39:36.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.b.service' 2024-04-02T17:39:36.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:36.671 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-02T17:39:36.671 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-02T17:39:36.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.c 2024-04-02T17:39:37.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:36 smithi164 systemd[1]: Stopping Ceph mon.c for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:37.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-c[36492]: 2024-04-02T17:39:37.061+0000 7f54fbd72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-c[36492]: 2024-04-02T17:39:37.061+0000 7f54fbd72700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T17:39:37.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:37 smithi164 podman[76632]: 2024-04-02 17:39:37.23510832 +0000 UTC m=+0.323282852 container died cd723794deac999f03f5f8620ea39709e399d3640695f9024b8ee1e996b75f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-c, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T17:39:37.821 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:37 smithi164 podman[76632]: 2024-04-02 17:39:37.620543033 +0000 UTC m=+0.708717570 container cleanup cd723794deac999f03f5f8620ea39709e399d3640695f9024b8ee1e996b75f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-c, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-02T17:39:37.822 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 02 17:39:37 smithi164 bash[76632]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mon-c 2024-04-02T17:39:37.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mon.c.service' 2024-04-02T17:39:38.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:38.062 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-02T17:39:38.062 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-02T17:39:38.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T17:39:38.298 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:38 smithi059 systemd[1]: Stopping Ceph mgr.x for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:39.011 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:39:38.644+0000 7f4e63800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:39.012 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x[31896]: 2024-04-02T17:39:38.644+0000 7f4e63800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-02T17:39:39.012 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:38 smithi059 podman[100695]: 2024-04-02 17:39:38.738292636 +0000 UTC m=+0.277601523 container died fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T17:39:39.298 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:39 smithi059 podman[100695]: 2024-04-02 17:39:39.137027935 +0000 UTC m=+0.676336853 container cleanup fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T17:39:39.298 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:39 smithi059 bash[100695]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x 2024-04-02T17:39:39.731 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Apr 02 17:39:39 smithi059 podman[100709]: 2024-04-02 17:39:39.371813327 +0000 UTC m=+0.621102022 container remove fdd3dd274a9790271f4bc56c40d6fcafc1952dd98c564770bbacdb0c5e695961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-02T17:39:39.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-04-02T17:39:39.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:39.892 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-02T17:39:39.892 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-02T17:39:39.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.y 2024-04-02T17:39:40.221 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:39 smithi154 systemd[1]: Stopping Ceph mgr.y for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:40.550 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:40 smithi154 podman[76092]: 2024-04-02 17:39:40.223232005 +0000 UTC m=+0.140679850 container died 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5) 2024-04-02T17:39:40.899 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:40 smithi154 podman[76092]: 2024-04-02 17:39:40.55273217 +0000 UTC m=+0.470180006 container cleanup 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:39:40.899 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:40 smithi154 bash[76092]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y 2024-04-02T17:39:40.899 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:40 smithi154 podman[76106]: 2024-04-02 17:39:40.762478041 +0000 UTC m=+0.536449043 container remove 59f70b80d2b62899c0f75ba0d08bf0e7d5371ee6f7fd8815c5aebe9cccf7f300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-y, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T17:39:40.899 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Apr 02 17:39:40 smithi154 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-02T17:39:40.963 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-04-02T17:39:41.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:41.001 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-02T17:39:41.002 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-02T17:39:41.002 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.z 2024-04-02T17:39:41.321 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:39:41 smithi164 systemd[1]: Stopping Ceph mgr.z for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:41.634 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:39:41 smithi164 podman[76757]: 2024-04-02 17:39:41.32930939 +0000 UTC m=+0.132119536 container died 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T17:39:42.008 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:39:41 smithi164 podman[76757]: 2024-04-02 17:39:41.725174498 +0000 UTC m=+0.527984683 container cleanup 4e13f0699e6dc9057ec0fc10ba3a254be6bfc863f2105164ffe0ceb7067e4b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T17:39:42.008 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 02 17:39:41 smithi164 bash[76757]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mgr-z 2024-04-02T17:39:42.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-04-02T17:39:42.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:42.168 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-02T17:39:42.168 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T17:39:42.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T17:39:42.548 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:42 smithi059 systemd[1]: Stopping Ceph osd.0 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:43.298 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0[44182]: 2024-04-02T17:39:42.925+0000 7f687ec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:43.298 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0[44182]: 2024-04-02T17:39:42.925+0000 7f687ec00700 -1 osd.0 96 *** Got signal Terminated *** 2024-04-02T17:39:43.298 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0[44182]: 2024-04-02T17:39:42.925+0000 7f687ec00700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:39:49.048 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:48 smithi059 podman[100814]: 2024-04-02 17:39:48.757841996 +0000 UTC m=+6.188406572 container died d7c5149a912e3d0e24bf32c5a14482191fe334738efa2e705bbfbc9f650c5d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T17:39:49.476 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:49 smithi059 podman[100814]: 2024-04-02 17:39:49.21669603 +0000 UTC m=+6.647260650 container cleanup d7c5149a912e3d0e24bf32c5a14482191fe334738efa2e705bbfbc9f650c5d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T17:39:49.477 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:49 smithi059 bash[100814]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T17:39:49.797 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:49 smithi059 podman[100845]: 2024-04-02 17:39:49.475942379 +0000 UTC m=+0.690844796 container remove d7c5149a912e3d0e24bf32c5a14482191fe334738efa2e705bbfbc9f650c5d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T17:39:50.571 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:50 smithi059 podman[100936]: 2024-04-02 17:39:50.214317182 +0000 UTC m=+0.037436043 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:39:50.571 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:50 smithi059 podman[100936]: 2024-04-02 17:39:50.31004798 +0000 UTC m=+0.133166854 container create 8e7de1e716ff5ace53f323f565d93919360be985cbef18c16e894a79fe64f274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS) 2024-04-02T17:39:50.571 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:50 smithi059 podman[100936]: 2024-04-02 17:39:50.571959334 +0000 UTC m=+0.395078215 container init 8e7de1e716ff5ace53f323f565d93919360be985cbef18c16e894a79fe64f274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T17:39:51.046 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:50 smithi059 podman[100936]: 2024-04-02 17:39:50.584753573 +0000 UTC m=+0.407872477 container start 8e7de1e716ff5ace53f323f565d93919360be985cbef18c16e894a79fe64f274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:39:51.047 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:50 smithi059 podman[100936]: 2024-04-02 17:39:50.662252964 +0000 UTC m=+0.485371868 container attach 8e7de1e716ff5ace53f323f565d93919360be985cbef18c16e894a79fe64f274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T17:39:51.298 INFO:journalctl@ceph.osd.0.smithi059.stdout:Apr 02 17:39:51 smithi059 podman[100936]: 2024-04-02 17:39:51.048230801 +0000 UTC m=+0.871349659 container died 8e7de1e716ff5ace53f323f565d93919360be985cbef18c16e894a79fe64f274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T17:39:51.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T17:39:51.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:39:51.882 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T17:39:51.882 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T17:39:51.883 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T17:39:52.189 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:51 smithi154 systemd[1]: Stopping Ceph osd.1 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:39:52.649 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:39:52.188+0000 7fdab002d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:39:52.649 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:39:52.188+0000 7fdab002d700 -1 osd.1 96 *** Got signal Terminated *** 2024-04-02T17:39:52.649 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1[41048]: 2024-04-02T17:39:52.188+0000 7fdab002d700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:39:57.899 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:57 smithi154 podman[76225]: 2024-04-02 17:39:57.620816758 +0000 UTC m=+5.537880948 container died dd676ac9f0318ce4749c33b426aa351a2a11a0893d2c28dc1dbea8b8662adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:39:58.281 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76225]: 2024-04-02 17:39:58.065006137 +0000 UTC m=+5.982070301 container cleanup dd676ac9f0318ce4749c33b426aa351a2a11a0893d2c28dc1dbea8b8662adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:39:58.281 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 bash[76225]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T17:39:58.649 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76242]: 2024-04-02 17:39:58.282962468 +0000 UTC m=+0.658405307 container remove dd676ac9f0318ce4749c33b426aa351a2a11a0893d2c28dc1dbea8b8662adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T17:39:59.109 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76336]: 2024-04-02 17:39:58.632096617 +0000 UTC m=+0.017915941 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:39:59.109 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76336]: 2024-04-02 17:39:58.750784881 +0000 UTC m=+0.136604241 container create 9cc1845cc83fd6d56816378d8c7c1e1c238041debb6c7b0bbe41838bb08299f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T17:39:59.110 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76336]: 2024-04-02 17:39:58.96200444 +0000 UTC m=+0.347823768 container init 9cc1845cc83fd6d56816378d8c7c1e1c238041debb6c7b0bbe41838bb08299f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:39:59.110 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:58 smithi154 podman[76336]: 2024-04-02 17:39:58.965537666 +0000 UTC m=+0.351356987 container start 9cc1845cc83fd6d56816378d8c7c1e1c238041debb6c7b0bbe41838bb08299f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326) 2024-04-02T17:39:59.110 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:59 smithi154 podman[76336]: 2024-04-02 17:39:59.111462615 +0000 UTC m=+0.497281942 container attach 9cc1845cc83fd6d56816378d8c7c1e1c238041debb6c7b0bbe41838bb08299f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T17:39:59.649 INFO:journalctl@ceph.osd.1.smithi154.stdout:Apr 02 17:39:59 smithi154 podman[76336]: 2024-04-02 17:39:59.588329636 +0000 UTC m=+0.974148956 container died 9cc1845cc83fd6d56816378d8c7c1e1c238041debb6c7b0bbe41838bb08299f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-1-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T17:40:00.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T17:40:00.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:00.228 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T17:40:00.228 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T17:40:00.228 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T17:40:00.519 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:00 smithi164 systemd[1]: Stopping Ceph osd.2 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:00.821 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2[41668]: 2024-04-02T17:40:00.519+0000 7f6e4d6c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:00.822 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2[41668]: 2024-04-02T17:40:00.519+0000 7f6e4d6c7700 -1 osd.2 96 *** Got signal Terminated *** 2024-04-02T17:40:00.822 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2[41668]: 2024-04-02T17:40:00.519+0000 7f6e4d6c7700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:06.393 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 podman[76897]: 2024-04-02 17:40:06.096233238 +0000 UTC m=+5.674882601 container died 9944b704785b18e8b9fa37330237dcbf720df11b76866e0522b5fd11862f5023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:06.822 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 podman[76897]: 2024-04-02 17:40:06.395213668 +0000 UTC m=+5.973862993 container cleanup 9944b704785b18e8b9fa37330237dcbf720df11b76866e0522b5fd11862f5023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T17:40:06.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 bash[76897]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T17:40:06.823 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 podman[76914]: 2024-04-02 17:40:06.638119423 +0000 UTC m=+0.538251756 container remove 9944b704785b18e8b9fa37330237dcbf720df11b76866e0522b5fd11862f5023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T17:40:07.223 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 podman[77007]: 2024-04-02 17:40:06.851849386 +0000 UTC m=+0.017782171 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:07.224 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:06 smithi164 podman[77007]: 2024-04-02 17:40:06.980486686 +0000 UTC m=+0.146419459 container create 1091317076dcee64699636b68394ae8f9bcfae8b28ff60806088e0cd457ca8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T17:40:07.224 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:07 smithi164 podman[77007]: 2024-04-02 17:40:07.225103122 +0000 UTC m=+0.391035896 container init 1091317076dcee64699636b68394ae8f9bcfae8b28ff60806088e0cd457ca8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:40:07.571 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:07 smithi164 podman[77007]: 2024-04-02 17:40:07.228669855 +0000 UTC m=+0.394602623 container start 1091317076dcee64699636b68394ae8f9bcfae8b28ff60806088e0cd457ca8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326) 2024-04-02T17:40:07.572 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:07 smithi164 podman[77007]: 2024-04-02 17:40:07.341770837 +0000 UTC m=+0.507703633 container attach 1091317076dcee64699636b68394ae8f9bcfae8b28ff60806088e0cd457ca8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T17:40:08.068 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 02 17:40:07 smithi164 podman[77007]: 2024-04-02 17:40:07.793262428 +0000 UTC m=+0.959195217 container died 1091317076dcee64699636b68394ae8f9bcfae8b28ff60806088e0cd457ca8bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:08.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T17:40:08.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:08.431 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T17:40:08.431 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T17:40:08.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T17:40:08.798 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:08 smithi059 systemd[1]: Stopping Ceph osd.3 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:09.548 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:40:09.056+0000 7fd793400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:09.548 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:40:09.056+0000 7fd793400700 -1 osd.3 96 *** Got signal Terminated *** 2024-04-02T17:40:09.548 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:40:09.056+0000 7fd793400700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:09.549 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:09 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:09.053+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:09.829 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:09.535+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:10.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:10 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:10.007+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:10.508+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:11.072 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:10.611+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:11.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:10 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:10.976+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:11.801 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:11.509+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:12.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:11.608+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:12.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:11 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:11.997+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:12.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:12 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:12.042+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:12.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:12.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:13.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:12.598+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:13.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:13 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:13.006+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:13.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:13 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:13.004+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:13.298 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:12 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:40:12.897+0000 7fd78ec00700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.894877+0000 front 2024-04-02T17:39:47.895394+0000 (oldest deadline 2024-04-02T17:40:12.595261+0000) 2024-04-02T17:40:13.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:13.492+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:14.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:13.639+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:14.298 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:13 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3[50706]: 2024-04-02T17:40:13.943+0000 7fd78ec00700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.894877+0000 front 2024-04-02T17:39:47.895394+0000 (oldest deadline 2024-04-02T17:40:12.595261+0000) 2024-04-02T17:40:14.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:14 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:14.042+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:14.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:14 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:14.001+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:14.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:14.518+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:15.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:14.624+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:15.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:14.687+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:15.116 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:15 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:15.044+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:15.116 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:14 smithi059 podman[101042]: 2024-04-02 17:40:14.839451641 +0000 UTC m=+5.949598038 container died 983df97c210042a5755f3e5d7749c53e834b32e28b565188805e6fa0dea6f9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:15.116 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:15 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:15.050+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:15.484 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:15 smithi059 podman[101042]: 2024-04-02 17:40:15.242122641 +0000 UTC m=+6.352269031 container cleanup 983df97c210042a5755f3e5d7749c53e834b32e28b565188805e6fa0dea6f9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:15.484 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:15 smithi059 bash[101042]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T17:40:15.797 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:15 smithi059 podman[101071]: 2024-04-02 17:40:15.48508053 +0000 UTC m=+0.638744512 container remove 983df97c210042a5755f3e5d7749c53e834b32e28b565188805e6fa0dea6f9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:15.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:15 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:15.525+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:16.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:15.670+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:16.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:15.692+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:16.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:16 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:16.081+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:16.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:16 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:16.031+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:16.580 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:16 smithi059 podman[101161]: 2024-04-02 17:40:16.210169967 +0000 UTC m=+0.037099589 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:16.580 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:16 smithi059 podman[101161]: 2024-04-02 17:40:16.327472768 +0000 UTC m=+0.154402413 container create 73ffeb1a3c3ebd9a3ea33a3f547d74a2483fc3592e198d0758292c05d5aaf9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T17:40:16.580 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:16 smithi059 podman[101161]: 2024-04-02 17:40:16.580957264 +0000 UTC m=+0.407886915 container init 73ffeb1a3c3ebd9a3ea33a3f547d74a2483fc3592e198d0758292c05d5aaf9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:16.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:16 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:16.545+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:16.983 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:16 smithi059 podman[101161]: 2024-04-02 17:40:16.592811566 +0000 UTC m=+0.419741184 container start 73ffeb1a3c3ebd9a3ea33a3f547d74a2483fc3592e198d0758292c05d5aaf9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T17:40:16.984 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:16 smithi059 podman[101161]: 2024-04-02 17:40:16.712974588 +0000 UTC m=+0.539904214 container attach 73ffeb1a3c3ebd9a3ea33a3f547d74a2483fc3592e198d0758292c05d5aaf9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T17:40:17.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:16.696+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:17.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:16.727+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:17.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:17 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:17.061+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:17.298 INFO:journalctl@ceph.osd.3.smithi059.stdout:Apr 02 17:40:17 smithi059 podman[101161]: 2024-04-02 17:40:17.041300717 +0000 UTC m=+0.868230355 container died 73ffeb1a3c3ebd9a3ea33a3f547d74a2483fc3592e198d0758292c05d5aaf9ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-02T17:40:17.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:16 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:16.983+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:17.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T17:40:17.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:17.816 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T17:40:17.816 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T17:40:17.816 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T17:40:17.847 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:17 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:17.538+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:18.071 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:17.694+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:18.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:17.691+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:18.116 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:17 smithi154 systemd[1]: Stopping Ceph osd.4 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:18.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:18 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:18.017+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:18.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:17 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:17.987+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:18.399 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:18 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:40:18.117+0000 7f2077037700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:18.399 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:18 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:40:18.117+0000 7f2077037700 -1 osd.4 96 *** Got signal Terminated *** 2024-04-02T17:40:18.399 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:18 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:40:18.117+0000 7f2077037700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:18.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:18 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:18.565+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:18.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:18 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:18.565+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:19.070 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:18.651+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:19.070 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:18.673+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:19.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:19 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:19.066+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:19.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:18 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:18.975+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:19.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:19 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:19.528+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:19.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:19 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:19.528+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:20.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:19.712+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:20.072 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:19.615+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:20.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:20 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:20.038+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:20.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:19 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:19.964+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:20.821 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:20 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:20.650+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:20.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:20 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:20.533+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:20.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:20 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:20.670+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:20.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:20 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:20.500+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:20.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:20 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:20.500+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:21.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:20 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:20.984+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:21.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:21.015+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:21.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:21.015+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:21.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:21.534+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:21.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:21.622+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:21.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:21.601+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:21.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:21.601+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:21.899 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:21 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:40:21.626+0000 7f206f623700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.672232+0000 front 2024-04-02T17:39:56.672189+0000 (oldest deadline 2024-04-02T17:40:20.772247+0000) 2024-04-02T17:40:21.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:21 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:21.524+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:21.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:21 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:21.524+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:22.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:22 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:22.044+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:22.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:22 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:22.044+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:22.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:21.961+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:22.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:21 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:21.961+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:22.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:22 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:22.554+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:22.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:22 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:22.609+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:22.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:22 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:22.609+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:22.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:22 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:22.665+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:22.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:22 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:22.665+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:22.898 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:22 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4[47893]: 2024-04-02T17:40:22.648+0000 7f206f623700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.672232+0000 front 2024-04-02T17:39:56.672189+0000 (oldest deadline 2024-04-02T17:40:20.772247+0000) 2024-04-02T17:40:22.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:22 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:22.494+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:22.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:22 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:22.494+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:23.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:23 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:23.067+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:23.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:23 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:23.067+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:23.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:22 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:22.942+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:23.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:22 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:22.942+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:23.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:23 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:23.106+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:23.767 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:23 smithi154 podman[76467]: 2024-04-02 17:40:23.580317957 +0000 UTC m=+5.553717364 container died 24c1c93cfc95760e77948bc7d38bcedc54bf6b16177d2b80ed2bc5d3b5fc648e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T17:40:23.767 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:23 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:23.479+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:23.767 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:23 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:23.479+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:23.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:23 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:23.540+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:23.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:23 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:23.599+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:23.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:23 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:23.599+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:23.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:23 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:23.648+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:23.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:23 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:23.648+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:24.100 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:23 smithi154 podman[76467]: 2024-04-02 17:40:23.850943558 +0000 UTC m=+5.824342962 container cleanup 24c1c93cfc95760e77948bc7d38bcedc54bf6b16177d2b80ed2bc5d3b5fc648e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T17:40:24.100 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:23 smithi154 bash[76467]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T17:40:24.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:24 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:24.042+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:24.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:24 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:24.042+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:24.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:23 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:23.939+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:24.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:23 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:23.939+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:24.395 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76486]: 2024-04-02 17:40:24.102253078 +0000 UTC m=+0.517599292 container remove 24c1c93cfc95760e77948bc7d38bcedc54bf6b16177d2b80ed2bc5d3b5fc648e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:24.395 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:24 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:24.145+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:24.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:24 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:24.452+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:24.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:24 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:24.452+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:24.649 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.300044911 +0000 UTC m=+0.018368949 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:24.649 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.44203673 +0000 UTC m=+0.160360762 container create 8c7b371ed12c1adb3d6c1bd78166b2bfd29552015e048541065669209de978ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326) 2024-04-02T17:40:24.981 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.684077051 +0000 UTC m=+0.402401092 container init 8c7b371ed12c1adb3d6c1bd78166b2bfd29552015e048541065669209de978ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T17:40:24.981 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.68745463 +0000 UTC m=+0.405778669 container start 8c7b371ed12c1adb3d6c1bd78166b2bfd29552015e048541065669209de978ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T17:40:24.981 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.758995721 +0000 UTC m=+0.477319777 container attach 8c7b371ed12c1adb3d6c1bd78166b2bfd29552015e048541065669209de978ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:25.071 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:24 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:24.587+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:25.072 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:24 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:24.581+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:25.072 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:24 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:24.581+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:25.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:24 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:24.602+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:25.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:24 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:24.602+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:25.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:25.035+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:25.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:25.035+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:25.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:24 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:24.962+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:25.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:24 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:24.962+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:25.318 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:25 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:25.140+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:25.318 INFO:journalctl@ceph.osd.4.smithi154.stdout:Apr 02 17:40:24 smithi154 podman[76578]: 2024-04-02 17:40:24.983061922 +0000 UTC m=+0.701385959 container died 8c7b371ed12c1adb3d6c1bd78166b2bfd29552015e048541065669209de978ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:25.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:25 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:25.453+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:25.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:25 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:25.453+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:25.592 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T17:40:25.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:25.676 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T17:40:25.676 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T17:40:25.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T17:40:25.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:25.579+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:25.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:25.538+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:25.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:25.538+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:25.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 systemd[1]: Stopping Ceph osd.5 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:25.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:25.592+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:25.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:25.592+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:26.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:25.953+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:26.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:25.953+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:26.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:26 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:26.040+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:26.299 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:26 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:26.040+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:26.321 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:25.997+0000 7fec4ca3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:26.321 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:25.997+0000 7fec4ca3b700 -1 osd.5 96 *** Got signal Terminated *** 2024-04-02T17:40:26.322 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:25 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:25.997+0000 7fec4ca3b700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:26.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:26 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:26.094+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:26.821 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:26.517+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:26.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:26.517+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:26.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:26.572+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:26.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:26.612+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:26.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:26.612+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:26.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:26 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:26.613+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:26.898 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:26 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:26.452+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:26.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:26 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:26.452+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:27.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:27 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:27.083+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:27.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:27 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:27.083+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:27.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:26 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:26.958+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:27.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:26 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:26.958+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:27.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:27 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:27.102+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:27.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:27 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:27.102+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:27.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:27.545+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:27.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:27.515+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:27.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:27.515+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:27.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:27.515+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:01.396922+0000 front 2024-04-02T17:40:01.396949+0000 (oldest deadline 2024-04-02T17:40:27.297065+0000) 2024-04-02T17:40:27.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:27.636+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:27.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:27.636+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:27.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:27 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:27.636+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:27.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:27 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:27.435+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:27.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:27 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:27.435+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:28.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:28.049+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:28.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:28.049+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:28.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:27 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:27.964+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:28.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:27 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:27.965+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:28.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:28 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:28.121+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:28.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:28 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:28.121+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:28.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:28.513+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:28.498+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:28.498+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:28.498+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:01.396922+0000 front 2024-04-02T17:40:01.396949+0000 (oldest deadline 2024-04-02T17:40:27.297065+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:28.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:28.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:28.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:28 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:28.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:28.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:28 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:28.410+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:28.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:28 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:28.410+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:28.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:28 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:28.410+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:29.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:29.000+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:29.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:29.000+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:29.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:28.949+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:29.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:28.949+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:29.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:28.949+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:29.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:29.128+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:29.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:29.128+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:29.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:29.496+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:29.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:29.449+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:29.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:29.449+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:29.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:29.449+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:01.396922+0000 front 2024-04-02T17:40:01.396949+0000 (oldest deadline 2024-04-02T17:40:27.297065+0000) 2024-04-02T17:40:29.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:29.620+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:29.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:29.620+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:29.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:29 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:29.620+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:29.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:29.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:29.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:29.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:29.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:29 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:29.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:30.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:29.964+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:30.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:29.964+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:30.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:29.985+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:30.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:29.985+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:30.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:29 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:29.985+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:30.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:30.143+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:30.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:30.143+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:30.821 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:30.492+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:45.694422+0000 front 2024-04-02T17:39:45.694807+0000 (oldest deadline 2024-04-02T17:40:09.794437+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:30.492+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.096673+0000 front 2024-04-02T17:39:56.096617+0000 (oldest deadline 2024-04-02T17:40:21.396402+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5[48651]: 2024-04-02T17:40:30.492+0000 7fec45027700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:01.396922+0000 front 2024-04-02T17:40:01.396949+0000 (oldest deadline 2024-04-02T17:40:27.297065+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:30.468+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:30.468+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:30.598+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:30.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:30.598+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:30.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:30 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:30.598+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:30.898 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:30.434+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:30.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:30.434+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:30.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:30 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:30.434+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:31.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:30.999+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:31.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:30.999+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:31.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:30.981+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:31.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:30.981+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:31.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:30 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:30.981+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:31.398 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:31 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:31.119+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:31.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:31 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:31.119+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:31.821 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:31 smithi164 podman[77128]: 2024-04-02 17:40:31.644337181 +0000 UTC m=+5.765846836 container died 35b27f7280b18aa79055366c2761a347f12c865373df52d041016239a45b34ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:31.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:31 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:31.510+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:31.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:31 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:31.510+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:31.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:31 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:31.621+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:31.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:31 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:31.621+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:31.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:31 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:31.621+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:31.898 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:31 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:31.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:31.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:31 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:31.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:31.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:31 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:31.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:32.297 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:31 smithi164 podman[77128]: 2024-04-02 17:40:31.99746052 +0000 UTC m=+6.118970156 container cleanup 35b27f7280b18aa79055366c2761a347f12c865373df52d041016239a45b34ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:32.297 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:31 smithi164 bash[77128]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:31.982+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:31.982+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:31.982+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:31.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:31.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:32.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:31 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:31.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:32.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:32 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:32.145+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:32.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:32 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:32.145+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:32.571 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77145]: 2024-04-02 17:40:32.29910512 +0000 UTC m=+0.658217473 container remove 35b27f7280b18aa79055366c2761a347f12c865373df52d041016239a45b34ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-02T17:40:32.572 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:32 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:32.510+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:32.572 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:32 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:32.510+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:32 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:32.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:32 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:32.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:32 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:32.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77239]: 2024-04-02 17:40:32.496845783 +0000 UTC m=+0.017944953 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77239]: 2024-04-02 17:40:32.616660773 +0000 UTC m=+0.137759931 container create c74c00ff2da18c930b9a0453c2fb515c8b92a3dbe03c8eaa163def246d08464b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T17:40:32.892 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77239]: 2024-04-02 17:40:32.8940229 +0000 UTC m=+0.415122051 container init c74c00ff2da18c930b9a0453c2fb515c8b92a3dbe03c8eaa163def246d08464b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:32.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:32 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:32.442+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:32.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:32 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:32.442+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:32.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:32 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:32.442+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:33.189 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77239]: 2024-04-02 17:40:32.897271496 +0000 UTC m=+0.418370645 container start c74c00ff2da18c930b9a0453c2fb515c8b92a3dbe03c8eaa163def246d08464b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T17:40:33.189 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:32 smithi164 podman[77239]: 2024-04-02 17:40:32.985672074 +0000 UTC m=+0.506771249 container attach c74c00ff2da18c930b9a0453c2fb515c8b92a3dbe03c8eaa163def246d08464b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:32.950+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:32.950+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:32.950+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:32.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:32.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:33.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:32.941+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:33.478 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 02 17:40:33 smithi164 podman[77239]: 2024-04-02 17:40:33.190458992 +0000 UTC m=+0.711558154 container died c74c00ff2da18c930b9a0453c2fb515c8b92a3dbe03c8eaa163def246d08464b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:33.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:33 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:33.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:33.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:33 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:33.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:33.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:33 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:33.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:33.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:33 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:33.170+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:33.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:33 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:33.170+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:33.736 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:33 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:33.519+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:33.737 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:33 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:33.519+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:33.737 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:33 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:33.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:33.737 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:33 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:33.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:33.737 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:33 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:33.605+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:33.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T17:40:33.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:33.801 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T17:40:33.801 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-02T17:40:33.802 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.6 2024-04-02T17:40:34.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:33 smithi059 systemd[1]: Stopping Ceph osd.6 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:34.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:33.958+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:34.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:33.958+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:34.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:33.958+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:34.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:33.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:34.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:33.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:34.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:33 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:33.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:34.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:34 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:34.166+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:34.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:34 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:34.166+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:34.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:34 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:34.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:34.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:34 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:34.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:34.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:34 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:34.405+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:34.798 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.393+0000 7fde76600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:34.798 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.393+0000 7fde76600700 -1 osd.6 96 *** Got signal Terminated *** 2024-04-02T17:40:34.798 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.393+0000 7fde76600700 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:34.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:34 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:34.517+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:34.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:34 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:34.517+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:34.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:34 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:34.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:34.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:34 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:34.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:34.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:34 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:34.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.981+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.981+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:34 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:34.981+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:35.022+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:35.022+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:35.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:35.022+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:35.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:35.157+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:35.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:35.157+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:35.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:35.389+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:35.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:35.389+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:35.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:35 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:35.389+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:35.550+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:35.550+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:35.550+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:35.601+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:35.601+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:35.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:35 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:35.601+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:35.948+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:35.948+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:35.948+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.035+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.035+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:36.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.035+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:36.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.035+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:36.402 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:36.150+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:36.402 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:36.150+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:36.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:36.403+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:36.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:36.403+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:36.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:36 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:36.403+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:37.071 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:36.576+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:37.072 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:36.576+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:37.072 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:36.576+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:37.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:36.643+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:37.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:36.643+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:37.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:36 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:36.643+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:37.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:36.935+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:37.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:36.935+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:37.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:36.935+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:37.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:37.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:37.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:37.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:36 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:36.990+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:37.361+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:37.361+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:37.361+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:37.181+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:37.181+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:37.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:37 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:37.181+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:37.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:37.560+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:37.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:37.560+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:37.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:37.560+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:37.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:37.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:37.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:37.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:37.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:37 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:37.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:38.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:37 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:37.941+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:38.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:37 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:37.941+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:38.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:37 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:37.941+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:38.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:37 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:37.941+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:11.328540+0000 front 2024-04-02T17:40:11.329015+0000 (oldest deadline 2024-04-02T17:40:37.228318+0000) 2024-04-02T17:40:38.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:38.036+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:38.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:38.036+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:38.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:38.036+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:38.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:38.036+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:38.358+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:38.358+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:38.358+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:38.358+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:38.138+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:38.138+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:38.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:38 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:38.138+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:38.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:38.512+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:38.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:38.512+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:38.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:38.512+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:38.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:38.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:38.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:38.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:38.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:38 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:38.638+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:39.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:38.923+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.921809+0000 front 2024-04-02T17:39:47.922205+0000 (oldest deadline 2024-04-02T17:40:09.022097+0000) 2024-04-02T17:40:39.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:38.923+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.424929+0000 front 2024-04-02T17:39:55.424896+0000 (oldest deadline 2024-04-02T17:40:20.724688+0000) 2024-04-02T17:40:39.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:38.923+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:05.427111+0000 front 2024-04-02T17:40:05.427167+0000 (oldest deadline 2024-04-02T17:40:31.327368+0000) 2024-04-02T17:40:39.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:38 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6[57523]: 2024-04-02T17:40:38.923+0000 7fde6ce00700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:11.328540+0000 front 2024-04-02T17:40:11.329015+0000 (oldest deadline 2024-04-02T17:40:37.228318+0000) 2024-04-02T17:40:39.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:39 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:39.002+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:39.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:39 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:39.002+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:39.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:39 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:39.002+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:39.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:39 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:39.002+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:39.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:39.324+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:39.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:39.324+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:39.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:39.324+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:39.399 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:39.324+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:39.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:39.094+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:39.400 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:39.094+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:39.400 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:39 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:39.094+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:39.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:39.478+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:39.478+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:39.478+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:39.663+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:39.663+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:39.663+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:39.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:39 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:39.663+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:40.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:40 smithi059 podman[101266]: 2024-04-02 17:40:40.182013582 +0000 UTC m=+5.911300485 container died 9513ee081eae048bbe6c1499acd287faeb9b9c7c78a4e2af7e3fd234b8c16fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T17:40:40.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:40.046+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:40.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:40.046+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:40.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:40.046+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:40.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:40 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:40.046+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:40.361 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:40.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:40.361 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:40.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:40.361 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:40.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:40.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:40.362+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:40.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:40.362+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:40.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:40.362+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:40.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:40 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:40.362+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:40.798 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:40 smithi059 podman[101266]: 2024-04-02 17:40:40.60998708 +0000 UTC m=+6.339273990 container cleanup 9513ee081eae048bbe6c1499acd287faeb9b9c7c78a4e2af7e3fd234b8c16fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T17:40:40.798 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:40 smithi059 bash[101266]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6 2024-04-02T17:40:40.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:40.446+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:40.446+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:40.446+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:40.646+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:40.646+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:40.646+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:40.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:40 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:40.646+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:41.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:40 smithi059 podman[101286]: 2024-04-02 17:40:40.815928668 +0000 UTC m=+0.627750113 container remove 9513ee081eae048bbe6c1499acd287faeb9b9c7c78a4e2af7e3fd234b8c16fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T17:40:41.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:41.040+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:41.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:41.040+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:41.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:41.040+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:41.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:41 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:41.040+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:41.393 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:41.106+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:41.393 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:41.106+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:41.393 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:41.106+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:41.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:41.394+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:41.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:41.394+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:41.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:41.394+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:41.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:41.394+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:41.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:41.481+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:41.481+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:41.481+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:41.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:41.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:41.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:41.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:41 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:41.610+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:41.898 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:41 smithi059 podman[101376]: 2024-04-02 17:40:41.513343102 +0000 UTC m=+0.045404273 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:41.898 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:41 smithi059 podman[101376]: 2024-04-02 17:40:41.603205718 +0000 UTC m=+0.135266925 container create 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:41.899 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:41 smithi059 podman[101376]: 2024-04-02 17:40:41.898985055 +0000 UTC m=+0.431046269 container init 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:42.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:41 smithi059 podman[101376]: 2024-04-02 17:40:41.91168401 +0000 UTC m=+0.443745224 container start 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T17:40:42.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:41 smithi059 podman[101376]: 2024-04-02 17:40:41.972039014 +0000 UTC m=+0.504100264 container attach 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-02T17:40:42.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:42.088+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:42.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:42.088+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:42.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:42.088+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:42.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:42 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:42.088+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:42.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:42.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:42.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:42.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:42.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:42.060+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:42.689 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:42 smithi059 podman[101376]: 2024-04-02 17:40:42.388018787 +0000 UTC m=+0.920080044 container died 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:42.516+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:42.516+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:42.516+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:42.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:42.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:42.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:42.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:42 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:42.642+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:42.898 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:42.441+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:42.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:42.441+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:42.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:42.441+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:42.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:42 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:42.441+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:43.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.6.service' 2024-04-02T17:40:43.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:43 smithi059 podman[101402]: 2024-04-02 17:40:43.203192639 +0000 UTC m=+0.797490339 container remove 2c7d1d207ed8f26f8f2584bb0edd850d41780e716fb46874e2a54559881b18e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:43 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.6.service: Deactivated successfully. 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:43 smithi059 systemd[1]: Stopped Ceph osd.6 for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.6.smithi059.stdout:Apr 02 17:40:43 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.6.service: Consumed 10min 18.524s CPU time. 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:43.094+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:43.094+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:43.094+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:43.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:43 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:43.094+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:43.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:43.361 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-02T17:40:43.361 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-02T17:40:43.361 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.7 2024-04-02T17:40:43.390 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.073+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:43.390 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.073+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:43.390 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.073+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:43.649 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 systemd[1]: Stopping Ceph osd.7 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:43.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:43.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:43.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:43.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:43.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:43.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:43.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:43.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:43.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:43.508+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:43.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:43.508+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:43.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:43.508+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:43.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:43.679+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:43.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:43.679+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:43.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:43.679+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:43.823 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:43 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:43.680+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.680+0000 7fdeaf0a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.680+0000 7fdeaf0a6700 -1 osd.7 96 *** Got signal Terminated *** 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:43 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:43.680+0000 7fdeaf0a6700 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:44.036+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:44.036+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:44.036+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:44.036 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:44.036+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:18.975964+0000 front 2024-04-02T17:40:18.975913+0000 (oldest deadline 2024-04-02T17:40:43.676075+0000) 2024-04-02T17:40:44.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:44.134+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:44.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:44.134+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:44.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:44.134+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:44.299 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:44 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:44.134+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:44.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:44.525+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:44.525+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:44.525+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:44.695+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:44.695+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:44.695+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:44.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:44 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:44.695+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:44.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:44.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:44.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:44.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:44.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:44.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:44.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:44.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:44.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:44 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:44.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:45.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:45.013+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:45.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:45.013+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:45.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:45.013+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:45.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:45.013+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:18.975964+0000 front 2024-04-02T17:40:18.975913+0000 (oldest deadline 2024-04-02T17:40:43.676075+0000) 2024-04-02T17:40:45.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:45.177+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:45.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:45.177+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:45.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:45.177+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:45.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:45 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:45.177+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:45.500+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:45.500+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:45.500+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:45.737+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:45.737+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:45.737+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:45.822 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:45 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:45.737+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:45.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:45.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:45.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:45.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:45.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:45.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:45.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:45.431+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:45.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:45 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:45.432+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:46.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:46.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:46.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:46.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:46.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:46.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:46.399 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:46.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:18.975964+0000 front 2024-04-02T17:40:18.975913+0000 (oldest deadline 2024-04-02T17:40:43.676075+0000) 2024-04-02T17:40:46.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:46.206+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:46.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:46.206+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:46.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:46.206+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:46.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:46 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:46.206+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:46.769 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:46.457+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:46.770 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:46.457+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:46.770 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:46.457+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:46.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:46.420+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:46.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:46.420+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:46.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:46.420+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:46.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:46.420+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:46.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:46.420+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:47.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:46.770+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:47.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:46.770+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:47.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:46.770+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:47.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:46.770+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:47.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:46 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:46.770+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:47.398 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:47.037+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:47.398 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:47.037+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:47.398 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:47.037+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:47.398 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:47.037+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:18.975964+0000 front 2024-04-02T17:40:18.975913+0000 (oldest deadline 2024-04-02T17:40:43.676075+0000) 2024-04-02T17:40:47.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:47.171+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:47.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:47.171+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:47.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:47.171+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:47.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:47.171+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:47.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:47 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:47.171+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:47.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:47.399+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:47.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:47.399+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:47.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:47.399+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:47.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:47.399+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:47.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:47 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:47.399+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:47.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:47.415+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:47.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:47.415+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:47.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:47.415+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:47.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:47.415+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:48.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:47.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:48.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:47.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:48.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:47.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:48.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:47.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:48.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:47 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:47.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:48.384 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:48.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:56.371733+0000 front 2024-04-02T17:39:56.371797+0000 (oldest deadline 2024-04-02T17:40:22.271949+0000) 2024-04-02T17:40:48.384 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:48.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.272516+0000 front 2024-04-02T17:40:02.272535+0000 (oldest deadline 2024-04-02T17:40:26.972525+0000) 2024-04-02T17:40:48.384 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:48.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.274847+0000 front 2024-04-02T17:40:13.274724+0000 (oldest deadline 2024-04-02T17:40:36.174645+0000) 2024-04-02T17:40:48.384 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7[54502]: 2024-04-02T17:40:48.048+0000 7fdeab69a700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:18.975964+0000 front 2024-04-02T17:40:18.975913+0000 (oldest deadline 2024-04-02T17:40:43.676075+0000) 2024-04-02T17:40:48.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:48.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:48.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:48.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:48.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:48.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:48.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:48.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:48.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:48 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:48.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:48.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:48.385+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:48.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:48.385+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:48.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:48.385+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:48.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:48.385+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:48.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:48 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:48.385+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:48.802 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:48.390+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:48.802 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:48.390+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:48.802 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:48.390+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:48.802 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:48.390+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:49.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:48.802+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:49.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:48.802+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:49.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:48.802+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:49.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:48.802+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:49.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:48 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:48.802+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:49.479 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:49 smithi154 podman[76695]: 2024-04-02 17:40:49.194073641 +0000 UTC m=+5.635732899 container died e32b812c85de2e0808ce92360d822fc20cd280f44ccde7a931e5e9be32df8753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:40:49.479 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:49 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:49.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:49.480 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:49 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:49.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:49.480 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:49 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:49.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:49.480 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:49 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:49.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:49.480 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:49 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:49.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:49.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:49.154+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:49.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:49.154+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:49.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:49.155+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:49.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:49.155+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:49.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:49 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:49.155+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:49.769 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:49.422+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:49.769 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:49.422+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:49.769 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:49.422+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:49.769 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:49.422+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:49.871 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:49 smithi154 podman[76695]: 2024-04-02 17:40:49.580410601 +0000 UTC m=+6.022069860 container cleanup e32b812c85de2e0808ce92360d822fc20cd280f44ccde7a931e5e9be32df8753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:49.871 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:49 smithi154 bash[76695]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7 2024-04-02T17:40:50.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:49.769+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:50.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:49.769+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:50.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:49.769+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:50.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:49.769+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:50.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:49 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:49.769+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:50.149 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:49 smithi154 podman[76712]: 2024-04-02 17:40:49.873473492 +0000 UTC m=+0.683598505 container remove e32b812c85de2e0808ce92360d822fc20cd280f44ccde7a931e5e9be32df8753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T17:40:50.469 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.092286732 +0000 UTC m=+0.018070674 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:50.469 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.207935377 +0000 UTC m=+0.133719296 container create e3126e927b603419c35f778d035b9d4ae644138d8c7083e53bff167bd69b9a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-02T17:40:50.469 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.411143218 +0000 UTC m=+0.336927153 container init e3126e927b603419c35f778d035b9d4ae644138d8c7083e53bff167bd69b9a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T17:40:50.469 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.414565146 +0000 UTC m=+0.340349072 container start e3126e927b603419c35f778d035b9d4ae644138d8c7083e53bff167bd69b9a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T17:40:50.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:50 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:50.203+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:50.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:50 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:50.203+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:50.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:50 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:50.203+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:50.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:50 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:50.203+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:50.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:50 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:50.203+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:50.745 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:50.467+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:50.745 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:50.467+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:50.745 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:50.467+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:50.745 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:50.467+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:50.899 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.477276319 +0000 UTC m=+0.403060251 container attach e3126e927b603419c35f778d035b9d4ae644138d8c7083e53bff167bd69b9a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T17:40:50.899 INFO:journalctl@ceph.osd.7.smithi154.stdout:Apr 02 17:40:50 smithi154 podman[76800]: 2024-04-02 17:40:50.700641611 +0000 UTC m=+0.626425530 container died e3126e927b603419c35f778d035b9d4ae644138d8c7083e53bff167bd69b9a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T17:40:50.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:50 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:50.470+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:50.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:50 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:50.470+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:50.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:50 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:50.470+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:50.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:50 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:50.470+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:50.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:50 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:50.470+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:51.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:50.746+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:51.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:50.746+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:51.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:50.746+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:51.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:50.746+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:51.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:50 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:50.746+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:51.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.7.service' 2024-04-02T17:40:51.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:51.321 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-02T17:40:51.321 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-02T17:40:51.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.8 2024-04-02T17:40:51.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:51.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:51.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:51.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:51.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:51.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:51.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:51.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:51.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:51 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:51.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:51.631 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 systemd[1]: Stopping Ceph osd.8 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:40:51.631 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.420+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:51.631 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.421+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:51.631 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.421+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:51.631 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.421+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:51.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:51.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:51.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:51.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:51.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:51.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:51.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:51.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:51.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:51.454+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:52.071 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.631+0000 7fb8dc988700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:40:52.071 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.631+0000 7fb8dc988700 -1 osd.8 96 *** Got signal Terminated *** 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:51.631+0000 7fb8dc988700 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:51.792+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:51.792+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:51.792+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:51.792+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:52.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:51 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:51.792+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:52.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:52.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:52.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:52.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:52.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:52.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:52.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:52.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:52.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:52 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:52.174+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:52.767 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:52.409+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:52.767 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:52.409+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:52.767 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:52.409+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:52.767 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:52.409+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:52.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:52.447+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:52.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:52.447+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:52.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:52.447+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:52.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:52.447+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:52.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:52 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:52.447+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:53.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:53.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:53.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:53.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:53.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:53.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:52 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:52.768+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:53.410 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:53.142+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:53.410 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:53.142+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:53.410 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:53.142+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:53.410 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:53.142+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:53.411 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:53 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:53.142+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:53.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:53.443+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:53.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:53.443+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:53.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:53.443+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:53.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:53.443+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:53.775 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:53.443+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:26.770183+0000 front 2024-04-02T17:40:26.770210+0000 (oldest deadline 2024-04-02T17:40:52.570724+0000) 2024-04-02T17:40:53.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:53 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:53.460+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:53.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:53 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:53.460+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:53.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:53 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:53.460+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:53.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:53 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:53.460+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:53.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:53 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:53.460+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:54.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:54.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:54.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:54.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:54.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:54.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:53 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:53.776+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:54.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:54.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:54.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:54.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:54.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:54.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:54 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:54.122+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:54.773 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:54.439+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:54.773 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:54.439+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:54.773 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:54.439+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:54.773 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:54.439+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:54.773 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:54.439+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:26.770183+0000 front 2024-04-02T17:40:26.770210+0000 (oldest deadline 2024-04-02T17:40:52.570724+0000) 2024-04-02T17:40:54.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:54 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:54.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:54.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:54 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:54.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:54.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:54 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:54.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:54.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:54 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:54.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:54.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:54 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:54.472+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:55.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:55.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:55.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:55.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:55.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:55.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:54 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:54.774+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:55.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:55.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:55.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:55.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:55.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:55.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:55 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:55.162+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:55.811 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:55.405+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:55.811 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:55.405+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:55.811 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:55.405+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:55.811 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:55.405+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:55.811 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:55.405+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:26.770183+0000 front 2024-04-02T17:40:26.770210+0000 (oldest deadline 2024-04-02T17:40:52.570724+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:55.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:55.521+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:40:56.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:56.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:56.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:56.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:56.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:56.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:55 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:55.811+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:56.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:56 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:56.118+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:56.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:56.363+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:57.165001+0000 front 2024-04-02T17:39:57.164914+0000 (oldest deadline 2024-04-02T17:40:20.064947+0000) 2024-04-02T17:40:56.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:56.363+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:02.365858+0000 front 2024-04-02T17:40:02.365999+0000 (oldest deadline 2024-04-02T17:40:29.966522+0000) 2024-04-02T17:40:56.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:56.363+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:10.467619+0000 front 2024-04-02T17:40:10.467809+0000 (oldest deadline 2024-04-02T17:40:35.167497+0000) 2024-04-02T17:40:56.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:56.363+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:21.470185+0000 front 2024-04-02T17:40:21.470096+0000 (oldest deadline 2024-04-02T17:40:46.769907+0000) 2024-04-02T17:40:56.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8[55564]: 2024-04-02T17:40:56.363+0000 7fb8d8f7c700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:26.770183+0000 front 2024-04-02T17:40:26.770210+0000 (oldest deadline 2024-04-02T17:40:52.570724+0000) 2024-04-02T17:40:56.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:56.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:56.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:56.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:56.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:56.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:56 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:56.497+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:40:57.243 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:57.243 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:57.244 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:57.244 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:57.244 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:57.244 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:56 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:56.859+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:57.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:57 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:57.137+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:57.556 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:57 smithi164 podman[77347]: 2024-04-02 17:40:57.245238258 +0000 UTC m=+5.728145546 container died 0b8f701355e11ce5f86b3da1d1f9648bde05ebd22686ff6f41697a62fdf6bf68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:57.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:57 smithi164 podman[77347]: 2024-04-02 17:40:57.657467378 +0000 UTC m=+6.140374655 container cleanup 0b8f701355e11ce5f86b3da1d1f9648bde05ebd22686ff6f41697a62fdf6bf68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:40:57.821 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:57 smithi164 bash[77347]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:57.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:57 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:57.471+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:40:58.272 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:57 smithi164 podman[77366]: 2024-04-02 17:40:57.883586545 +0000 UTC m=+0.642004466 container remove 0b8f701355e11ce5f86b3da1d1f9648bde05ebd22686ff6f41697a62fdf6bf68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T17:40:58.272 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:58.272 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:58.272 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:58.272 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:58.273 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:58.273 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:57 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:57.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:58.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:58 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:58.133+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:58.568 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.177198898 +0000 UTC m=+0.017877624 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:40:58.568 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.275856739 +0000 UTC m=+0.116535464 container create c4e99c09a73e58cd084b3e4cb01a85910fd629f3299be44912edab9626798cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:40:58.568 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.503450652 +0000 UTC m=+0.344129380 container init c4e99c09a73e58cd084b3e4cb01a85910fd629f3299be44912edab9626798cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-02T17:40:58.568 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.506461096 +0000 UTC m=+0.347139814 container start c4e99c09a73e58cd084b3e4cb01a85910fd629f3299be44912edab9626798cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T17:40:58.568 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.570001548 +0000 UTC m=+0.410680293 container attach c4e99c09a73e58cd084b3e4cb01a85910fd629f3299be44912edab9626798cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T17:40:58.822 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 02 17:40:58 smithi164 podman[77458]: 2024-04-02 17:40:58.800316946 +0000 UTC m=+0.640995673 container died c4e99c09a73e58cd084b3e4cb01a85910fd629f3299be44912edab9626798cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:58.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:58 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:58.516+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:40:59.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:58 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:58.884+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:40:59.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.8.service' 2024-04-02T17:40:59.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:40:59.479 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-02T17:40:59.479 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-02T17:40:59.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.9 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:40:59.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:40:59.156+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:40:59.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:40:59 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:40:59.507+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:00.048 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:40:59 smithi059 systemd[1]: Stopping Ceph osd.9 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:00.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:00.071 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:00.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:00.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:00.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:00.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:40:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:40:59.879+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:00.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.086+0000 7f163d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:00.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.086+0000 7f163d000700 -1 osd.9 96 *** Got signal Terminated *** 2024-04-02T17:41:00.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.086+0000 7f163d000700 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:41:00.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:00 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:00.193+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:00.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:00 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:00.487+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:01.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:01.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:01.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:01.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:01.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:01.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:00 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:00.917+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:01.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:41:01.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:41:01.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:41:01.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:41:01.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:41:01.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:01 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:01.186+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:01.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:01.650 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:01 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:01.489+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:02.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:01 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:01.890+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:41:02.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:02 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:02.231+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:02.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:02.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:02 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:02.463+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:03.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:03.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:03.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:03.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:03.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:03.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:02 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:02.905+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:41:03.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:41:03.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:03 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:03.183+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:36.877028+0000 front 2024-04-02T17:40:36.876943+0000 (oldest deadline 2024-04-02T17:41:02.677648+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:03.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:03 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:03.465+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:04.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:03.938+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:04.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:47.053800+0000 front 2024-04-02T17:39:47.054002+0000 (oldest deadline 2024-04-02T17:40:11.153430+0000) 2024-04-02T17:41:04.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:55.855764+0000 front 2024-04-02T17:39:55.855874+0000 (oldest deadline 2024-04-02T17:40:21.756017+0000) 2024-04-02T17:41:04.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.559987+0000 front 2024-04-02T17:40:04.559754+0000 (oldest deadline 2024-04-02T17:40:28.659940+0000) 2024-04-02T17:41:04.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:13.763962+0000 front 2024-04-02T17:40:13.764167+0000 (oldest deadline 2024-04-02T17:40:35.464245+0000) 2024-04-02T17:41:04.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:22.970673+0000 front 2024-04-02T17:40:22.970504+0000 (oldest deadline 2024-04-02T17:40:46.470771+0000) 2024-04-02T17:41:04.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.773301+0000 front 2024-04-02T17:40:28.773174+0000 (oldest deadline 2024-04-02T17:40:54.073440+0000) 2024-04-02T17:41:04.549 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:04 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9[64551]: 2024-04-02T17:41:04.168+0000 7f1633800700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:36.877028+0000 front 2024-04-02T17:40:36.876943+0000 (oldest deadline 2024-04-02T17:41:02.677648+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:04.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:04 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:04.469+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:05.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:04 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:04.968+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:05.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:05 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:05.436+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:06.189 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:05 smithi059 podman[101491]: 2024-04-02 17:41:05.920050466 +0000 UTC m=+5.961346090 container died 120e32f7acade8504cc34313fa97e69b8da4a05938b7ef4fb08ca867b6854972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-02T17:41:06.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:06.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:06.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:06.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:06.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:06.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:05 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:05.928+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:06.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:06 smithi059 podman[101491]: 2024-04-02 17:41:06.302131512 +0000 UTC m=+6.343427176 container cleanup 120e32f7acade8504cc34313fa97e69b8da4a05938b7ef4fb08ca867b6854972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T17:41:06.548 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:06 smithi059 bash[101491]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:06.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:06 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:06.430+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:07.047 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:06 smithi059 podman[101508]: 2024-04-02 17:41:06.594572147 +0000 UTC m=+0.662405740 container remove 120e32f7acade8504cc34313fa97e69b8da4a05938b7ef4fb08ca867b6854972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:07.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:06 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:06.963+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:07.798 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:07 smithi059 podman[101598]: 2024-04-02 17:41:07.437302768 +0000 UTC m=+0.124521758 container create c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T17:41:07.798 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:07 smithi059 podman[101598]: 2024-04-02 17:41:07.350817019 +0000 UTC m=+0.038035994 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:07.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:07 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:07.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:08.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:07 smithi059 podman[101598]: 2024-04-02 17:41:07.834738643 +0000 UTC m=+0.521957709 container init c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-04-02T17:41:08.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:07 smithi059 podman[101598]: 2024-04-02 17:41:07.847040179 +0000 UTC m=+0.534259215 container start c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:41:08.298 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:07 smithi059 podman[101598]: 2024-04-02 17:41:07.922773427 +0000 UTC m=+0.609992423 container attach c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:41:08.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:08.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:08.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:08.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:08.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:08.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:07.956+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:08.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:08 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:08.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:08.666 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:08 smithi059 podman[101598]: 2024-04-02 17:41:08.299984969 +0000 UTC m=+0.987204005 container died c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326) 2024-04-02T17:41:09.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.9.service' 2024-04-02T17:41:09.231 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:09 smithi059 podman[101624]: 2024-04-02 17:41:09.109180248 +0000 UTC m=+0.792916353 container remove c4a0f3ad9f682e7838e431aef9bf4c9c3f2e1aa6137368c9914ebf58a4d78d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T17:41:09.231 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:09 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.9.service: Deactivated successfully. 2024-04-02T17:41:09.231 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:09 smithi059 systemd[1]: Stopped Ceph osd.9 for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:41:09.231 INFO:journalctl@ceph.osd.9.smithi059.stdout:Apr 02 17:41:09 smithi059 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.9.service: Consumed 9min 14.275s CPU time. 2024-04-02T17:41:09.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:09.270 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-02T17:41:09.270 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-02T17:41:09.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.10 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:09.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:08 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:08.990+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:09.576 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 systemd[1]: Stopping Ceph osd.10 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:09.576 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:09.576 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.387+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.576+0000 7f6ab23ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.576+0000 7f6ab23ff700 -1 osd.10 96 *** Got signal Terminated *** 2024-04-02T17:41:09.577 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:09 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:09.576+0000 7f6ab23ff700 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:41:10.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:10.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:10.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:10.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:10.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:10.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:09 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:09.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:10.899 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:10 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:10.406+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:11.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:11.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:11.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:11.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:11.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:11.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:10 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:10.993+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:11.800 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:11.800 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:11.800 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:11.801 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:11.801 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:11.801 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:11.801 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:11 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:11.418+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:12.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:12.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:12.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:12.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:12.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:12.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:12 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:12.040+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.374+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.374+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.374+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.374+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.375+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.375+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:12.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:12 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:12.375+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:13.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:13.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:13.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:13.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:13.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:13.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.009+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:13.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:13 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:13.379+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:14.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:13 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:13.979+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:46.630451+0000 front 2024-04-02T17:39:46.630938+0000 (oldest deadline 2024-04-02T17:40:09.530354+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.131748+0000 front 2024-04-02T17:39:54.131688+0000 (oldest deadline 2024-04-02T17:40:18.231865+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.633582+0000 front 2024-04-02T17:40:04.633415+0000 (oldest deadline 2024-04-02T17:40:28.133371+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.035388+0000 front 2024-04-02T17:40:12.035287+0000 (oldest deadline 2024-04-02T17:40:37.935009+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:17.935413+0000 front 2024-04-02T17:40:17.935385+0000 (oldest deadline 2024-04-02T17:40:44.335934+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:28.937441+0000 front 2024-04-02T17:40:28.937383+0000 (oldest deadline 2024-04-02T17:40:54.837345+0000) 2024-04-02T17:41:14.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:14 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10[61132]: 2024-04-02T17:41:14.392+0000 7f6aaa9eb700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-04-02T17:40:34.838009+0000 front 2024-04-02T17:40:34.838107+0000 (oldest deadline 2024-04-02T17:41:00.737754+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:15.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:14 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:14.980+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:15.340 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 podman[76908]: 2024-04-02 17:41:15.02957115 +0000 UTC m=+5.562660715 container died 8d74e0cffc0cc935f4c9c980ab6c09a093f47ca78bc23d6e520b6ed89cdba41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T17:41:15.340 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 podman[76908]: 2024-04-02 17:41:15.342024316 +0000 UTC m=+5.875113903 container cleanup 8d74e0cffc0cc935f4c9c980ab6c09a093f47ca78bc23d6e520b6ed89cdba41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T17:41:15.592 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 bash[76908]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10 2024-04-02T17:41:15.893 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 podman[76924]: 2024-04-02 17:41:15.59430539 +0000 UTC m=+0.563774297 container remove 8d74e0cffc0cc935f4c9c980ab6c09a093f47ca78bc23d6e520b6ed89cdba41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-02T17:41:16.149 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 podman[77018]: 2024-04-02 17:41:15.798478253 +0000 UTC m=+0.018177685 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:41:16.149 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:15 smithi154 podman[77018]: 2024-04-02 17:41:15.912051416 +0000 UTC m=+0.131750889 container create b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T17:41:16.149 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 podman[77018]: 2024-04-02 17:41:16.130955599 +0000 UTC m=+0.350655037 container init b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T17:41:16.149 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 podman[77018]: 2024-04-02 17:41:16.134030494 +0000 UTC m=+0.353729913 container start b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:16.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:15 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:15.964+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:16.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 podman[77018]: 2024-04-02 17:41:16.230910032 +0000 UTC m=+0.450609471 container attach b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T17:41:16.649 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 podman[77018]: 2024-04-02 17:41:16.422619935 +0000 UTC m=+0.642319362 container died b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T17:41:16.967 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.10.service' 2024-04-02T17:41:16.993 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 podman[77044]: 2024-04-02 17:41:16.942970924 +0000 UTC m=+0.510758745 container remove b01c9a62d0342a7c121aa3a4bb51ed48a4aa11ec7f6351fe47e20ebb8ef2e940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T17:41:16.993 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.10.service: Deactivated successfully. 2024-04-02T17:41:16.993 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 systemd[1]: Stopped Ceph osd.10 for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:41:16.993 INFO:journalctl@ceph.osd.10.smithi154.stdout:Apr 02 17:41:16 smithi154 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.10.service: Consumed 8min 41.739s CPU time. 2024-04-02T17:41:17.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:17.008 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-02T17:41:17.008 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-02T17:41:17.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.11 2024-04-02T17:41:17.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:16 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:16.953+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:17.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 systemd[1]: Stopping Ceph osd.11 for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:17.821 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.342+0000 7f3cabed7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:17.821 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.342+0000 7f3cabed7700 -1 osd.11 96 *** Got signal Terminated *** 2024-04-02T17:41:17.821 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.342+0000 7f3cabed7700 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T17:41:18.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:18.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:17 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:17.978+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:19.223 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:19.224 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:18 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:18.961+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:20.241 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:20.241 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:20.241 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:20.242 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:20.242 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:20.242 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:20.242 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:19 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:19.969+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:21.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.008+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-02T17:40:55.519632+0000 front 2024-04-02T17:40:55.519685+0000 (oldest deadline 2024-04-02T17:41:20.219694+0000) 2024-04-02T17:41:22.321 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-04-02T17:39:43.207822+0000 front 2024-04-02T17:39:43.207750+0000 (oldest deadline 2024-04-02T17:40:14.408128+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-04-02T17:39:54.408839+0000 front 2024-04-02T17:39:54.408639+0000 (oldest deadline 2024-04-02T17:40:22.609260+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-02T17:40:04.310647+0000 front 2024-04-02T17:40:04.310737+0000 (oldest deadline 2024-04-02T17:40:26.610766+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-04-02T17:40:12.412662+0000 front 2024-04-02T17:40:12.412560+0000 (oldest deadline 2024-04-02T17:40:38.813215+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-04-02T17:40:20.514154+0000 front 2024-04-02T17:40:20.514143+0000 (oldest deadline 2024-04-02T17:40:45.814035+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-02T17:40:29.315186+0000 front 2024-04-02T17:40:29.315247+0000 (oldest deadline 2024-04-02T17:40:52.215286+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-04-02T17:40:47.918728+0000 front 2024-04-02T17:40:47.918710+0000 (oldest deadline 2024-04-02T17:41:13.218763+0000) 2024-04-02T17:41:22.322 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:21 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11[62196]: 2024-04-02T17:41:21.981+0000 7f3ca44c3700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-02T17:40:55.519632+0000 front 2024-04-02T17:40:55.519685+0000 (oldest deadline 2024-04-02T17:41:20.219694+0000) 2024-04-02T17:41:23.115 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:22 smithi164 podman[77574]: 2024-04-02 17:41:22.827175498 +0000 UTC m=+5.623869444 container died 14621066e8bf7cc4fcd2976ce1d8c0d02e375652d6a2e84a1c4862ae6be1e3ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T17:41:23.115 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:23 smithi164 podman[77574]: 2024-04-02 17:41:23.117048001 +0000 UTC m=+5.913741965 container cleanup 14621066e8bf7cc4fcd2976ce1d8c0d02e375652d6a2e84a1c4862ae6be1e3ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T17:41:23.571 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:23 smithi164 bash[77574]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11 2024-04-02T17:41:23.571 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:23 smithi164 podman[77603]: 2024-04-02 17:41:23.342908156 +0000 UTC m=+0.509532845 container remove 14621066e8bf7cc4fcd2976ce1d8c0d02e375652d6a2e84a1c4862ae6be1e3ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T17:41:24.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:23 smithi164 podman[77696]: 2024-04-02 17:41:23.690391336 +0000 UTC m=+0.018104547 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T17:41:24.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:23 smithi164 podman[77696]: 2024-04-02 17:41:23.802223098 +0000 UTC m=+0.129936311 container create f2bfce75b918fbd5e4e265de34718fb1b38f4d2684996aac9d33e06ec48621e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T17:41:24.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:24 smithi164 podman[77696]: 2024-04-02 17:41:24.029799982 +0000 UTC m=+0.357513202 container init f2bfce75b918fbd5e4e265de34718fb1b38f4d2684996aac9d33e06ec48621e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:41:24.072 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:24 smithi164 podman[77696]: 2024-04-02 17:41:24.033159715 +0000 UTC m=+0.360872921 container start f2bfce75b918fbd5e4e265de34718fb1b38f4d2684996aac9d33e06ec48621e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T17:41:24.571 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:24 smithi164 podman[77696]: 2024-04-02 17:41:24.11276922 +0000 UTC m=+0.440482454 container attach f2bfce75b918fbd5e4e265de34718fb1b38f4d2684996aac9d33e06ec48621e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:41:24.571 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 02 17:41:24 smithi164 podman[77696]: 2024-04-02 17:41:24.32844841 +0000 UTC m=+0.656161620 container died f2bfce75b918fbd5e4e265de34718fb1b38f4d2684996aac9d33e06ec48621e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T17:41:24.886 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@osd.11.service' 2024-04-02T17:41:24.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:24.929 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-02T17:41:24.930 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-02T17:41:24.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.a 2024-04-02T17:41:25.298 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:25 smithi059 systemd[1]: Stopping Ceph mds.a for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:25.797 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a[71483]: 2024-04-02T17:41:25.492+0000 7f6d87800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:25.798 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:25 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a[71483]: 2024-04-02T17:41:25.492+0000 7f6d87800700 -1 mds.a *** got signal Terminated *** 2024-04-02T17:41:27.423 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:27 smithi059 podman[101718]: 2024-04-02 17:41:27.133826563 +0000 UTC m=+1.755439712 container died 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T17:41:27.702 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:27 smithi059 podman[101718]: 2024-04-02 17:41:27.535930196 +0000 UTC m=+2.157543368 container cleanup 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T17:41:27.702 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:27 smithi059 bash[101718]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a 2024-04-02T17:41:28.048 INFO:journalctl@ceph.mds.a.smithi059.stdout:Apr 02 17:41:27 smithi059 podman[101732]: 2024-04-02 17:41:27.70336498 +0000 UTC m=+0.552459796 container remove 323fd44a3977a0615b3c44145f826e820560ca3fb17a626554056f8c49a4d9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-a, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:41:28.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.a.service' 2024-04-02T17:41:28.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:28.331 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-02T17:41:28.331 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-02T17:41:28.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.d 2024-04-02T17:41:28.797 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:28 smithi059 systemd[1]: Stopping Ceph mds.d for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:29.298 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: 2024-04-02T17:41:28.961+0000 7f4728000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:29.298 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:28 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d[70491]: 2024-04-02T17:41:28.961+0000 7f4728000700 -1 mds.d *** got signal Terminated *** 2024-04-02T17:41:30.431 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:30 smithi059 podman[101837]: 2024-04-02 17:41:30.159794827 +0000 UTC m=+1.329973093 container died 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T17:41:30.798 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:30 smithi059 podman[101837]: 2024-04-02 17:41:30.547576403 +0000 UTC m=+1.717754679 container cleanup 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T17:41:30.798 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:30 smithi059 bash[101837]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d 2024-04-02T17:41:31.298 INFO:journalctl@ceph.mds.d.smithi059.stdout:Apr 02 17:41:30 smithi059 podman[101851]: 2024-04-02 17:41:30.940367669 +0000 UTC m=+0.766310090 container remove 0f1ece883dc81d2252a54740f378263f21c08e28bcdeb53afad787dd22555317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T17:41:31.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.d.service' 2024-04-02T17:41:31.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:31.603 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-02T17:41:31.603 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-02T17:41:31.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.g 2024-04-02T17:41:32.047 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:31 smithi059 systemd[1]: Stopping Ceph mds.g for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:32.548 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:41:32.207+0000 7f32b6000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:32.548 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:32 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g[71135]: 2024-04-02T17:41:32.207+0000 7f32b6000700 -1 mds.g *** got signal Terminated *** 2024-04-02T17:41:34.414 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:34 smithi059 podman[101958]: 2024-04-02 17:41:34.14924183 +0000 UTC m=+2.058936685 container died 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T17:41:34.702 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:34 smithi059 podman[101958]: 2024-04-02 17:41:34.526503046 +0000 UTC m=+2.436197901 container cleanup 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T17:41:34.702 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:34 smithi059 bash[101958]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g 2024-04-02T17:41:35.047 INFO:journalctl@ceph.mds.g.smithi059.stdout:Apr 02 17:41:34 smithi059 podman[101972]: 2024-04-02 17:41:34.702365454 +0000 UTC m=+0.535371665 container remove 609691d8ea2273c15c76951ef1cec276b628edea7c9f3c0ed92b09669447073a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T17:41:35.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.g.service' 2024-04-02T17:41:35.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:35.388 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-02T17:41:35.388 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-02T17:41:35.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.j 2024-04-02T17:41:35.797 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:35 smithi059 systemd[1]: Stopping Ceph mds.j for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:36.298 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j[70144]: 2024-04-02T17:41:35.993+0000 7fe0c7800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:36.298 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:35 smithi059 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j[70144]: 2024-04-02T17:41:35.993+0000 7fe0c7800700 -1 mds.j *** got signal Terminated *** 2024-04-02T17:41:40.298 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:40 smithi059 podman[102076]: 2024-04-02 17:41:40.030970715 +0000 UTC m=+4.195655123 container died d0735e66ce624e88849361b2e50c9e5fff7d887b533e7f31429c897b6e3daa11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T17:41:40.578 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:40 smithi059 podman[102076]: 2024-04-02 17:41:40.437858751 +0000 UTC m=+4.602543202 container cleanup d0735e66ce624e88849361b2e50c9e5fff7d887b533e7f31429c897b6e3daa11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T17:41:40.579 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:40 smithi059 bash[102076]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j 2024-04-02T17:41:41.047 INFO:journalctl@ceph.mds.j.smithi059.stdout:Apr 02 17:41:40 smithi059 podman[102090]: 2024-04-02 17:41:40.580171917 +0000 UTC m=+0.534669523 container remove d0735e66ce624e88849361b2e50c9e5fff7d887b533e7f31429c897b6e3daa11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-j, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T17:41:41.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.j.service' 2024-04-02T17:41:41.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:41.271 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-02T17:41:41.271 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-02T17:41:41.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.b 2024-04-02T17:41:41.574 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:41 smithi154 systemd[1]: Stopping Ceph mds.b for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:41.836 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: 2024-04-02T17:41:41.575+0000 7f7331a09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:41.836 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:41 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b[66817]: 2024-04-02T17:41:41.575+0000 7f7331a09700 -1 mds.b *** got signal Terminated *** 2024-04-02T17:41:45.857 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:45 smithi154 podman[77123]: 2024-04-02 17:41:45.586174976 +0000 UTC m=+4.119844078 container died 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T17:41:46.133 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:45 smithi154 podman[77123]: 2024-04-02 17:41:45.933829413 +0000 UTC m=+4.467498534 container cleanup 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T17:41:46.133 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:45 smithi154 bash[77123]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b 2024-04-02T17:41:46.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.b.service' 2024-04-02T17:41:46.383 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:46 smithi154 podman[77149]: 2024-04-02 17:41:46.134890135 +0000 UTC m=+0.543238711 container remove 7104f6875f3a1799a313b62123f9bce3d06a63ac08cb01db38ab146058f1f8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T17:41:46.383 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:46 smithi154 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.b.service: Deactivated successfully. 2024-04-02T17:41:46.383 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:46 smithi154 systemd[1]: Stopped Ceph mds.b for 8286be62-f112-11ee-b647-cb9ed24678a4. 2024-04-02T17:41:46.383 INFO:journalctl@ceph.mds.b.smithi154.stdout:Apr 02 17:41:46 smithi154 systemd[1]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.b.service: Consumed 16min 18.887s CPU time. 2024-04-02T17:41:46.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:46.397 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-02T17:41:46.397 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-02T17:41:46.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.e 2024-04-02T17:41:46.649 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:46 smithi154 systemd[1]: Stopping Ceph mds.e for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:47.148 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: 2024-04-02T17:41:46.737+0000 7f701af97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:47.149 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:46 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e[66180]: 2024-04-02T17:41:46.737+0000 7f701af97700 -1 mds.e *** got signal Terminated *** 2024-04-02T17:41:50.399 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:50 smithi154 podman[77256]: 2024-04-02 17:41:50.129258074 +0000 UTC m=+3.510375897 container died cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T17:41:50.802 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:50 smithi154 podman[77256]: 2024-04-02 17:41:50.487613558 +0000 UTC m=+3.868731375 container cleanup cedc45bfbd01fec8ac010fd9345bfe82e099bfacb4e8492a1d7641286d140391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T17:41:50.803 INFO:journalctl@ceph.mds.e.smithi154.stdout:Apr 02 17:41:50 smithi154 bash[77256]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-e 2024-04-02T17:41:50.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.e.service' 2024-04-02T17:41:51.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:51.007 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-02T17:41:51.007 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-02T17:41:51.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.h 2024-04-02T17:41:51.331 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:51 smithi154 systemd[1]: Stopping Ceph mds.h for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:51.648 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: 2024-04-02T17:41:51.332+0000 7fbe6f351700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:51.649 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:51 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h[66490]: 2024-04-02T17:41:51.332+0000 7fbe6f351700 -1 mds.h *** got signal Terminated *** 2024-04-02T17:41:54.381 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:54 smithi154 podman[77378]: 2024-04-02 17:41:54.127195861 +0000 UTC m=+2.904666369 container died c5f867ee1cc4ce6cc9bedda3d07a9404099cac71086041ebbaae22349d406ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T17:41:54.381 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:54 smithi154 podman[77378]: 2024-04-02 17:41:54.382567641 +0000 UTC m=+3.160038140 container cleanup c5f867ee1cc4ce6cc9bedda3d07a9404099cac71086041ebbaae22349d406ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T17:41:54.649 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:54 smithi154 bash[77378]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h 2024-04-02T17:41:54.649 INFO:journalctl@ceph.mds.h.smithi154.stdout:Apr 02 17:41:54 smithi154 podman[77392]: 2024-04-02 17:41:54.542166719 +0000 UTC m=+0.413447400 container remove c5f867ee1cc4ce6cc9bedda3d07a9404099cac71086041ebbaae22349d406ee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-h, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS) 2024-04-02T17:41:54.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.h.service' 2024-04-02T17:41:54.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:54.791 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-02T17:41:54.791 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-02T17:41:54.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.k 2024-04-02T17:41:55.101 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:54 smithi154 systemd[1]: Stopping Ceph mds.k for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:55.398 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k[65856]: 2024-04-02T17:41:55.102+0000 7f9fcc0ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:55.399 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:55 smithi154 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k[65856]: 2024-04-02T17:41:55.102+0000 7f9fcc0ff700 -1 mds.k *** got signal Terminated *** 2024-04-02T17:41:58.399 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:58 smithi154 podman[77499]: 2024-04-02 17:41:58.132036647 +0000 UTC m=+3.124451450 container died 54a6c83cd357b181da5202990439492bb6f6a7e6ac506ec78801440394d45ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T17:41:58.669 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:58 smithi154 podman[77499]: 2024-04-02 17:41:58.486481912 +0000 UTC m=+3.478896658 container cleanup 54a6c83cd357b181da5202990439492bb6f6a7e6ac506ec78801440394d45ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T17:41:58.669 INFO:journalctl@ceph.mds.k.smithi154.stdout:Apr 02 17:41:58 smithi154 bash[77499]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-k 2024-04-02T17:41:58.871 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.k.service' 2024-04-02T17:41:58.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:41:58.906 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-02T17:41:58.906 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-02T17:41:58.907 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.c 2024-04-02T17:41:59.263 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:41:58 smithi164 systemd[1]: Stopping Ceph mds.c for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:41:59.530 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:41:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: 2024-04-02T17:41:59.264+0000 7fc7e60c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:41:59.531 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:41:59 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c[67231]: 2024-04-02T17:41:59.264+0000 7fc7e60c9700 -1 mds.c *** got signal Terminated *** 2024-04-02T17:42:02.423 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:42:02 smithi164 podman[77802]: 2024-04-02 17:42:02.12747418 +0000 UTC m=+3.027240215 container died 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T17:42:02.772 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:42:02 smithi164 podman[77802]: 2024-04-02 17:42:02.5231382 +0000 UTC m=+3.422904224 container cleanup 6e81fcef5a121fa5e619d8cb50f663b81eca2a5a613c84db63b3f5bc4c22eaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True) 2024-04-02T17:42:02.772 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 02 17:42:02 smithi164 bash[77802]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-c 2024-04-02T17:42:02.905 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.c.service' 2024-04-02T17:42:02.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:42:02.945 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-02T17:42:02.945 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-02T17:42:02.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.f 2024-04-02T17:42:03.321 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:03 smithi164 systemd[1]: Stopping Ceph mds.f for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:42:03.321 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: 2024-04-02T17:42:03.285+0000 7fcc8dba5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:42:03.321 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:03 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f[66923]: 2024-04-02T17:42:03.285+0000 7fcc8dba5700 -1 mds.f *** got signal Terminated *** 2024-04-02T17:42:06.376 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:06 smithi164 podman[77936]: 2024-04-02 17:42:06.123304315 +0000 UTC m=+2.940267455 container died 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T17:42:06.759 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:06 smithi164 podman[77936]: 2024-04-02 17:42:06.477003414 +0000 UTC m=+3.293966555 container cleanup 8297056f5b349709672633baf65b71d291d587be66a2a2d02be373c3c8be72de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-02T17:42:06.759 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 02 17:42:06 smithi164 bash[77936]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-f 2024-04-02T17:42:06.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.f.service' 2024-04-02T17:42:06.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:42:06.947 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-02T17:42:06.948 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-02T17:42:06.948 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.i 2024-04-02T17:42:07.287 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:07 smithi164 systemd[1]: Stopping Ceph mds.i for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:42:07.572 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i[67538]: 2024-04-02T17:42:07.288+0000 7ffa43737700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:42:07.572 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:07 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i[67538]: 2024-04-02T17:42:07.288+0000 7ffa43737700 -1 mds.i *** got signal Terminated *** 2024-04-02T17:42:10.321 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:10 smithi164 podman[78058]: 2024-04-02 17:42:10.126460623 +0000 UTC m=+2.945450583 container died 994940dd911fc5f0e5b924daf6dfb8acd59279806b09fa8414e0860d0b2c3734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T17:42:10.746 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:10 smithi164 podman[78058]: 2024-04-02 17:42:10.555034667 +0000 UTC m=+3.374024612 container cleanup 994940dd911fc5f0e5b924daf6dfb8acd59279806b09fa8414e0860d0b2c3734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T17:42:10.746 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 02 17:42:10 smithi164 bash[78058]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-i 2024-04-02T17:42:10.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.i.service' 2024-04-02T17:42:10.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:42:10.999 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-02T17:42:11.000 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-02T17:42:11.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.l 2024-04-02T17:42:11.321 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:11 smithi164 systemd[1]: Stopping Ceph mds.l for 8286be62-f112-11ee-b647-cb9ed24678a4... 2024-04-02T17:42:11.821 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: 2024-04-02T17:42:11.325+0000 7f4297725700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T17:42:11.821 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:11 smithi164 ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l[66609]: 2024-04-02T17:42:11.325+0000 7f4297725700 -1 mds.l *** got signal Terminated *** 2024-04-02T17:42:14.484 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:14 smithi164 podman[78179]: 2024-04-02 17:42:14.126171922 +0000 UTC m=+2.905091534 container died 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T17:42:14.783 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:14 smithi164 podman[78179]: 2024-04-02 17:42:14.585796702 +0000 UTC m=+3.364716311 container cleanup 48b62ec8ce0b9a37154f50ed42a1cbbdd953f8e246aaa92fe093c34958e6f32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T17:42:14.783 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 02 17:42:14 smithi164 bash[78179]: ceph-8286be62-f112-11ee-b647-cb9ed24678a4-mds-l 2024-04-02T17:42:15.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8286be62-f112-11ee-b647-cb9ed24678a4@mds.l.service' 2024-04-02T17:42:15.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T17:42:15.038 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-02T17:42:15.038 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T17:42:15.384 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:42:20.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T17:42:20.703 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:42:23.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T17:42:23.579 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T17:42:26.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:42:26.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:42:26.320 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T17:42:26.348 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T17:42:26.350 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi059/crash 2024-04-02T17:42:26.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T17:42:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T17:42:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-04-02T17:42:26.466 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi154/crash 2024-04-02T17:42:26.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T17:42:26.538 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T17:42:26.538 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-04-02T17:42:26.540 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi164/crash 2024-04-02T17:42:26.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T17:42:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T17:42:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-04-02T17:42:26.582 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T17:42:26.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T17:42:26.742 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-02T17:14:45.695976+0000 mds.g (mds.1) 2 : cluster [WRN] Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/fe2f5ae4-3a17-4f52-aafe-ca5caee1cbd8/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.g log and `damage ls` output for details 2024-04-02T17:42:26.743 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-02T17:42:26.743 INFO:tasks.cephadm:Compressing logs... 2024-04-02T17:42:26.743 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T17:42:26.748 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T17:42:26.751 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.10.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.b.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T17:42:26.800 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.k.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.h.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.7.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.e.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.y.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T17:42:26.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.b.log 2024-04-02T17:42:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.b.log: 93.4% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-02T17:42:26.820 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T17:42:26.822 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T17:42:26.823 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T17:42:26.823 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T17:42:26.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.l.log 2024-04-02T17:42:26.824 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T17:42:26.824 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.i.log 2024-04-02T17:42:26.824 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.11.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.z.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.8.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.f.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.8.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T17:42:26.827 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T17:42:26.828 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.c.log 2024-04-02T17:42:26.828 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T17:42:26.828 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.2.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.c.log 2024-04-02T17:42:26.830 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.c.log: 93.5% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.z.log.gz 2024-04-02T17:42:26.832 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.6.log 2024-04-02T17:42:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.a.log 2024-04-02T17:42:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T17:42:26.835 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.g.log 2024-04-02T17:42:26.835 INFO:teuthology.orchestra.run.smithi164.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T17:42:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.j.log 2024-04-02T17:42:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T17:42:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.x.log 2024-04-02T17:42:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T17:42:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T17:42:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T17:42:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T17:42:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.9.log 2024-04-02T17:42:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.d.log 2024-04-02T17:42:26.844 INFO:teuthology.orchestra.run.smithi059.stderr: 94.4% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.j.log.gz 2024-04-02T17:42:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.a.log 2024-04-02T17:42:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.9.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.d.log: /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.a.log: 86.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T17:42:26.852 INFO:teuthology.orchestra.run.smithi059.stderr: 94.4% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-02T17:42:26.927 INFO:teuthology.orchestra.run.smithi059.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T17:42:26.963 INFO:teuthology.orchestra.run.smithi164.stderr: 83.1% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T17:42:26.972 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T17:42:26.990 INFO:teuthology.orchestra.run.smithi059.stderr: 83.1% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T17:42:27.006 INFO:teuthology.orchestra.run.smithi154.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T17:42:27.009 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T17:42:27.140 INFO:teuthology.orchestra.run.smithi164.stderr: 90.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T17:42:27.150 INFO:teuthology.orchestra.run.smithi154.stderr: 83.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T17:42:27.192 INFO:teuthology.orchestra.run.smithi154.stderr: 90.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T17:42:27.253 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T17:42:27.254 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T17:42:27.432 INFO:teuthology.orchestra.run.smithi164.stderr: 86.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T17:42:27.528 INFO:teuthology.orchestra.run.smithi154.stderr: 86.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T17:42:28.248 INFO:teuthology.orchestra.run.smithi059.stderr: 90.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-02T17:42:36.081 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-02T17:42:39.755 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-02T17:42:54.907 INFO:teuthology.orchestra.run.smithi059.stderr: 92.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-02T17:43:40.809 INFO:teuthology.orchestra.run.smithi059.stderr: 95.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.d.log.gz 2024-04-02T17:44:10.639 INFO:teuthology.orchestra.run.smithi059.stderr: 93.5% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T17:44:12.237 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.9.log.gz 2024-04-02T17:44:19.738 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-02T17:44:27.345 INFO:teuthology.orchestra.run.smithi059.stderr: 93.5% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T17:44:45.117 INFO:teuthology.orchestra.run.smithi154.stderr: 95.3% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.e.log.gz 2024-04-02T17:44:56.641 INFO:teuthology.orchestra.run.smithi164.stderr: 95.1% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.f.log.gz 2024-04-02T17:45:06.395 INFO:teuthology.orchestra.run.smithi164.stderr: 95.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.l.log.gz 2024-04-02T17:45:08.084 INFO:teuthology.orchestra.run.smithi154.stderr: 95.3% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.k.log.gz 2024-04-02T17:45:53.153 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T17:46:02.071 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.11.log.gz 2024-04-02T17:46:16.820 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T17:46:36.174 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T17:46:40.606 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.8.log.gz 2024-04-02T17:47:40.639 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.10.log.gz 2024-04-02T17:47:50.617 INFO:teuthology.orchestra.run.smithi059.stderr: 93.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.g.log.gz 2024-04-02T17:47:50.622 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-02T17:47:50.622 INFO:teuthology.orchestra.run.smithi059.stderr:real 5m23.837s 2024-04-02T17:47:50.622 INFO:teuthology.orchestra.run.smithi059.stderr:user 6m24.646s 2024-04-02T17:47:50.622 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m43.126s 2024-04-02T17:48:14.767 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-02T18:01:41.125 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.c.log.gz 2024-04-02T18:01:56.351 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.i.log.gz 2024-04-02T18:01:56.379 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-02T18:01:56.379 INFO:teuthology.orchestra.run.smithi164.stderr:real 19m29.610s 2024-04-02T18:01:56.379 INFO:teuthology.orchestra.run.smithi164.stderr:user 8m19.036s 2024-04-02T18:01:56.379 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m23.035s 2024-04-02T18:05:32.156 INFO:teuthology.orchestra.run.smithi154.stderr: 93.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.h.log.gz 2024-04-02T18:06:04.404 INFO:teuthology.orchestra.run.smithi154.stderr: 93.2% -- replaced with /var/log/ceph/8286be62-f112-11ee-b647-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-02T18:06:04.422 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-04-02T18:06:04.422 INFO:teuthology.orchestra.run.smithi154.stderr:real 23m37.657s 2024-04-02T18:06:04.422 INFO:teuthology.orchestra.run.smithi154.stderr:user 9m5.846s 2024-04-02T18:06:04.422 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m26.230s 2024-04-02T18:06:04.423 INFO:tasks.cephadm:Archiving logs... 2024-04-02T18:06:04.423 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi059/log 2024-04-02T18:06:04.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T18:06:49.044 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi154/log 2024-04-02T18:06:49.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T18:08:14.706 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi164/log 2024-04-02T18:08:14.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T18:09:32.258 INFO:tasks.cephadm:Removing cluster... 2024-04-02T18:09:32.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force 2024-04-02T18:09:32.627 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T18:09:34.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force 2024-04-02T18:09:34.619 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T18:09:35.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8286be62-f112-11ee-b647-cb9ed24678a4 --force 2024-04-02T18:09:35.915 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 8286be62-f112-11ee-b647-cb9ed24678a4 2024-04-02T18:09:36.608 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T18:09:36.608 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T18:09:36.670 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T18:09:36.688 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T18:09:36.711 INFO:tasks.cephadm:Teardown complete 2024-04-02T18:09:36.711 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T18:09:36.723 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T18:09:36.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T18:09:36.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T18:09:36.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T18:09:36.892 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T18:09:36.903 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T18:09:36.903 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T18:09:36.911 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T18:09:36.914 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T18:09:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-02T18:09:36.928 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-02T18:09:36.960 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-02T18:09:36.971 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T18:09:36.971 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-02T18:09:36.971 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 -451us[ -446us] +/- 94ms 2024-04-02T18:09:36.971 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -76us[ -71us] +/- 65ms 2024-04-02T18:09:36.971 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +516us[ +521us] +/- 86ms 2024-04-02T18:09:36.972 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T18:09:37.030 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T18:09:37.031 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-02T18:09:37.031 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 -446us[ -446us] +/- 94ms 2024-04-02T18:09:37.031 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -86us[ -86us] +/- 65ms 2024-04-02T18:09:37.031 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +519us[ +519us] +/- 86ms 2024-04-02T18:09:37.031 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 -442us[ -422us] +/- 94ms 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -72us[ -51us] +/- 65ms 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +511us[ +531us] +/- 86ms 2024-04-02T18:09:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T18:09:37.047 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T18:09:37.060 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T18:09:37.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T18:09:37.071 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T18:09:37.369 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T18:09:37.513 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T18:09:37.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T18:09:37.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:37.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T18:09:37.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:37.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T18:09:37.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:37.804 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T18:09:37.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T18:09:37.826 INFO:teuthology.task.internal:Duration was 4656.264458 seconds 2024-04-02T18:09:37.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T18:09:37.837 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T18:09:37.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T18:09:37.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T18:09:37.849 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T18:09:37.914 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T18:09:37.922 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T18:09:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T18:09:38.437 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T18:09:38.437 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-04-02T18:09:38.437 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T18:09:38.505 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-04-02T18:09:38.505 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T18:09:38.533 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-02T18:09:38.533 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T18:09:38.564 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T18:09:38.564 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T18:09:38.572 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T18:09:38.575 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T18:09:38.668 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T18:09:38.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T18:09:38.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T18:09:38.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T18:09:39.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T18:09:39.231 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T18:09:39.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T18:09:39.334 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T18:09:39.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T18:09:39.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T18:09:39.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.449 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.484 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-02T18:09:39.485 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-04-02T18:09:39.585 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-04-02T18:09:39.643 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:39.692 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:39.706 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T18:09:39.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T18:09:39.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T18:09:39.731 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T18:09:39.733 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi059 2024-04-02T18:09:39.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T18:09:39.861 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi154 2024-04-02T18:09:39.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T18:09:39.900 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741/remote/smithi164 2024-04-02T18:09:39.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T18:09:39.940 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T18:09:39.940 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T18:09:39.949 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T18:09:39.951 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T18:09:40.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T18:09:40.016 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T18:09:40.016 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T18:09:40.028 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T18:09:40.028 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T18:09:40.036 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T18:09:40.038 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T18:09:40.050 INFO:teuthology.orchestra.run.smithi154.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 18:09 /home/ubuntu/cephtest 2024-04-02T18:09:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 18:09 /home/ubuntu/cephtest 2024-04-02T18:09:40.087 INFO:teuthology.orchestra.run.smithi059.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 18:09 /home/ubuntu/cephtest 2024-04-02T18:09:40.091 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T18:09:40.103 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T18:09:40.128 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T18:09:40.148 DEBUG:teuthology.nuke:shortname: smithi059 2024-04-02T18:09:40.148 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T18:09:40.169 DEBUG:teuthology.nuke:shortname: smithi154 2024-04-02T18:09:40.169 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T18:09:40.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.765305', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0PBAFqV/Q22FEAfo6IvUERCi2g8DxBjtVmbyEoVG7xzNjjbh67HeZUODTlnjDs3icS9VS/LMa1QCtyUwxPKKQ='} 2024-04-02T18:09:40.188 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-02T18:09:40.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T18:09:40.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.768812', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+/LCXaxbHmi1QtlK4diCXeI/IX5gDL6Yy5CK5vt8JAPhdyq4EI2J0W71d/5hyVwo02vwVVRhNL2+aWFP09Uxs='} 2024-04-02T18:09:40.215 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635741', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:43:05.767155', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGuWeq6VyF3aYT05Vlzd42xoDvvsas+dXzRTTJMnkWUPDepYSRShSaPWtogStPbRk8aODJ+PVTtS5dPjqUaaLHA='} 2024-04-02T18:09:40.226 INFO:teuthology.orchestra.console.smithi059:Power off 2024-04-02T18:09:40.226 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T18:09:40.250 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-04-02T18:09:40.250 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:40.267 INFO:teuthology.orchestra.console.smithi154:Power off 2024-04-02T18:09:40.267 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T18:09:40.283 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-02T18:09:40.294 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-04-02T18:09:40.294 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:40.309 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-02T18:09:40.309 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T18:09:40.320 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-02T18:09:40.331 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-02T18:09:40.332 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:40.356 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-02T18:09:44.284 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:44.321 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:44.357 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:44.397 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-02T18:09:44.433 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-02T18:09:44.471 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-02T18:09:48.401 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:48.434 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:48.472 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:48.512 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-02T18:09:48.549 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-02T18:09:48.582 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-02T18:09:52.514 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:52.551 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:52.583 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T18:09:52.625 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-04-02T18:09:52.661 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-04-02T18:09:52.694 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-02T18:09:52.726 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-04-02T18:09:52.762 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-04-02T18:09:52.794 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-02T18:09:52.900 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 4656.264457702637 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-04-02T18:09:52.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T18:09:52.981 INFO:teuthology.run:FAIL